Chernoff's distribution

In probability theory, Chernoff's distribution, named after Herman Chernoff, is the probability distribution of the random variable

Z={\underset {s\in \mathbf {R} }{\operatorname {argmax} }}\ (W(s)-s^{2}),

where W is a "two-sided" Wiener process (or two-sided "Brownian motion") satisfying W(0) = 0. If

V(a,c)={\underset {s\in \mathbf {R} }{\operatorname {argmax} }}\ (W(s)-c(s-a)^{2}),

then V(0, c) has density

f_{c}(t)={\frac {1}{2}}g_{c}(t)g_{c}(-t)

where g_c has Fourier transform given by

{\hat {g}}_{c}(s)={\frac {(2/c)^{1/3}}{\operatorname {Ai} (i(2c^{2})^{-1/3}s)}},\ \ \ s\in \mathbf {R}

and where Ai is the Airy function. Thus f_c is symmetric about 0 and the density ƒ_Z = ƒ₁. Groeneboom (1989)^[1] shows that

f_{Z}(z)\sim {\frac {1}{2}}{\frac {4^{4/3}|z|}{\operatorname {Ai} '({\tilde {a}}_{1})}}\exp \left(-{\frac {2}{3}}|z|^{3}+2^{1/3}{\tilde {a}}_{1}|z|\right){\text{ as }}z\rightarrow \infty

where ${\tilde {a}}_{1}\approx -2.3381$ is the largest zero of the Airy function Ai and where $\operatorname {Ai} '({\tilde {a}}_{1})\approx 0.7022$ . In the same paper, Groeneboom also gives an analysis of the process $\{V(a,1):a\in \mathbf {R} \}$ . The connection with the statistical problem of estimating a monotone density is discussed in Groeneboom (1985).^[2] Chernoff's distribution is now known to appear in a wide range of monotone problems including isotonic regression.^[3]

The Chernoff distribution should not be confused with the Chernoff geometric distribution^[4] (called the Chernoff point in information geometry) induced by the Chernoff information.

History

Groeneboom, Lalley and Temme^[5] state that the first investigation of this distribution was probably by Chernoff in 1964,^[6] who studied the behavior of a certain estimator of a mode. In his paper, Chernoff characterized the distribution through an analytic representation through the heat equation with suitable boundary conditions. Initial attempts at approximating Chernoff's distribution via solving the heat equation, however, did not achieve satisfactory precision due to the nature of the boundary conditions.^[5] The computation of the distribution is addressed, for example, in Groeneboom and Wellner (2001).^[7]

The connection of Chernoff's distribution with Airy functions was also found independently by Daniels and Skyrme^[8] and Temme,^[9] as cited in Groeneboom, Lalley and Temme. These two papers, along with Groeneboom (1989), were all written in 1984.^[5]

References

^ Groeneboom, Piet (1989). "Brownian motion with a parabolic drift and Airy functions". Probability Theory and Related Fields. 81: 79–109. doi:10.1007/BF00343738. MR 0981568. S2CID 119980629.
^ Groeneboom, Piet (1985). Le Cam, L.E.; Olshen, R. A. (eds.). Estimating a monotone density. Proceedings of the Berkeley conference in honor of Jerzy Neyman and Jack Kiefer, vol. II. pp. 539–555.
^ Groeneboom, Piet; Jongbloed, Geurt (2018). "Some Developments in the Theory of Shape Constrained Inference". Statistical Science. 33 (4): 473–492. doi:10.1214/18-STS657. S2CID 13672538.
^ Nielsen, Frank (2022). "Revisiting Chernoff Information with Likelihood Ratio Exponential Families". Entropy. MDPI. 24 (10): 1400. doi:10.3390/e24101400. PMC 9601539. PMID 37420420.
^ ^a ^b ^c Groeneboom, Piet; Lalley, Steven; Temme, Nico (2015). "Chernoff's distribution and differential equations of parabolic and Airy type". Journal of Mathematical Analysis and Applications. 423 (2): 1804–1824. arXiv:1305.6053. doi:10.1016/j.jmaa.2014.10.051. MR 3278229. S2CID 119173815.
^ Chernoff, Herman (1964). "Estimation of the mode". Annals of the Institute of Statistical Mathematics. 16: 31–41. doi:10.1007/BF02868560. MR 0172382. S2CID 121030566.
^ Groeneboom, Piet; Wellner, Jon A. (2001). "Computing Chernoff's Distribution". Journal of Computational and Graphical Statistics. 10 (2): 388–400. CiteSeerX 10.1.1.369.863. doi:10.1198/10618600152627997. MR 1939706. S2CID 6573960.
^ Daniels, H.E.; Skyrme, T.H.R. (1985). "The maximum of a random walk whose mean path has a maximum". Advances in Applied Probability. 17 (1): 85–99. doi:10.2307/1427054. JSTOR 1427054. MR 0778595. S2CID 124603511.
^ Temme, N.M. (1985). "A convolution integral equation solved by Laplace transformations" (PDF). Journal of Computational and Applied Mathematics. 12–13: 609–613. doi:10.1016/0377-0427(85)90052-4. MR 0793989. S2CID 120496241.

Probability distributions (list)

Discrete
univariate

with finite support	Benford Bernoulli beta-binomial binomial categorical hypergeometric negative Poisson binomial Rademacher soliton discrete uniform Zipf Zipf–Mandelbrot
with infinite support	beta negative binomial Borel Conway–Maxwell–Poisson discrete phase-type Delaporte extended negative binomial Flory–Schulz Gauss–Kuzmin geometric logarithmic mixed Poisson negative binomial Panjer parabolic fractal Poisson Skellam Yule–Simon zeta

Continuous
univariate

supported on a bounded interval	arcsine ARGUS Balding–Nichols Bates beta beta rectangular continuous Bernoulli Irwin–Hall Kumaraswamy logit-normal noncentral beta PERT raised cosine reciprocal triangular U-quadratic uniform Wigner semicircle
supported on a semi-infinite interval	Benini Benktander 1st kind Benktander 2nd kind beta prime Burr chi chi-squared noncentral inverse scaled Dagum Davis Erlang hyper exponential hyperexponential hypoexponential logarithmic F noncentral folded normal Fréchet gamma generalized inverse gamma/Gompertz Gompertz shifted half-logistic half-normal Hotelling's T-squared inverse Gaussian generalized Kolmogorov Lévy log-Cauchy log-Laplace log-logistic log-normal log-t Lomax matrix-exponential Maxwell–Boltzmann Maxwell–Jüttner Mittag-Leffler Nakagami Pareto phase-type Poly-Weibull Rayleigh relativistic Breit–Wigner Rice truncated normal type-2 Gumbel Weibull discrete Wilks's lambda
supported on the whole real line	Cauchy exponential power Fisher's z Kaniadakis κ-Gaussian Gaussian q generalized normal generalized hyperbolic geometric stable Gumbel Holtsmark hyperbolic secant Johnson's S_U Landau Laplace asymmetric logistic noncentral t normal (Gaussian) normal-inverse Gaussian skew normal slash stable Student's t Tracy–Widom variance-gamma Voigt
with support whose type varies	generalized chi-squared generalized extreme value generalized Pareto Marchenko–Pastur Kaniadakis κ-exponential Kaniadakis κ-Gamma Kaniadakis κ-Weibull Kaniadakis κ-Logistic Kaniadakis κ-Erlang q-exponential q-Gaussian q-Weibull shifted log-logistic Tukey lambda