Real analysis and argmax

In this post I give a proof of an exercise of real analysis that has found an application to statistical mode estimation (see the Remark below). I wish to thank gerard0, remarque, and La vieille for their contribution.

Proposition. Let f be an upper-semicontinuous map from [a, b] (a < b) to \mathbb{R}_+. Suppose that there is a unique \theta \in \mathrm{arg max} f (f(\theta) = \sup_{x \in [a, b]} f(x)). Then for every continuous map \psi : [a, b] \mapsto \mathbb{R} we have

\displaystyle \frac{\int_a^b f(t)^n \psi(t) \, dt}{\int_a^b f(t)^n \, dt} \rightarrow_{n \rightarrow \infty} \psi(\theta).

Proof. First remark that the uniqueness of \theta implies that f is not identically zero. We denote by u_n the sequence at stake. We are looking for an upper bound of

\displaystyle u_n - \psi(\theta) = \frac{\int_a^b f(t)^n (\psi(t) - \psi(\theta)) \, dt}{\int_a^b f(t)^n \, dt}.

So let \epsilon > 0. The map \psi is continuous (at \theta) so there exists an open interval I containing \theta such that | \psi(t) - \psi(\theta) | \leq \epsilon for all t \in I. Defining M = \sup_{t \in [a, b]} \psi(t) we obtain

\displaystyle | u_n - \psi(\theta) | \leq \epsilon + 2 M \frac{\int_J f(t)^n \, dt}{\int_a^b f(t)^n \, dt},

where J denotes the compact subset [a, b] \setminus I. It remains to show that the ratio \frac{\int_J f(t)^n \, dt}{\int_a^b f(t)^n \, dt} tends to 0. But it is a well known result (see e.g. here for a proof relying on Chebyshev’s inequality) that

\displaystyle \left(\int_a^b f(t)^n \, dt \right)^{1/n} \rightarrow \sup_{t \in [a, b]} f(t) = f(\theta),

and similarly

\displaystyle \left(\int_J f(t)^n \, dt \right)^{1/n} \rightarrow \sup_{t \in J} f(t) =: f(\theta_1).

(The existence of \theta_1 is ensured by the upper-semicontinuity of f and the compactness of J.) This implies that

\displaystyle \log \frac{\int_J f(t)^n \, dt}{\int_a^b f(t)^n \, dt} \sim n \log \alpha,

where \alpha is the ratio f(\theta_1)/f(\theta), which is strictly less than 1 by uniqueness of \theta. Thus, n \log \alpha tends to -\infty, and so does \log \frac{\int_J f(t)^n \, dt}{\int_a^b f(t)^n \, dt}. This shows that \frac{\int_J f(t)^n \, dt}{\int_a^b f(t)^n \, dt} tends to 0, and concludes the proof.

Remark. Suppose that f is actually an upper-semicontinuous probability density with a unique mode \theta. Taking \psi : t \mapsto t the previous result states that

\displaystyle \theta = \lim_{n \rightarrow \infty} E[X_n],

where, for each n, X_n is a random variable with density f^n / \int f(t)^n \, dt. This result is at the core of the mode estimator proposed by Grenander (1965).

References.

Grenander, Ulf (1965). Some direct estimates of the mode. Ann. Math. Statist. 36:131-138

Advertisements