Real analysis and argmax

In this post I give a proof of an exercise of real analysis that has found an application to statistical mode estimation (see the Remark below). I wish to thank gerard0, remarque, and La vieille for their contribution.

Proposition. Let $f$ be an upper-semicontinuous map from $[a, b]$ ($a < b$) to $\mathbb{R}_+$. Suppose that there is a unique $\theta \in \mathrm{arg max} f$ ($f(\theta) = \sup_{x \in [a, b]} f(x)$). Then for every continuous map $\psi : [a, b] \mapsto \mathbb{R}$ we have

$\displaystyle \frac{\int_a^b f(t)^n \psi(t) \, dt}{\int_a^b f(t)^n \, dt} \rightarrow_{n \rightarrow \infty} \psi(\theta)$.

Proof. First remark that the uniqueness of $\theta$ implies that $f$ is not identically zero. We denote by $u_n$ the sequence at stake. We are looking for an upper bound of

$\displaystyle u_n - \psi(\theta) = \frac{\int_a^b f(t)^n (\psi(t) - \psi(\theta)) \, dt}{\int_a^b f(t)^n \, dt}$.

So let $\epsilon > 0$. The map $\psi$ is continuous (at $\theta$) so there exists an open interval $I$ containing $\theta$ such that $| \psi(t) - \psi(\theta) | \leq \epsilon$ for all $t \in I$. Defining $M = \sup_{t \in [a, b]} \psi(t)$ we obtain

$\displaystyle | u_n - \psi(\theta) | \leq \epsilon + 2 M \frac{\int_J f(t)^n \, dt}{\int_a^b f(t)^n \, dt}$,

where $J$ denotes the compact subset $[a, b] \setminus I$. It remains to show that the ratio $\frac{\int_J f(t)^n \, dt}{\int_a^b f(t)^n \, dt}$ tends to $0$. But it is a well known result (see e.g. here for a proof relying on Chebyshev’s inequality) that

$\displaystyle \left(\int_a^b f(t)^n \, dt \right)^{1/n} \rightarrow \sup_{t \in [a, b]} f(t) = f(\theta)$,

and similarly

$\displaystyle \left(\int_J f(t)^n \, dt \right)^{1/n} \rightarrow \sup_{t \in J} f(t) =: f(\theta_1)$.

(The existence of $\theta_1$ is ensured by the upper-semicontinuity of $f$ and the compactness of $J$.) This implies that

$\displaystyle \log \frac{\int_J f(t)^n \, dt}{\int_a^b f(t)^n \, dt} \sim n \log \alpha$,

where $\alpha$ is the ratio $f(\theta_1)/f(\theta)$, which is strictly less than $1$ by uniqueness of $\theta$. Thus, $n \log \alpha$ tends to $-\infty$, and so does $\log \frac{\int_J f(t)^n \, dt}{\int_a^b f(t)^n \, dt}$. This shows that $\frac{\int_J f(t)^n \, dt}{\int_a^b f(t)^n \, dt}$ tends to $0$, and concludes the proof.

Remark. Suppose that $f$ is actually an upper-semicontinuous probability density with a unique mode $\theta$. Taking $\psi : t \mapsto t$ the previous result states that

$\displaystyle \theta = \lim_{n \rightarrow \infty} E[X_n]$,

where, for each $n$, $X_n$ is a random variable with density $f^n / \int f(t)^n \, dt$. This result is at the core of the mode estimator proposed by Grenander (1965).

References.

Grenander, Ulf (1965). Some direct estimates of the mode. Ann. Math. Statist. 36:131-138