【Stats300A】L4

From Data Reduction to Risk Reduction

凸函数，严格凸函数定义（ $f (γ x + (1 - γ) y) \leq γ f (x) + (1 - γ) f (y)$ ）
Jensen’s Inequality
If $f : C \to R$ is convex on an open set $C$ , and $E (X)$ exists, then

f (E (X)) \leq E (f (X))

Rao-Blackwell 定理

Rao-Blackwell Theorem

Assume that $T$ is a sufficient statistic.
Assume that we have a loss function $L (θ, d)$ which is strictly convex in $d$ , and that $δ (X)$ is an estimator of $g (θ)$ with finite risk $R (θ, δ)$
Let $η (t) = E [δ (X) | T (X) = t]$ , then

R (θ, η) < R (θ, δ), \forall θ

unless $δ = η$ with probability 1 (i.e. $δ$ was a function of $T$ )

如果损失函数 $L$ 为仅为凸函数（非严格凸），则 $R (θ, η) \leq R (θ, δ)$
Rao-Blackwell 定理给出了一个改进充分统计量的方法
统计量应该依赖于充分统计 $T$ ，否则一定可以改进

UMVU Estimators

U-estimable

Let $X \sim P_{θ}, θ \in Ω$ , $g (θ)$ real valued. $g (θ)$ is U-estimable if $\exists δ (X), E_{θ} [δ (X)] = g (θ)$ for all $θ$ .
(That is, $g (θ)$ has an unbiased estimator.)

Rao-Blackwell 定理在无偏估计下推论
假设 $T$ 是 $g (θ)$ 充分无偏估计，则 $η (t) = E [δ (X) | T (X) = t]$ 也是无偏估计，且 $V a r_{θ} η (T) \leq V a r_{θ} δ (X)$

无偏估计下的 Rao-Blackwell 定理证明

由全期望公式，可知该统计量无偏

E_{θ} {η (T)} = E_{θ} [E {δ (X) ∣ T}] = E_{θ} {δ (X)} = g (θ)

下面证明方差减小

\begin{aligned} V a r_{θ} {δ} & = E_{θ} {δ - E_{θ} δ}^{2} \\ = E_{θ} {δ - E_{θ} (δ | T) + E_{θ} (δ | T) - E_{θ} δ}^{2} \\ = E_{θ} {δ - η (T)}^{2} + V a r_{θ} {η (T)} + 2 E_{θ} {[δ - E_{θ} (δ | T)] \cdot [E_{θ} (δ | T) - E_{θ} δ]} \end{aligned}

下证 $E_{θ} {[δ - E_{θ} (δ | T)] \cdot [E_{θ} (δ | T) - E_{θ} η]} \geq 0$ 即可得证 $V a r_{θ} δ (X) \geq V a r_{θ} η (T)$

\begin{aligned} E_{θ} {[δ - E_{θ} (δ ∣ T)] [E_{θ} (δ ∣ T) - E_{θ} (δ)]} \\ = E_{T} (E_{θ} {[δ - E_{θ} (δ ∣ T)] [E_{θ} (δ ∣ T) - E_{θ} (δ)] ∣ T}) \\ = E_{T} (E_{θ} {[δ - E_{θ} (δ ∣ T)] ∣ T} \cdot [E_{θ} (δ ∣ T) - E_{θ} (δ)]) \\ = E_{T} ([E_{θ} (δ ∣ T) - E_{θ} (δ ∣ T)] \cdot [E_{θ} (δ ∣ T) - E_{θ} (δ)]) = 0 \end{aligned}

主要通过第一个等式利用全期望公式，定理得证^[1]

或者可直接证 $E {(η - θ)^{2}} \leq E {(δ - θ)^{2}}$ ^[2]，由于 ( $E (X^{2}) \leq E (X)^{2}$ )

\begin{aligned} E {(η - θ)^{2}} & = E {(E [δ ∣ T] - θ)^{2}} \\ = E {E [(δ - θ) ∣ T]^{2}} \\ \leq E {E [(δ - θ)^{2} ∣ T]} = E {(δ - θ)^{2}} \end{aligned}

尽管不一定存在一致最优估计，但可以通过增加限制使得该限制下最优，比如下述 UMRUE，UMVUE 即考虑无偏估计下 uniformly minimize risk

UMVU, UMRU estimator

An estimator $δ^{*}$ is UMVU (uniformly minimum variance unbiasedestimator) or UMRU (uniformly minimum risk unbiased estimator) if for any other unbiased estimator $δ$ ,

R (θ, η) \leq R (θ, δ), \forall θ

for all $θ$ . It is uniquely UMVU if the above inequality is strict for some $θ$ .

Lehmann-Scheffe Theorem

admissable