Basu’s theorem – Wikipedia

In statistics, Basu’s theorem states that any boundedly complete minimal sufficient statistic is independent of any ancillary statistic. This is a 1955 result of Debabrata Basu.^[1]

It is often used in statistics as a tool to prove independence of two statistics, by first demonstrating one is complete sufficient and the other is ancillary, then appealing to the theorem.^[2] An example of this is to show that the sample mean and sample variance of a normal distribution are independent statistics, which is done in the Example section below. This property (independence of sample mean and sample variance) characterizes normal distributions.

Statement[edit]

Let

${displaystyle (P_{theta };theta in Theta )}$

be a family of distributions on a measurable space

${displaystyle (X,{mathcal {A}})}$

and

${displaystyle T,A}$

measurable maps from

${displaystyle (X,{mathcal {A}})}$

to some measurable space

${displaystyle (Y,{mathcal {B}})}$

. (Such maps are called a statistic.) If

${displaystyle T}$

is a boundedly complete sufficient statistic for

${displaystyle theta }$

, and

${displaystyle A}$

is ancillary to

${displaystyle theta }$

, then conditional on

${displaystyle theta }$

,

${displaystyle T}$

is independent of

${displaystyle A}$

. That is,

${displaystyle Tperp A|theta }$

.

Proof[edit]

Let

${displaystyle P_{theta }^{T}}$

and

${displaystyle P_{theta }^{A}}$

be the marginal distributions of

${displaystyle T}$

and

${displaystyle A}$

respectively.

Denote by

${displaystyle A^{-1}(B)}$

the preimage of a set

${displaystyle B}$

under the map

${displaystyle A}$

. For any measurable set

${displaystyle Bin {mathcal {B}}}$

we have

${displaystyle P_{theta }^{A}(B)=P_{theta }(A^{-1}(B))=int _{Y}P_{theta }(A^{-1}(B)mid T=t) P_{theta }^{T}(dt).}$

The distribution

${displaystyle P_{theta }^{A}}$

does not depend on

${displaystyle theta }$

because

${displaystyle A}$

is ancillary. Likewise,

${displaystyle P_{theta }(cdot mid T=t)}$

does not depend on

${displaystyle theta }$

because

${displaystyle T}$

is sufficient. Therefore

${displaystyle int _{Y}{big [}P(A^{-1}(B)mid T=t)-P^{A}(B){big ]} P_{theta }^{T}(dt)=0.}$

Note the integrand (the function inside the integral) is a function of

${displaystyle t}$

and not

${displaystyle theta }$

. Therefore, since

${displaystyle T}$

is boundedly complete the function

${displaystyle g(t)=P(A^{-1}(B)mid T=t)-P^{A}(B)}$

is zero for

${displaystyle P_{theta }^{T}}$

almost all values of

${displaystyle t}$

and thus

${displaystyle P(A^{-1}(B)mid T=t)=P^{A}(B)}$

for almost all

${displaystyle t}$

. Therefore,

${displaystyle A}$

is independent of

${displaystyle T}$

.

Example[edit]

Independence of sample mean and sample variance of a normal distribution[edit]

Let X₁, X₂, …, X_n be independent, identically distributed normal random variables with mean μ and variance σ².

Then with respect to the parameter μ, one can show that

${displaystyle {widehat {mu }}={frac {sum X_{i}}{n}},}$

the sample mean, is a complete and sufficient statistic – it is all the information one can derive to estimate μ, and no more – and

${displaystyle {widehat {sigma }}^{2}={frac {sum left(X_{i}-{bar {X}}right)^{2}}{n-1}},}$

the sample variance, is an ancillary statistic – its distribution does not depend on μ.

Therefore, from Basu’s theorem it follows that these statistics are independent conditional on

${displaystyle mu }$

, conditional on

${displaystyle sigma ^{2}}$

.

This independence result can also be proven by Cochran’s theorem.

Further, this property (that the sample mean and sample variance of the normal distribution are independent) characterizes the normal distribution – no other distribution has this property.^[3]

References[edit]