In studying for my upcoming econometrics examination, I have been dissatisfied with the proof my professor utilized, and our text leaves the problem rather nebulous. I figured I would take some quick notes about the proof I think works well.

First, the variance operation (V) removes the addition of any non-stochastic elements and extracts the square of any non-stochastic multiplier, e.g., if we have constants a, and b with a stochastic variable X, then


The second form simply states a truism about the definition of variance, where the Cov function is the covariance of the two stochastic parameters.

Second, let me state the ordinary least squares (OLS) classical regression statement. It says,


where a is the intercept coefficient of this equation, b is the slope coefficient, and e is our stochastic disturbance.

Third, I will simply state the value of our estimator for b, let me call it b'.


A very nice proof, amongst other valuable information in a rigorous mathematical presentation, can be found in some notes I ran across (link). Since I was unable to find a proof for the variance of our intercept estimator, even though that link said it was going to prove it, I will simply build off its work to develop the proof with the given information above.

Let us proceed.



Given (ii*) we can replace Y and X by their means Y and X, respectively, which stems from the fact that the regression passes through their means, a rather simple, but auxiliary, proof (see lemma at the end of this blog). By rearranging terms in this modified form of (ii) we can denote a' by,


but given (ii) we can replace Y by the means of the given values in (ii), i.e.,


This will be the form of which we will take the variance. Notice that only e and b' are stochastic. The actual population parameters a and b are not, and X is assumed non-stochastic in our OLS assumptions.

To begin the meat of our proof, recall (i) that allows us to eliminate a.


Where in the third step I split the LHS and RHS as non-stochastic and stochastic, respectively. Now that we have the addition of two stochastic parameters, we can use (i*) to state that


But part of our assumptions will entail that the coefficients of our regression be independent of our error term (e). Thus, the covariance is zero. Furthermore, our assumptions entail that V{e} is equal to the population variance sigma squared. But we have e = 1/n * ∑e. Thus, by (i*), the definition of variance, we have V{e} = ∑σ²/n² , but the sum of n sigma squares is just nσ². Thus, V{e} = nσ²/n² = σ²/n. To recapitulate,


But notice that (b - b')X = bX - b'X, where the left term on the RHS is non-stochastic since b is non-stochastic and X are non-stochastic, so X is non-stochastic. Thus, we can make use of (i) again and remove that part of the term and pull X out of the variance function. So,


But notice that we already have V{b'} by (iii). Thus,


Since both terms on the RHS have sigma squared, we can factor them out.


Which is precisely what was to be shown.






To replicate the proof in all its steps without explication:


Define


Thus,


But V{e} = ∑σ²/n² = nσ²/n² = σ²/n, and Cov(b', e) = 0. So,


Therefore, V{a'} = σ^{2 *} [X² / ∑(X-X)² + 1/n].

Q.E.D.




lemma: X and Y satisfy (ii*) based on the first-order conditions for optimization.

We want to minimize a and b in ∑(Y - a - bX)². This entails the first order condition that the partial derivative of this in terms of a leads to setting


But since it is equal to zero we can divide by -2 to trim the fat. So,


But we can divide both sides by n and maintain the equality. So,


Thus, the coordinates (X, Y) satisfy our OLS regression equation.