Eric Rasmusen’s Weblog

In doing statistics, when should we weight different observations differently?

Suppose I have 10 independent observations of $x$ and I want to estimate the population mean, $\mu$. Why should I use the unweighted sample mean rather than weighting the first observation .91 and each of the rest by .01?

Either way, I get an unbiased estimate, but the unweighted mean gives me lower variance of the estimator. If I use just observation 1 (a weight of 100% on it) then my estimator has the variance of the disturbance. If I use two observations, then a big positive disturbance on observation 1 might be cancelled out by a big negative on observation 2. Indeed, the worst case is that observation 2 also has a big positive disturbance, in which case I am no worse off by having it. I do not want to overweight any one observation, because I want mistakes to cancel out as evenly as possible.

All this is completely free of the distribution of the disturbance term. It doesn’t rely on the Central Limit Theorem, which says that as $n$ increases then the distribution of the estimator approaches the normal distribution (if I don’t use too much weighting, at least!).

If I knew that observation 1 had a smaller disturbance on average, then I *would* want to weight it more heavily. That’s heteroskedasticity.

This entry was posted on Friday, September 28th, 2007 at 5:21 am and is filed under Math. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.