Eagle Eye (indyeagleeye) wrote,
Eagle Eye

John Rentoul: More on the polls

I see that Calum 1693 has taken exception to my use of poll averages again, in the comments on my post last night, which reported the mean of ComRes, Angus Reid, ICM, Ipsos-MORI, Populus and YouGov. (Those are the six members of the British Polling Council that have a track record going back at least to last year.)

Conservative   36%
Labour              27%
Lib Dem            27% 

Worth reproducing our previous exchange on this subject.

Calum 1693: John, mathematically, averaging polls is utterly bogus. It's meaningless. A poll is one possible value of a random variable. You took 3000 samples to estimate a value of that variable and you know roughly how accurate that estimate is likely to be. However, if you then take another 3000, and another 3000, and indeed another 30,000, you don't change the margin of error by any meaningful amount (or we could do one poll of thirty thousand people and knock this tedious voting business on the head). Your average poll still has the same margin of error the others do and it gives you no new information.

My reply: Thanks calum_1693; you make me feel old, as I have had this debate every election since at least 1992. Bogus is not a useful word here. This is not about margin of error. And it's true that averaging is no likelier to give you the "right" answer than choosing one poll and taking its smoothed average (to minimise the effect of outliers). But the point is that we do not know which poll(s) are right, so averaging all the ones that meet quality criteria is the best way of summarising the information that is available.

Calum 1693: Well, I was 11 in 1992, so you do indeed have the edge on me there. I still don't quite follow your logic, though. If each of the polls is conducted in the same way, averaging them produces a value with a 95% confidence interval that is still plus or minus about 3% (it is slightly narrower than the individual polls, but not by much). Now, polls are conducted differently, yes, but in practice they do not produce systemically different results - in other words, they are massaged to be (in the long run) in line with everyone else's numbers! If I have five pens, seven pencils, and a Sharpie, I don't have an average of 4.3 writing implements: I have five pens, seven pencils, and a Sharpie. With the amount of polling going on at this stage, we should expect to be seeing an "odd" poll every day or two and a really bizarre one roughly weekly, and without paying too much attention, that's what we seem to be getting.

Anyway, opinion polls are boring. The policy implications of Clegg as Prime Minister: how much Lib Demmery could reasonably withstand first contact with reality? That would be an interesting column.

My reply: You are right, but the key point is in parentheses: in the long run, polls are adjusted to be in line with each other. In the short run, different methods produce different results. An average is simply a way of trying to capture as much relevant information as possible.

Tags: opinion polls

  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your IP address will be recorded