Home | Opinion | Columnists | Mark Mellman

A lot more than just numbers

With everyone dissecting polls, I thought some not-so-secret secrets of poll-watching might be appreciated.

My most important caveat — simply looking at the margin in a single poll, at one point in time, without regard to the type of race, is often foolhardy.

The incumbent rule — Commentators frequently ignore what should be the best-known rule of poll interpretation: that most late undecideds, in a race against an incumbent, will break toward the challenger.

Thus, the important number is not the margin between an incumbent and a challenger, but rather the percentage of the vote garnered by the incumbent. By this standard, one would expect to see seven GOP Senate incumbents lose their seats next week.

However, this is not some “iron law” — it reflects an underlying psychological mechanism that may prove less prevalent this cycle.

Traditionally, incumbents are much better-known than their challengers. So when voters say they are “undecided,” what they really mean is that they know the incumbent and do not particularly want to vote for him or her. However, they do not yet know enough about the challenger to support that candidate. By Election Day, these voters sniff out just enough information to justify casting a ballot against the incumbent. Thus, in the past decade, on average, 70 percent of undecideds broke to challengers.

It doesn’t always work that way, however. In 2006, our client, Sen. Maria Cantwell (D-Wash.), received four points more than any public poll predicted, while her challenger actually ended up with fewer votes than he had in all but one late public poll. Sen. Jon Tester (D-Mont.) only increased his vote by half a point from the average, while the incumbent he defeated gained almost three points over the poll average. Indeed, in 2006, five of the 11 competitive Senate races violated the incumbent rule, with undecided voters breaking toward incumbents rather than challengers.

Undecideds break to incumbents when they detest the challenger even more than they dislike the incumbent.

With all the money spent attacking challengers these days, fewer voters go into the final days of contested campaigns either ignorant of or ambivalent about challengers.

Knowing when the incumbent rule is likely to operate, and when it is not, requires thorough analysis of undecided voters’ attitudes, which few public poll reports provide.

Momentum —  Consider momentum instead of focusing exclusively on each candidate’s level of support. If one contender is gaining rapidly in the closing days, the trend may well continue through Election Day, again rendering the margin at any one point in time only modestly useful.

Late public polls in ’06 had Sen. Cantwell leading first by 12, then by 13 and finally by 16 points before she won by a 19-point margin, three days after the final survey.

Means vs. Medians — Several years ago I argued here that most every poll had some value and the results of each should be taken into account in evaluating candidates’ standings. That led in part to a variety of more and less complex poll-averaging efforts.

Averages are certainly more useful than single polls, but as Francis Galton, the Victorian polymath who founded meteorology and psychometrics, wrote, averages give “power to cranks in proportion to their crankiness.”

Two astrophysicists computed the median poll results by state in 2004 and correctly predicted the presidential outcome everywhere but Hawaii. RealClearPolitics’ averaged results inaccurately suggested Bush wins in both Hawaii and Wisconsin, having been derailed in the Badger State by a cranky Gallup poll showing Bush with an eight-point margin in a state he narrowly lost. Watch the median, not just those means.

Polls are not as simple as they look. Margins in a single poll can do as much to mislead as to inform.

Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the majority leaders of both the House and Senate.