Home | Opinion | Columnists | Mark Mellman

The meaning of those poll responses

Regular readers have heard me rail against the inaccuracy of early polling, which often fails to presage ultimate electoral outcomes.

Yet I have also maintained that presidential elections are predictable based on the fundamentals — incumbency, war/peace, prosperity and the like.

ADVERTISEMENT
Recognizing the implicit contradiction, political scientists Gary King and Andrew Gelman asked, in one of academia’s best-titled papers, “Why Are American Presidential Election Campaign Polls So Variable When Voters Are So Predictable?”

Their answer focuses on learning during the campaign, and while they may be right, I fear the problem runs deeper.

For 25 years, I’ve counseled clients and colleagues to consider psychological research demonstrating that people are very poor reporters of their own decision-making processes. It reveals that we have little reason to believe much of what people tell us directly in focus groups and polls about why they do what they do.

Though it is heresy for a pollster to say it, the evidence also suggests people are only mediocre predictors of their own behavior. Responses to horserace questions a year out may be a special case of faulty prediction.

Hardly an original thought, it can be traced at least to Russian exile and founder of Harvard’s sociology department Pitirim Sorokin, who titled his 1936 paper, “Can One Predict His Own Behavior 24 Hours In Advance?” His answer, based on a study of federal employees, was a resounding no: When asked how much time they would devote to various activities during the subsequent eight-hour workday, the average person was off by five hours.

Since Sorokin’s time, experiments and experience have validated his fundamental insight. Forty-three percent of students in one experiment predicted they would participate in a campus blood drive, but only 20 percent actually did. More than eight in 10 Cornell students predicted they would buy daffodils to benefit the American Cancer Society, yet just 43 percent purchased one. In another experiment, every participant predicted they would donate part of their fee to charity, but only 62 percent did.

Student subjects in psych experiments and federal workers aren’t the only ones who fail to predict their own behavior. Just 14 percent of those who told pollsters that they intended to purchase a computer did so, while 60 percent of those who were certain to buy a car in another study did not.

People are not even very good predictors of consequential career decisions. Forty percent of sailors who had told pollsters they were “very likely” to remain in the Navy left the service within five years; nearly one in five was gone within three. At the same time, half of those who claimed they were unlikely to stay nevertheless remained sailors.

We are not fools when it comes to predicting our own behavior but we seem better at predicting others’ behavior than our own.

In forecasting our own behavior, we appear to underweight the situational and over-weight our immediate dispositions; we ignore the influence of habit in predicting our own behavior, but recognize its role in the actions of others.

The implication for polling and politics: Polls may vary more widely than voters’ actual ballots because respondents have difficulty accurately predicting their own behavior, especially in the early stages of a campaign. Thus, even when John Kerry, flush from his primary victories or in the days following Abu Ghraib, led George Bush in the polls, most Americans rightly believed Bush would emerge victorious.

Perhaps politics provides another example of knowing the minds of others better than we know our own.

Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the majority leaders of both the House and Senate.

More in Mark Mellman

Mellman: Do polls see backlash over order?

Read more »