Home | Opinion | Columnists | Mark Mellman

Reading polls to suit the story

Quality polling can improve political journalism by keeping horserace stories honest, or  at least tethered to some reality.

Quality polling can improve political journalism by keeping horserace stories honest, or  at least tethered to some reality.

Most of the time, poll data lead journalists to more accurate conclusions. In New Jersey, however, we witnessed the inversion of that process: The polls were bent to conform to the story journalists wanted to tell, even though it was largely unsupported by the data. (Full disclosure: Sen. Jon Corzine, the Democratic candidate for governor, is our client.)

In the closing days of the campaign, personal invective from the senator’s ex-wife appeared in the press and became the central thrust of Republican candidate Douglas Forrester’s ads. The story was too juicy for reporters to resist: a political campaign turned on its head by comments from a former spouse.

But instead of acting as a factual check on drooling reporters, polls were widely misinterpreted by their usually careful sponsors to support a theory that had little basis in fact.

Typical of the breathless reporting was a piece headlined: “Corzine falters as dirt flies,” which argued the “race looked set for a nail-biting finish yesterday with two new polls showing Sen. Jon Corzine paying a price for attack ads featuring his disgruntled ex-wife.”

The evidence?

First a Marist poll that, under the heading “What a difference a day makes,” offered this “analysis”: “The race has taken a decidedly different turn. On Wednesday, Jon Corzine had a comfortable lead of 13 points over Doug Forrester. … On Thursday, after the flurry of … ads and accusations, Corzine has a four-point margin.”

The “analysis” failed to mention there was no change among registered voters from Wednesday to Thursday. All of the purported difference was among likely voters who seem to represent about 55 percent of Marist’s sample. Never mind critiques of likely-voter models here and elsewhere — Marist’s sample of likely voters for Thursday is about 205 with a margin of error of about 7 points and a margin of error for comparison of over 10. Thus, the differences may indicate no real change at all.

Moreover, there is no indication that the two nights of the survey were designed to produce independently representative samples each night. The second evening may simply have been more Republican than the first night’s interviews. Unless this were being controlled, it would not be surprising for the sample characteristics to vary widely on two different nights.

Finally reporters, and Marist itself, fail to focus on the contrary evidence in their own survey. Marist’s release concludes the Forrester ad “raises doubts” about Corzine, even though only 18 percent overall and 6 percent of Corzine voters say it will make them less likely to support the senator. Anyone practiced in looking at such data knows these are meager results.

Indeed, more people were less likely to vote for Forrester as a result of the ad (23 percent) than said it would weigh against Corzine. But since backlash is not part of the fun story line, these inconvenient facts were not mentioned in any report I’ve seen.

Reporters also took a Farleigh Dickinson University (FDU) poll showing Corzine just two points ahead as evidence of the problems being caused by the ad. But compared to the prior FDU poll there were just two points of movement. Statistically it was barely a ripple.

Even the Eagleton poll worked to get in the act. Since its poll indicated only one point of movement, Eagleton was reduced to citing vague changes in the night-by-night horserace among the subsample of 229 independents to write several lines about the influence of these charges on the race.

In short, none of these polls provides any evidence that the quote roiled the race in any meaningful way. Yet each was willingly made to serve that argument. It is a bad place for polls to be. Scientific surveys should be a corrective to inaccurate inferences about public opinion, not a source of ready confirmation for any story line that seems interesting to journalists.

Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982, including Sen. John Kerry (D-Mass.) last year.