REPRINT

By Martin Plissner

NY Times Op-Ed

Sunday, October 29, 2000

WASHINGTON - A week of volatility in the public," Bernard Shaw of CNN concluded Thursday afternoon as George W. Bush's lead over Al Gore in the CNN-USA Today Gallup "tracking" poll soared to 7 percentage points.

The week had begun with the poll showing Governor Bush trouncing Vice President Gore by 9 points, about the same margin Bill Clinton dispatched Bob Dole by four years ago. By 5 o'clock Monday afternoon, however, the Bush lead had slipped to a mere two points. On Tuesday and Wednesday, it was dead even, and then came that 7 points on Thursday. To cap off the week, on Friday Gallup had Mr. Bush soaring toward a 52 percent to 39 percent landslide.

This roller coaster ride is not confined to Gallup, still the polling brand name most familiar to the public. During a relatively uneventful eight days, ending Sept. 22, a 14-point Bush landslide in the first of a series of Newsweek polls melted away to 2 points at the end. And the variations on many days among the multitude of different polls - in point spread and even in who is leading - have become a subject of as much derision as the deceptive claims of the candidates in this year's election season.

Like the candidates, the pollsters naturally hold themselves blameless. Seldom in their accounting for day-to-day and poll-to-poll disparities do you hear a word about the inherent fuzziness (to borrow a word from Governor Bush) of their own numbers. The fuzziness, they imply, is all in the heads of an insufficiently attentive and caring public, especially this year's reigning suspect as the source of all volatility: independent women. "They bounce around," says Frank Newport, Gallup's executive editor, "depending on the last thing they heard on the news."

Neither Gallup, of course, nor most of the other survey researchers struggling to cope with this supposed pogo-stick behavior are fly-by-night operators. The polling organizations are run for the most part by veteran professionals who identify their surveys as "scientific" to distinguish themselves from competitors who use less refined methodologies, and this year, as in the past, they will assure you they are doing everything perfectly right.

How can this be true and still yield the wildly swinging results we are seeing? Well, there are, of course, those flighty independent women. But, much more to the point, there is sampling error. Plus or minus 3 or 4 percentage points are the figures commonly posted for this error in an overall sample of "likely voters." What the public is seldom being told in this election season is that the potential range of error applies separately to the numbers for both Mr. Bush and Mr. Gore.

What this means is that the spread between the candidates, which is all that anybody talks about, can be off by as much as 8 percentage points and that discrepancy can still be "within sampling error." And that's before you factor in the error, impossible to calculate, arising from the fact that as few as a third of the targeted respondents in the scientific model may have been reached and induced to answer questions.

It is in the tonier, "in-depth" analytical reporting in the press, even more than in the down-to-earth horse-race stories about the campaign, that you get the greatest exposure to sampling error. Take the incessant breakouts on the voting intentions of men and women. Here the size of each group is half of the full sample, and the sampling error on the Bush-Gore spread grows from, say, 8 percent to 11 or 12 percent.

Out of that, and perhaps some other things as well, you get such howlers as the discrepancy at the close of the Democratic convention between Gallup's figures and those of Newsweek's Princeton Survey Research. In polls a day apart, Newsweek had Mr. Gore ahead of Mr. Bush among likely male voters, while Gallup had Mr. Bush ahead by 19 points. Or consider a recent USA Today story that attributed a sudden overall shift in its Gallup tracking poll to "independent voters, mainly women." The sampling error on the Bush-Gore margin in that poll's sub-sample of independent women would be in the neighborhood of 16 points.

On Nov. 7, of course, the deplorably unstable subjects of all this scientific research will register the final bounce, and this gives the entire polling industry reason to be deeply worried. The business world spends zillions of dollars a year for surveys of the choices Americans are likely to make on all sorts of things, with little hard evidence of the relation to reality of the research they are paying for. The nearest thing there is to an objective check on the validity of opinion surveys comes once every four years with the final polls before a presidential election.

Four years ago there were sizable differences between what the final polls predicted and the actual election results - with most of the polls, as it happened, overstating the Clinton margin. Though this evoked some grumbling from Republicans, it got relatively little attention from the general public, since Mr. Clinton did in fact win quite comfortably. The National Council of Public Polls, the vigilant watchdog of polling's image, soon put out a report thick with tables purporting to show that 1996 had been one of the industry's "best years ever" in forecasting an election outcome.

In the much closer election that may be coming, discrepancies on the order of those in 1996, if they went, on balance, the wrong way, could be very bad for business. In over half a century, only one final poll has ever come down on the wrong side of a presidential election, but the closer the election, the greater the risk. This year it's not just the politicians who have reason to be nervous.

Martin J. Plissner is the former executive political director of CBS News. He is the author of "The Control Room: How Television Calls the Shots in Presidential Elections."