|
|
|
A Post-Mortem of the 2003 Knoxville Mayoral
Election
Dr. Michael M.
Gant
continued from Page 1
Lending credence to these expectations were two polls conducted by SurveyUSA for WBIR-TV. The first, conducted in July,
showed Haslam ahead of Rogero
by 51% to 38%. The second poll, conducted the week before the primary
election, indicated Haslam's advantage had
increased to fifteen percentage points. Internal polling conducted by the Haslam campaign also seemed to indicate a comfortable
margin of victory.
Apparently, the only poll that indicated a fairly close contest was the
survey of registered voters conducted by SSRI, and sponsored in part by the
News Sentinel. Our poll, conducted in mid-August, showed Haslam leading Rogero among
registered voters by only 27% to 22%. Among likely voters, the margin was
32% to 25%. Not surprisingly, given when the poll was conducted, we found a
large contingent of undecided voters; over 40% of those surveyed said they
had not decided for whom to vote.
In the August 24th edition of the News Sentinel, I was quoted as saying,
"It seems like everybody thought it wouldn't be much of a contest. But
it looks like it's going to be much closer." Dr. William Lyons, Haslam campaign manager, suggested that SSRI's polling showed far too many undecided voters,
and that their internal polling showed the percentage of undecideds to be "in the teens." He also
suggested our sample was too small. . Let's take each point in turn.
With respect to the number of undecideds, Dr.
Lyons probably meant that the percentage of undecided voters in the SSRI
poll was artificially high, since we know that in the context of election
polls, many respondents who say they are undecided will end up not voting.
One way to try to correct for this is to identify respondents who are least
likely to vote, and eliminate them from the analysis. This usually reduces
the percentage of undecided respondents. For example, among all registered
voters in our sample, 49% said they were unsure for whom they would vote in
the Mayoral election. When we reduced the sample to those who said they
were "very likely" or "somewhat likely" to vote, the undecideds dropped to 42%. When we further reduced the
sample to registered voters, who said they were at least somewhat likely to
vote, who reported having voted in the 1999 Mayoral election, and could
recall for whom they voted, the percentage of undecided voters dropped to
about 36%.
Still, 36% is a far cry from a number "in the teens," as Dr.
Lyons reported finding. Why the discrepancy? One possible explanation is
that Dr. Lyons might have used a different way to cull out the least likely
voters. Or, perhaps the samples were drawn differently. SSRI uses Random
Digit Dialing methodology that is widely accepted as standard in the
profession. Perhaps we used different sampling frames, the list of all
theoretically possible respondents to a survey. At SSRI we use sampling
frames that are as current as technology allows.
Dr. Lyons also said, "I would have started with a much bigger
sample." This statement implies that if we had drawn a larger sample,
we would have found fewer undecided voters, and
greater support for Haslam. Dr. Lyons is a
colleague of mine, and has a well-deserved reputation as a first-rate
sample survey expert. Thus, I can only conclude that Dr. Lyons was
misquoted, because the assumption that a larger sample would have produced
different results is incorrect.
To be more precise, once a sample size of about 100-150 respondents is
achieved, larger samples will not produce levels of candidate support that
are much different, nor will larger samples appreciably reduce the number
of undecided voters. What larger samples do provide is greater confidence
in the findings; technically, larger samples yield smaller standard errors.
The level of confidence is usually conveyed by pollsters by reporting the
margin of error. The larger the sample, the smaller the margin of error.
This makes sense. In polling, we are trying to estimate how a large
group of people think (e.g., whether people support a state income tax), or
how they will behave (e.g., vote in an election for Mayor), based on
information from a small group of respondents. The larger the number of
respondents, the more information, and the greater confidence we have in
our findings. But, the numbers reported, such as the percentage of
undecided voters, would change very little, if at all.
Now, let's return to the accuracy of polls. This is easy to check; we
simply compare the election outcome with the findings reported by various
polls. Of course, the "undecideds" are
no longer relevant since they either did not vote or, by definition, if
they voted they had decided how to vote. Haslam
defeated Rogero by 52.6% to 46.4%, a margin of
6.2 percentage points. The final SurveyUSA poll
reported by WBIR-TV showed Haslam leading by
fifteen percentage points, almost 250% larger than the actual margin. We do
not know what the Haslam polls but, given their
comments on election night, it is not unreasonable to suggest they found Haslam's lead in double digits--that is, at least ten
percentage points, much larger than the final outcome.
What about the poll conducted by SSRI for the News Sentinel? Among
likely voters, we found Haslam leading Rogero by 7.2 percentage points, very close to the
actual outcome, and much closer than the other polls reported. When we
completely eliminated the undecided voters, the margin was no higher than
ten percentage points. But eliminating the undecideds
assumes either that they will not vote, which is the case for many of them;
or that they will vote exactly like those who reported a preference said
they would vote, clearly an untenable assumption. If we assume that the undecideds who do vote will split their votes evenly,
the margin between Haslam and Rogero
remains 7.2 percentage points, within a percentage point of the final
margin.
Clearly, the poll conducted by SSRI was on target. The results of our
polls during the November 2002 elections were the target of similar
criticisms, even though we showed Phil Bredesen
defeating Van Hilleary by a narrow margin in the Gubernatorial contest, and
Lamar Alexander defeating Bob Clement by a wide margin in the Senatorial
election which, of course, is exactly what happened.
Public opinion polls can be an important source of information during
elections, and can contribute in a meaningful way to democratic politics.
But consumers of polls must be careful. When reading poll results, ask some
questions: Who conducted the poll? What is their track record? How do they
conduct their surveys? How were the questions worded? If this information
is not available, why not? Who sponsored the poll? Do they have a stake in
the outcome of the poll? If so, discount their findings. The value of polls
depends almost entirely on their source.
Dr. Gant is Professor of Political Science, and Director of the Social
Science Research Institute at the University
of Tennessee, Knoxville. He may be reached via e-mail
at mgant1@utk.edu
.
|
|
