Tuesday, October 01, 2002

posted 10/01/2002 07:27:00 PM by Daniel   0 comments

FURTHER UPDATE: Western Political Review (10/8/02) reports L.A. Times poll bias.

Knowledgeable political observers keep piling on, as indicated by the following excerpts [the Western Political Review requires a $9.95/mo subscription]:

WesternPR has learned from reliable sources that the Times Poll was weighted in a way to take a larger sample of Democrats. In short, our source maintains the Times used a sample made up of 50% Democrats, well over the registration figures provided by the Secretary of State’s office of 44.73%. That’s just over a 5-point head start if you are a Democrat.

Remember that the Times didn’t represent that they were mirroring statewide registration figures. ... "This is why you can’t take polls at their face value," a highly regarded campaign consultant told WesternPR. "The Times can poll any way they like --- and we all know they like Democrats."

According to our sources, there is one more tidbit of information about the LA Times’ polling techniques worthy of mentioning. When they conduct a poll and get numbers that don’t appeal to them…as they did not to long ago in a similar Davis-Simon match-up, they simply don’t publish it.

If the Times' sample overestimated the Democrats by 5%, it must have also underestimated the Republicans and others/undecideds by 5%. If (for example) the Republicans were underestimated by 3% and the undecideds underestimated by 2%, then that alone would transform Davis' 10 point lead into the 2 point lead I calculated below.

UPDATE: California Political Review shares my doubts about the L.A.Times poll.

John Jorsett posted an October 3rd "Capitol Watch" email from California Political Review which says the following:

Polling experts criticized an L.A. Times poll released this week for its sampling methods and results that starkly contradict other recent surveys. The Times employed "random digit dialing"; calling randomly-generated telephone numbers. Respondents are asked whether they are registered, whether they plan to vote, and about their past voting. This unverifiable information is used to define survey results as polling of "registered" and "high propensity" voters. Both parties' campaign polling uses official voter files to select samples and to categorize voting propensity, not relying on the voters' recollection and honesty. Also, according to the Times Poll, "the entire sample was weighted slightly to conform with census figures for sex, race, age, education, region and registration." Thus, if the sample's proportion of Republicans is larger than GOP registration, Republicans are dropped to conform.

Polling experts criticized the Times' projection that election turnout will favor Democrats by 13 percent, 53 to 40: four points more than Democrats' statewide registration advantage. The Times Poll also found a 51 percent approval rating for Gray Davis's "handling of his job as governor." Other surveys consistently find lower approval ratings. Last week's Public Policy Institute of California survey found Davis's job approval at 42 percent among likely voters, with 52 percent disapproval.

* * *

Times overstated Democrats' popularity compared with other surveys. Although Gov. Davis has consistently polled at or near 40 percent in several recent surveys, the Times Poll showed him moving up to 45 percent. Polling experts said they believe Davis is between 6 and 10 points ahead; the Times placed him at the high end of that margin, leading 45 to 35.

The Times contradicted findings in down-ticket races. An early September Field Poll showed Tom McClintock leading Steve Westly for controller 42 to 30; in the Times, Westly led 44 to 35. Internal polls give Keith Olberg 2 points over Kevin Shelley for secretary of state; in the Times, Shelley leads by 14. GOP insurance commissioner nominee Gary Mendoza is within 4 or 5 points of John Garamendi in other surveys; in the Times, Garamendi leads by 22.

++++++++++++++++++++++++++++++++++++++

There's something strange about today's (10/1/2002) Los Angeles Times poll.

According to the L.A. Times, "Gray Davis has opened a substantial lead [10 percentage points] over Bill Simon Jr. in the race for California governor." But is that really true? Other data from the poll shows some anomalous numbers when compared to a Field Poll in early September.

Consequently I've done some statistical analysis which suggests that Simon has sharply narrowed the gap, and trails Davis by only about two percentage points.

Anyone who glances over my blog will quickly see that I am no big fan of Simon, whose incompetence as a candidate is exceeded only by Davis' corruption as a governor. So I'm certainly not about to whitewash Simon's poll results. But I also don't have a lot of respect for the reliability of the Los Angeles Times' surveys, which appear to me even more suspect than the Field Poll.

Here's what's strange: The Times Poll has radically different (and more Democrat-leaning) results for the seven down-ticket statewide races than the Field Poll, as indicated in the table below. A particularly striking example is the race for Insurance Commissioner: Garamendi led Mendoza by 5 points in the Field Poll but by 22 points in the Times Poll. Another example is the State Controller race, where McClintock led Westly by 12 points in the Field Poll, but trailed Westly by 9 points in the Times Poll.

	Vote for Democrat		Vote for Republican		Undecided / other
office	L.A.Times	Field	L.A.Times	Field	L.A.Times	Field
Att. General	53	45	31	33	16	22
Treasurer	47	39	29	37	24	24
Ins Commissioner	51	40	29	35	20	25
School Sup	34	23	15	19	51	58
Lt. Gov	52	46	31	36	17	18
Sec of State	46	32	32	34	22	34
Controller	44	30	35	42	21	28
Governor	45	38	35	31	30	31

Such results make no sense. No one is paying much attention to the down-ticket races, and the candidates are mostly hoarding what money they have for advertising during the final weeks of the campaign. There is nothing that could move the numbers so sharply in favor of the Democrats. One or the other of these polls (or both) must be seriously in error.

Aside from the usual random fluctations (i.e., the "margin of error") that all polling is subject to, there are many other potential sources of error. The exact wording of questions, the training of the interviewers, the times of day when polling takes place, the randomness of the selection method, etc. all play a part. Even assuming that a polling organization is honest and competent (always a debatable assumption), there remains the "secret sauce" that each polling firm employs: Each organization has its own unique weighting formula to try to rebalance the survey sample for age, gender, ethnicity, political party allegiance, geography, and likelihood of voting. Very small perturbations in those formulas can have a substantial effect on the reported results.

Intuitively, the Field Poll would seem to make more sense on the down-ticket races. A 5 point lead by Garamendi is much more believable than a blow-out 22 point lead. And other surveys I've seen indicate that McClintock is ahead of Westly, so a Westly lead of 9 points seems highly unlikely.

So I made the initial assumption that the Field Poll is the more accurate, and that the down-ticket numbers have probably not changed much during the three weeks between the polls. Then I did a least squares fit between the Field and Times results for the down-ticket races, to calculate correction factors. It turned out that you should multiply the pro-Democrat results in the L.A. Times Poll by 0.7871; multiply the pro-Rep results by 1.1632, and multiply the undecided/other results by 1.1944. Then I applied this to the L.A. Times numbers for governor, normalized the results so that the sum equalled 100 percent, and obtained the following:

Davis (at 31.6%) trailed Simon (at 36.4%) by 4.7%, with 32% undecided.

But I hate to make the assumption that the Field Poll is entirely accurate, and the L.A. Times Poll is entirely wrong, with regard to the down-ticket races. So I split the difference by averaging the two polls, and used the averages of the down-ticket races to re-calculate correction factors. Under this assumption, you should multiply the pro-Democrat results in the L.A. Times Poll by 0.8935; multiply the pro-Rep results by 1.0816, and multiply the undecided/other results by 1.0972. I applied this to the L.A. Times numbers for governor, normalized the results so that the sum equalled 100 percent, and obtained the following:

Davis (at 36.2%) led Simon (at 34.1%) by 2.1%, with 29.7% undecided.

I tried various other options in my spreadsheet, such as testing the two subgroups that the Field Poll divided their sample into (they asked half their sample about three of the down-ticket races, and the other half of their sample about the other four down-ticket races). The result correction coefficients were similar, which again suggests that the Field Poll results for down-ticket races are more reliable.

So my surprising conclusion is that the race for California Governor has actually tightened, and may in fact be a statistical dead heat.