Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
Dewey Wins Again?

By cestmoi in Op-Ed
Thu Oct 02, 2003 at 06:28:16 AM EST
Tags: Politics (all tags)
Politics

We're entering the final week of the California recall campaign, and as one might expect, there are poll results. What's surprising is that the poll has either captured a significant shift in California politics or the polls are severely skewed.


There's a picture of Harry Truman holding up a 90 point headline that says, "Dewey Defeats Truman." The picture has appeared in most college statistic books published ever since.

The lesson statisticians drew from their failure to accurately forecast the 1948 election was that it is crucial to choose a sample that accurately reflects the population. Dewey won among households that had telephones in 1948. Unfortunately for Dewey, in 1948, not all households had phones.

CNN/USA Today/Gallup released a poll today showing Arnold Schwarzenegger leading by a significant margin. If you scroll 2/3rd's of the way down you find that Gallup randomly selected 1007 Californians for its poll. Of those 1007, 787 were registered to vote. Of those, 356 are identified as Republican or "Republican Leaners," and 368 are identified as Democrats or "Democratic Leaners." The poll's writeup doesn't define the term "leaner."

Here's what's odd about those numbers. If you go to the California Secretary of State's website, you'll find a breakdown of California voters. I've tabulated the state's numbers and Gallup's numbers to make the disparities clearer.

% % %
Registered Democrats Republicans
State 69 44 35
Gallup 78 45 45
These are the key points:
  • The percentage of registered voters has jumped 9%.
  • Most of the newly registered voters are Republican (10 out 11).

So one of two things has happened: either the voter makeup in California has shifted significantly towards the right or the poll's sample is skewed. The problem with the former interpretation is the preponderance of new voters registered as Republicans. In a state as fractious as California, only one chose to be a Democrat, and none chose to be an independent? To me, it's not credible.

Weapons of Mass Indirection
The LA Times ran a poll a couple of weeks ago that purported to show McClintock with 18% of the vote. That number was at variance with an earlier Field poll that showed McClintock at 13%. If you looked closely at the Times's sample, you found a disproportionate number of Asians. The poll sample comprised 18% Asian vs. 11% in the general population. None of the subsequent polls have corroborated the Times.

The Times's poll sample was so skewed that some conservative pundits began speculating that the poll was intended to falsely encourage Tom McClintock into staying in the race. McClintock was a distant third behind Cruz Bustamante and Arnold Schwarzenegger. The thinking being that if McClintock stayed, he would siphon votes from Schwarzenegger and thus ensure Bustamante's election should Davis lose the recall. Davis deployed a similar strategy of encoraging his weaker opponent in the 2002 election. Davis donated $10 Million to Bill Simon, the then second place Republican, and ran ads that smeared the front running Republican, Richard Riordan. Riordan lost the primary to Simon who in turn lost to Davis in the general election.

Maybe the Gallup poll is accurate this time around, but somehow, I can't help but think of Mr. Dewey.

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Related Links
o picture
o poll
o breakdown
o LA Times
o Also by cestmoi


Display: Sort:
Dewey Wins Again? | 102 comments (78 topical, 24 editorial, 0 hidden)
Cool story (4.00 / 2) (#6)
by khallow on Mon Sep 29, 2003 at 10:08:14 PM EST

It's a cool way to figure out these conflicting polls. If the polls don't reflect in some way the underlying distribution of parties, ethnic groups, or some other measurable criteria, then there's a really good reason to dispute the poll.

Stating the obvious since 1969.

There's been some discussion (4.80 / 5) (#7)
by aphrael on Mon Sep 29, 2003 at 10:15:39 PM EST

There's been some discussion of this at California Insider, the weblog of a Sacramento Bee political reporter; apparently the sample used by the Gallup poll was quite different from the samples used in the earlier Field and LATimes polls.

The inadequacy of polls has been something that people following this election closely have known since day one: nobody has a well-tested turnout model. Most polling agencies base their estimates for who is going to turnout on who turned out in the last election of this type; that data doesn't exist for this election.

Add into this the fact that absentee ballots turned in the day of the election, and provisional ballots, won't be counted for days, and it becomes clear that whatever is reported next Tuesday night may become fiction by next Thursday morning.

more discussion (4.66 / 3) (#8)
by khallow on Mon Sep 29, 2003 at 10:42:40 PM EST

The Field Poll issued it's own report on why and how the sample used by the LA Times poll differed from the Field Poll. In summary, the LA Times had a group of 18% nonwhite, nonhispanics in its poll that would vote strongly for Bustamante and against the recall. So either some ethnic groups like the Asian Americans have stopped sitting on the fence (unlikely) or the LA Times has a few too many blacks or perhaps American Indians in its poll.

Stating the obvious since 1969.
[ Parent ]

All fired up (4.60 / 5) (#10)
by Dphitz on Tue Sep 30, 2003 at 12:30:16 AM EST

I would bet the majority of those registering are the ones who are really fired up about this election (those who really want Davis out of office).  With Davis' record, Bustamante wanting to raise taxes by $10 Bil if elected, and the political pandering to special interest groups and illegal immigrants, this is the most fired up this state has ever been about an election.  And the people most fired up about it are probably conservative or tired of the Democratic leadership (me).  


God, please save me . . . from your followers

I dunno (4.00 / 5) (#11)
by andamac on Tue Sep 30, 2003 at 01:01:18 AM EST

I'm not so much fired up as grim. Every time I hear the recall mentioned since that debate debacle another little part of me dies.

Truth be told, I'd still really like to put "Fuck Off" in the write-in spot. I can't even be sure that I'd be voting if it wasn't for the two propositions on the ballot.

[ Parent ]

i think (4.00 / 1) (#13)
by the77x42 on Tue Sep 30, 2003 at 04:42:28 AM EST

if arnold didn't run, voters would vote against the recall.


"We're not here to educate. We're here to point and laugh." - creature
"You have some pretty stupid ideas." - indubitable ‮

wait for it... (4.27 / 11) (#14)
by MMcP on Tue Sep 30, 2003 at 10:30:04 AM EST

wait for it...

but if Arnold did run, it would be a TOTAL RECALL

I am ashamed.

[ Parent ]

wait for it... (4.00 / 1) (#63)
by the hermit on Thu Oct 02, 2003 at 09:37:02 AM EST

wait for it...

Arnold is Running, Man! He's Running!

-the hermit

[ Parent ]

It's not the same (5.00 / 8) (#17)
by aphrael on Tue Sep 30, 2003 at 12:21:07 PM EST

I'm not convinced the analogy is appropriate. One of the things that's mostly forgotten about the Dewey defeats Truman fuck-up is that it wasn't just a breakdown of election night exit polling; the leading poll organizations of the time had consistently shown a Dewey victory since mid-spring. According to David McCullough's biography of Truman, Roper and Gallup had even stopped conducting polls by early October, convinced that there was no point and the polls were a waste of money; Dewey was so confident of victory that he had put together a shadow government; and leading Democratic party officials were convinced that Truman had no chance. The entire conventional wisdom showed a nasty defeat for Truman ... and yet Truman shocked everyone by pulling it off.

It's well understood by California newspapers and pundits that poll results are an unreliable indicator for this election because nobody has a reliable turnout model; the wide variance between the three major polling organizations (Gallup, LA Times, and Field) reinforces this point. And there's none of the overwhelming conventional wisdom that characterized the 1948 presidential election. So the comparison fails.

Dewey as an example (4.75 / 4) (#21)
by cestmoi on Tue Sep 30, 2003 at 01:13:44 PM EST

I ignored the other aspects that led to Truman's surprise victory because, as you point out, they don't apply in this case. Nonetheless, sampling error was a key component in Dewey's misplaced confidence.

The issue I'm trying to raise is the idea that an organization could intentionally exploit sampling errors to induce a sense of over confidence in an opponent. The Times's case was an example of that kind of a ploy - keep McClintok in the race to damage Schwarzeneggar's chances.

You may be correct. The variance between the three polls maybe due to different guesses as to who will vote next Tuesday. And that ambiguity is what makes it possible to manipulate a poll's outcome to achieve a desired effect.

What is really interesting is if the poll is accurate. It would signal a sea change in California.

[ Parent ]

Hmm. (4.75 / 4) (#23)
by aphrael on Tue Sep 30, 2003 at 01:19:07 PM EST

Your article does a bad job of explaining that it's about the possibility of intentional exploitation of poll results; I didn't get that impression at all.

I think there is a high likelihood that 70-80% of recall-related new registrations are republicans; that's only to be expected when the registrations are happening because of voter excitement at the recall of an unpopular democratic governor. However, a Schwarzenegger victory wouldn't necessarily be indicative of a sea change.

The primary problem in contemporary California politics, aside from the low quality of our politicians, is that the social conservatism of the state Republican party alienates 60-70% of the voters while, at the same time, it is impossible for a non-social-conservative to win a Republican primary. This has given the Democrats a virtual lock on statewide office. A surge in Republican registration indicates a great disgust with the Democrats - but it doesn't indicate a change in the view of Republican social policies. Schwarzenegger can win because it's a fractured field and he doesn't support those policies; unless the surge in Republican registration enables other social moderates to win Republican primaries, however, it's not going to help any other Republican candidate get elected.

[ Parent ]

Republican Primaries (4.00 / 1) (#28)
by Bad Harmony on Tue Sep 30, 2003 at 08:02:20 PM EST

While what you say may be true, as a Republican, I would say that if I wanted a Democrat for my party's nominee, I would have voted for one. I used to live in a predominately Democratic district that had a Republican representative. The problem was that she was so far off on the left wing of the Republican Party that she could have passed for a moderate Democrat. She only looked conservative when compared to some of the Democratic candidates for the office. I didn't like voting for her, knowing that on most of the issues I cared about, she could be counted on to vote with the Democrats.

54º40' or Fight!
[ Parent ]

There's the problem, see. (4.50 / 2) (#29)
by aphrael on Tue Sep 30, 2003 at 08:15:33 PM EST

The Republican party in California is basically an amalgamation of a socially conservative group with a goldwater-style republican/libertarian republican group. The latter can't win the primary. The former can't win a general election.

[ Parent ]
Not sure what you're saying (5.00 / 1) (#40)
by epepke on Wed Oct 01, 2003 at 02:24:44 PM EST

So all of the leading pollsters with respect to Dewey versus Truman were stupid and wrong for months and months, and they agreed with leading Democratic party officials, who I guess are some sort of Brain Trust or something. And this means that pollsters aren't stupid and wrong, exactly, how?


The truth may be out there, but lies are inside your head.--Terry Pratchett


[ Parent ]
translation (none / 0) (#58)
by aphrael on Thu Oct 02, 2003 at 02:45:02 AM EST

The story's author seems to be saying that this poll might be wrong and that's like the 1948 presidential election. I think the analogy is inapt.

[ Parent ]
I don't see that (none / 0) (#87)
by epepke on Fri Oct 03, 2003 at 03:55:33 AM EST

I think that the original article simply focuses on the 1948 Presidential election and Schwarzenegger as examples of a bigger problem.


The truth may be out there, but lies are inside your head.--Terry Pratchett


[ Parent ]
And I for one ... (3.90 / 11) (#19)
by Stavr0 on Tue Sep 30, 2003 at 12:59:03 PM EST

... welcome our new Austrian overlord. I'd like to remind them that as a trusted Kuro5hin user, I can be helpful in rounding up others to toil in his underground sauerkraut caves.
- - -
Pax Americana : Oderint Dum Metuant
Hey fucker (1.62 / 16) (#30)
by debacle on Tue Sep 30, 2003 at 09:45:07 PM EST

Dewey didn't win last time! Hah!

It tastes sweet.
Your grasp of irony is astounding. (5.00 / 1) (#41)
by Canthros on Wed Oct 01, 2003 at 03:11:13 PM EST

As is your reading comprehension.

--
It's now obvious you are either A) Gay or B) Female, or possibly both.
RyoCokey
[ Parent ]
Irony has nothing to do with it. (2.66 / 3) (#44)
by debacle on Wed Oct 01, 2003 at 03:31:42 PM EST

But thanks for that lesson in literary terms.

I didn't even read the article, by the way.

It tastes sweet.
[ Parent ]

Didn't read the article? (4.33 / 3) (#45)
by Canthros on Wed Oct 01, 2003 at 03:32:28 PM EST

Yes, that much was obvious.

--
It's now obvious you are either A) Gay or B) Female, or possibly both.
RyoCokey
[ Parent ]
your posts confuse me. (3.66 / 3) (#55)
by rmg on Thu Oct 02, 2003 at 01:02:04 AM EST

at first you question his reading comprehension, then you say it is obvious that he did not read. but if it was obvious that he did not read, then how could you have drawn any conclusions about his reading comprehension? do you see my problem?

i'm having some difficulty wrapping my brain around your posts in this thread. perhaps you could elaborate a bit on what you mean here.

_____ intellectual tiddlywinks
[ Parent ]

Regarding "leaning" voters (4.14 / 7) (#31)
by flamingweasel on Tue Sep 30, 2003 at 11:02:23 PM EST

I worked at a call center which handled a lot of election polls last year, and for almost every poll we had a question which went something like this: "In the last few elections, would you say you've voted for mostly Republican candidates or mostly Democratic candidates?" (Of course, the order was switched at random, and the wording was changed often.) When someone would try to say they were independent, we were told to push them to chose between the two, and mark them as "whatever leaning," or "somewhat whatever," or a half dozen other similar classifications. People like to think of themselves as independents, but statistically very very few people are honestly independent.

Umm, your conclusion doesn't follow. (5.00 / 4) (#36)
by dark on Wed Oct 01, 2003 at 08:25:01 AM EST

You just demonstrated bad statistics. You didn't accept "independent" as an answer, and ta-dah, not many people were "independent". Did you push people to change their description if they said "Republican" or "Democrat"?

[ Parent ]
Keep calm. Everything is not a conspiracy. (4.50 / 2) (#68)
by flamingweasel on Thu Oct 02, 2003 at 12:16:33 PM EST

Of course we weren't asked to change their description. That would be silly. The people who wrote the polls (not me) knew that few people were honestly independent. There are some out there, so the answer was accepted. But if someone was presented with the question, "Are you a Dem or a Rep," and said, "Uh, I'm In-dee-pendent," we were asked to question that. And nine times out of ten, the jackass on the other end of the phone, who had just so proudly declared themselves "Independent!", would admit to voting for a certain party in the last 5 (or more) elections they voted in.

[ Parent ]
The way I see it (5.00 / 1) (#74)
by CENGEL3 on Thu Oct 02, 2003 at 01:28:26 PM EST

The people who wrote the polls started out with an assumption (That few people were truely indpendent) and created a methodology that would support that assumption.

For instance, how did you define "voting for X party" in an election... most elections have multiple offices on each ballot. Did "voting for X party" mean to you that every candidate they voted for in that election (President, Senator, Representative, Governor, State Legislator, Mayor, Sheriff, etc) was a member of X party...or did you simply ask who they voted for President?

[ Parent ]

No more Independents (5.00 / 3) (#39)
by Weembles on Wed Oct 01, 2003 at 02:03:09 PM EST

So you are saying that unless a person votes exactly 50/50 Republican/Democrat then they are partisan?

Most people I know consider being an 'independant' as voting on each candidate based on his or her individual qualifications rahter than their party identification. If that means they vote for more Republicans, then so be it. Next year perhaps they end up voting straight Democrat. Or Communist, for that matter.

[ Parent ]

You just made the same mistake (none / 0) (#69)
by flamingweasel on Thu Oct 02, 2003 at 12:24:43 PM EST

... 90% of the people taking the poll make. Look at what the poll asks again. "In the last few elections, would you say you voted for mostly blah blah blah." It doesn't ask you whether you're a Dem or a Rep, it asks you who you've voted for, mostly, in the last few elections. If you find that you've voted for 9 Democrats and 2 Republicans in the last few elections, well then you've voted for mostly Democrats, and you should bloody well answer the question that way.

Claiming you're an independent is like a journalist claiming they're impartial.

[ Parent ]
And your mistake is... (5.00 / 1) (#75)
by Pihkal on Thu Oct 02, 2003 at 01:29:00 PM EST

...assuming that just because people are always forced to pick between apples and oranges in elections doesn't mean that they wouldn't rather eat something else from the produce section, if given the chance.

"I will not be pushed, filed, stamped, indexed, briefed, debriefed or numbered!"
-- Number 6
[ Parent ]
This is starting to piss me off. (4.50 / 2) (#84)
by flamingweasel on Thu Oct 02, 2003 at 04:31:22 PM EST

Look, for one I'm not my former employer. Secondly, I am essentially a radical; of course I know that some people would rather vote for a party other than the Dempublican Republocrats. I'm one of them. But if someone asks me if, in the last few elections, I've voted for mostly Democrats or mostly Republicans, I'm going to tell them I voted mostly Democrats. And that's totally okay. Why? Because it's true! I'm an independent in that I'd vote for a third party* if given the chance, but in general when forced to chose between Republicans and Democrats I chose the Democrat.

Finally, most people can barely put a sentence together regarding their political beliefs. They love the Rep/Dem labels because it makes the process a lot easier, and much more like that thar wrasslin' they love to watch on the TV.

*: Except in the next presidential election, in which I am going to be a yellow-dog Democrat.

[ Parent ]
I'd have screwed up your results... (5.00 / 1) (#42)
by Quantumpanda on Wed Oct 01, 2003 at 03:20:08 PM EST

In the last few elections, I've voted for no Republican or Democratic candidates. I've voted only for third-party or independent candidates since, oh, 1998, I think. (It's an issue of principle...I find the two-party dominance of the system repugnant, considering that the Constitution doesn't mention political parties at all.) How would your poll have handled that? (Probably, you'd have simply thrown my responses out...which invalidates the results.)

People are stupid. But we usually can't kill them, so we have to settle for the next best thing: we laugh at them.
[ Parent ]
Actually... (none / 0) (#70)
by flamingweasel on Thu Oct 02, 2003 at 12:37:41 PM EST

You'd be put in with the independents. No responses were simply thrown out unless the respondent was fucking with us. In states with large third parties, we would often have a (very small) quota for the largest third party.

And before someon jumps all over the word quota saying, "D00d! J00 just proved ur poll was statistically unsound!!!1!": Every area has a certain proportion of men and women, old and young, Republicans, Democrats, real Independents, and third party voters. We were given a table with all of these values on it (often on different axes, so towards the end of polls we'd need, say, male Republicans between the ages of 18 and 30). The quota was probably based on the last election's voter turnout and results. The computer kept track of all these boxes, and they would close as the quotas for those demographics would get filled as people took the poll.

Finally, a general response to all those who responded to the grandparent comment: Do you think the people who write the polls are stupid? Do you think they've not taken statistics courses, maybe even more than you? While the interpretation of the results is almost always flawed, and the base assumptions may be flawed (for example, assuming stable demographics from the last election until now), the methodology is probably pretty sound. They've been doing this for a while.

[ Parent ]
No (5.00 / 1) (#98)
by epepke on Sat Oct 04, 2003 at 12:06:38 PM EST

Finally, a general response to all those who responded to the grandparent comment: Do you think the people who write the polls are stupid? Do you think they've not taken statistics courses, maybe even more than you?

No. I believe what was recounted by my father, after sharing a few beers with someone whose last name began with "G" and had six letters in it and who was still in the Family Business: "Yeah, we give 'em what they want."


The truth may be out there, but lies are inside your head.--Terry Pratchett


[ Parent ]
Not so (none / 0) (#102)
by Brandybuck on Sun Oct 05, 2003 at 08:45:30 PM EST

You'd be put in with the independents.

Not so. In the past couple of years I can recall three telephone polls on upcoming issues where I was asked my political leaning (democrat/republican) and then when I answered "libertarian" or "other", I was thanked for participating and that there were no more questions.

There was one recent case involving a local issue where the poll lady flat out told me she couldn't use my answers because they didn't match any of her checkboxes. For example, "what's your opinion on the current city council, very favorable, favorable, neutral, unfavorable or very unfavorable" with my answer being "which member?" She admited she marked "neutral" on the form.

[ Parent ]

Your article's logic has a major flaw. (5.00 / 1) (#43)
by Quantumpanda on Wed Oct 01, 2003 at 03:30:05 PM EST

You appear to have made the assumption that a discrepancy between the proportions in the sample and the proportions of the population means that the selection methodology was flawed. This is not necessarily the case. A proper sample is selected entirely at random. The odds of the proportions matching exactly are slim, and the odds of a close match are only so-so.

The hallmark of a proper statistical sample is randomness, and randomness inherently includes the possibility of the sample ending up being nonrepresentative. That's why a single poll isn't very useful..its random nature could place its results at the extreme of the variance margin. Multiple polls, from different samples, will provide much more accurate results; the random selection of multiple samples helps minimize the effects of any one sample being skewed.

(Not to mention the fact that any poll of eligible voters that includes people who aren't registered to vote is not likely to accurately reflect the voting results...because people who aren't registered to vote don't vote! Their opinions on the matter are irrelevant to the vote totals.)

People are stupid. But we usually can't kill them, so we have to settle for the next best thing: we laugh at them.

Well there only is a single poll, so ... (none / 0) (#61)
by duelin markers on Thu Oct 02, 2003 at 07:58:42 AM EST

The hallmark of a proper statistical sample is randomness, and randomness inherently includes the possibility of the sample ending up being nonrepresentative. That's why a single poll isn't very useful.
Alright, well given that results are invariably published by news agencies based on single polls, shouldn't an effort be made to make the single sample representative? These polls are most assuredly NOT "proper statistics."

[ Parent ]
"proper statistics" (4.00 / 1) (#86)
by e4 on Thu Oct 02, 2003 at 10:12:38 PM EST

Alright, well given that results are invariably published by news agencies based on single polls, shouldn't an effort be made to make the single sample representative? These polls are most assuredly NOT "proper statistics."
The randomness can be overcome with a sufficiently large sample. Most polls include a margin of error. The margin of error is related to sample size. The larger your sample, the smaller your margin of error. Relying on a single poll can certainly be useful, if the sample is large enough and random enough to minimize skewed results.

One thing I always found fascinating about the statistics of opinion polls, is that margin of error has everything to do with sample size, and nothing to do with the size of the total population. This little counterintuitive formula means that a poll of 1600 randomly selected people has a margin of error of 2.5%, regardless of whether your conducting your poll for Peoria or for the entire United States.

Of course, you can still get bad survey results by asking leading or poorly constructed poll question, or just by chance.

Good info here if you feel like reading more about polling and statistics.

[ Parent ]

Almost.... (none / 0) (#99)
by Carter Butts on Sun Oct 05, 2003 at 04:30:45 AM EST

One thing I always found fascinating about the statistics of opinion polls, is that margin of error has everything to do with sample size, and nothing to do with the size of the total population. This little counterintuitive formula means that a poll of 1600 randomly selected people has a margin of error of 2.5%, regardless of whether your conducting your poll for Peoria or for the entire United States.

This is almost correct, but not quite. You are referring to the distribution of the sample mean for an infinite population; under fairly mild conditions, this is asymptotically normal with a variance which depends on the sample size, but not on the size of the population (which is good, because the population is by assumption infinite!). For random samples from a finite population, however, the variance of the estimator is indeed affected by sample size. The intuition for this is straightforward: if I were to repeatedly take a census of the entire population (for some strange reason), the sample mean would equal the population mean every time. Thus, the variance of the sample mean would have to be 0, which is most certainly not what the infinite population model would predict.

For those who are interested, the standard deviation (sqrt(variance)) for a the sample proportion (i.e., the mean of a dichotomous variable) from a finite population is

sigma = sqrt(p(1-p)/n * (N-n)/(N-1))

where p is the true proportion, n is the sample size, and N is the size of the population. Three things should be noted. First, as you would expect, the (N-n)/(N-1) term goes to 1 as N goes to infinity....so, for large populations, you are correct that the population size is essentially irrelevant. Second, even when this factor does matter, its effect is to deflate the variance of the estimator -- this means that neglecting it can only make our assessments of the standard error more conservative, not less. Third, and finally, observe that the standard error here depends on the true proportion, p. Thus, the mean for a random sample of size 100 will have a different standard deviation if 5% of the population agrees with a poll item than if the agreement rate is 50%.

In practice, one usually ignores the finite population correction unless the sample size is roughly comparable to the total population (e.g., n/N>5% or so). (If it really affects your results otherwise, you're probably up to no good anyway! :-))

That was far more than anyone probably wanted to know about this, but here's my consolation prize: you can use the above to construct your very own confidence intervals! By the central limit theorem, the distribution of the sample mean (for a proportion) is asymptotically normal with mean given by the true mean and standard deviation given by the expression for sigma above. Thus, if p is the observed proportion for the population, p +/- 1.96*sigma is an (approximate) 95% confidence interval for the sample proportion. Great for impressing your friends and neighbors....

-Carter

PS. Of course, if you actually wanted a 95% probability interval for an unknown proportion -- like a good Bayesian -- you'd want to use the 2.5% and 97.5% quantiles of a Beta(np+0.5,n(1-p)+0.5) distribution (assuming a Jefrey's prior). This is for the infinite population case.

[ Parent ]

The Dewey fiasco (2.16 / 6) (#46)
by ucblockhead on Wed Oct 01, 2003 at 05:48:58 PM EST

The reason for the bad polling data in Dewey vs. Truman is fairly simple. (And it was at a time when polling was in its infancy.) The polls were conducted over the phone. Well, it turns out that in 1948, the telephone was not yet ubiquitous in American homes. In particular, rural voters were less likely to have them. Truman's support was mostly rural, Dewey's mostly urban. Hence the screwed up poll results.

In terms of this topic, well, I think any op-ed on this subject is in danger of looking silly next Wednesday. Wait for the real results, then deconstruct the polls.
-----------------------
This is k5. We're all tools - duxup

Ach! Please read article before posting. (5.00 / 1) (#47)
by jmzero on Wed Oct 01, 2003 at 06:13:03 PM EST

In article: ...Dewey won among households that had telephones in 1948. Unfortunately for Dewey, in 1948, not all households had phones...
.
"Let's not stir that bag of worms." - my lovely wife
[ Parent ]
Ah! (none / 0) (#50)
by aphrael on Wed Oct 01, 2003 at 07:33:38 PM EST

But you're saying you think the polls could be wrong - without providing any theory that explains how they're wrong and what fundamental chunk of the electorate they're missing.

[ Parent ]
Well... (4.00 / 1) (#67)
by jmzero on Thu Oct 02, 2003 at 12:10:22 PM EST

But you're saying you think the polls could be wrong - without...

I didn't intend to say anything subject related, my point was about the parent comment.

And I don't deny that his post added some information to that available in the article.  However, I think it's fairly clear from his wording that he hadn't noticed that the main point had been included in the article.  

Mostly I just hate to see the creeping /.'ism (posting whatever comes to mind without reading the article or other comments) invade K5.
.
"Let's not stir that bag of worms." - my lovely wife
[ Parent ]

Phone Ownership Was Party-Determined (none / 0) (#82)
by John Fulwider on Thu Oct 02, 2003 at 03:31:50 PM EST

Coincidentally, we just talked about the Dewey Defeats Truman headline last night in class. According to my professor (and how could she be wrong!), the poll results were skewed in Dewey's favor because Republicans (presumably Dewey voters) owned the majority of phones. So not very many phones from which to form a sample, and most of the phones in the bad sample were owned by members of one political party.

[ Parent ]
LA Times confirms. (5.00 / 1) (#49)
by aphrael on Wed Oct 01, 2003 at 07:32:15 PM EST

When the CNN/Gallup poll came out, California poll watchers were scratching their heads; it was wildly at odds with what we had been seeing from other agencies. It seemed like an outlier with a possibly mad methodology (their percentages of registered voters were nonsensical).

The LA Times released a new poll today which mostly confirms the results, with a better registered voter breakdown. It could still be wrong, but there are now two polls that show the recall winning by a large margin and Schwarzenegger stomping the opposition.

I was startled by the polls a week and a half ago that showed Davis with a chance of pulling out; this jibes much more with what I - in liberal Santa Cruz - sense as the mood of the electorate.

Polls and expectations (none / 0) (#51)
by Yanks Rule on Wed Oct 01, 2003 at 08:21:36 PM EST

From what i've been seeing, from Instapundit, Kausfiles, and Wientrab at the Sacremento Bee is that the earlier LA Times poll was complete BS, and that the LAT has been steadily switching its stance. Today's LA Times (or maybe yesterday's?) now says that the recall has gained momentum, when the other polls have been reletively constant.

So i wouldn't be to quick to trust the LAT...Personally, I used to like the LAT, but now it just disgusts me that they are such liberal lapdogs. Even their news articled are heaviliy biased.

"I do think we live in dangerous times, and anybody who looks at the world and says this is the time to be a wuss--I can't buy that anymore. " -- Dennis Miller
[ Parent ]

I'm unconvinced. (none / 0) (#57)
by aphrael on Thu Oct 02, 2003 at 02:37:36 AM EST

The earlier poll was reasonably consistent with the Field poll that had preceded it, except that it showed (a) slightly lower support for the recall, and (b) massively greater support for McClintock. Both seem to be reasonable results of sampling error.

It looks like the trend from the start was anti-recall and pro-schwarzenegger -- eg., the recall's numbers steadily declined until last week, as did Bustamante's. But then the recall numbers picked up; I wonder if the campaigns have any idea why?

[ Parent ]

One possibility (none / 0) (#72)
by cestmoi on Thu Oct 02, 2003 at 01:17:52 PM EST

But then the recall numbers picked up; I wonder if the campaigns have any idea why?

I'm not a campaign but here's one interpretation. It's possible that Bustamante's sleaziness vis-a-vis trying to justify the $4 Million donation coupled with the court order telling him to give it back coupled with his "I can't, I've already spent it" defense pushed a lot of fence sitters into the pro-recall camp. The Democratic leadership has tried to portray the recall as a right-wing coup and, in so doing, fail to grasp the point that a lot of us in the middle don't like the sleaze that goes with campaign funding. Davis and Bustamante epitomize the very worst of it.

I'm voting for Arnold, not because I think he's any great shakes, but because I want Sacramento to wake up to the idea that politicians are not there to enrich themselves - they're there to govern. If they can't understand that, they don't belong there. To me, that's what the recall is about.

[ Parent ]

Arnold's campaign (5.00 / 1) (#73)
by aphrael on Thu Oct 02, 2003 at 01:22:06 PM EST

Arnold's campaign is doing something absolutely evil that ought to be outlawed: it's borrowing money that won't be repaid until after the election. IE., once he's elected, he can solicit donations to the campaign which will be used to repay the loan. But of course he's not going to be beholden to those people in any way, and they aren't going to be trying to buy influence with the new governor.

Yuck.

[ Parent ]

Two ways to read it (none / 0) (#77)
by cestmoi on Thu Oct 02, 2003 at 02:30:02 PM EST

There are two ways to read the loan.

One way is that Schwarzeneggar's short of cash and went to the bank to borrow some. Just because he makes $20 Million per movie doesn't mean it's all lying around in cash. If he repays the loan out of his own pocket, the loan is not evil.

If he does as you presume he's going to do and he pulls a Davis Strong Arm Manuever, then I agree, he's as bad as Davis.

Until the loan's repaid, I'll withhold judgement.

[ Parent ]

The sleaze factor... (none / 0) (#91)
by geoswan on Fri Oct 03, 2003 at 04:39:46 PM EST

OK, I am another foreigner. So explain this to me... Didn't this whole recall thing happen because one guy in Congress spent 20 000 000 $ to finance a petition to initiate the recall? Doesn't that strike you as sleazy?

[ Parent ]
Not quite (none / 0) (#94)
by cestmoi on Fri Oct 03, 2003 at 11:47:37 PM EST

The recall started because an individual, Ted Costa, initiated the process. The recall was moving slowly forward when Darrell Issa, a congressman, started a parallel recall campaign and infused it with $2,000,000 of his own money. That's two, not twenty, million. Presumably, Issa funded the recall expenses because he thought that funding the recall would give him a good chance at winning the governorship if Davis was recalled. When Issa's past problems involving stolen cars caught up with him, he withdrew from the race but the recall went on anyway. Much to the dismay of Davis's shill, S. Issa, who plunked down $3,500 to get his name on the ballot and promptly vanished.

The thing is not a lot of us have a couple of million dollars we can spend on a recall - Issa did have it and he spent it. However, once it looked like the recall wasn't going to fail the way the 31 previous recalls attempts did, people came out in droves to support it. Issa had funded the recall but people donated anyway.

Davis is easily one of the most corrupt governors this state has had the misfortune to have. Fortunately, we can do something about it.

[ Parent ]

Sigh (none / 0) (#101)
by Brandybuck on Sun Oct 05, 2003 at 08:23:44 PM EST

Sigh. The thing I hate the most about the US media is the false perception it gives foreigners about the US people.

The money donated by Darrel Issa paid for a lot of signature gatherers, but it didn't buy one single signature, as the US media implies. Here's how a recall (or initiative or referendum works). You need to gather a certain amount of valid sigatures from registered voters. Petitions are printed up and circulated. I can go get a petition and have five friends sign it without spending one dime. Or I can hire someone to sit all day in front of a grocery store and collect a couple hundred signatures. In no case is anyone paid to sign anything. The time period needed to collect these signatures is simply too short for any non-funded effort to succeed. What Issa's money paid for was these people to sit in front of grocery stores all day long. What is amazing is that about 40 to 50% (can't remember exact number) of the valid signatures collected in this recall did NOT come from paid gatherers.

Issa didn't even start this movement. People were pissed at Davis for months prior to his money. I heard talk of recall long before Issa entered the picture. This was truly a grass-roots effort at recall, which only succeeded in the allotted time frame because a few people, such as Issa, donated enough money to finance signature gathering.

[ Parent ]

Echoing a previous thought (5.00 / 2) (#52)
by Joh3n on Wed Oct 01, 2003 at 09:29:48 PM EST

I would argue that the rise in republican numbers is recall-centric, since the majority of people voting pro-recall will be republican ore republican-leaning.  Moreover, I would not be shocked to learn that a significant portion of people are registering now simply for the chance to vote yes on the recall, and realized they needed to do so before a certain last minute date, thus the sudden upswing in registered voter numbers.

------------------------------------------
It must be nice to glide on your percieved intelligence and pop culture references.
-yankeehack

STOP! (1.85 / 7) (#54)
by rmg on Thu Oct 02, 2003 at 12:29:56 AM EST

hammer time.

_____ intellectual tiddlywinks

This happens ALL THE TIME (5.00 / 4) (#56)
by Estanislao Martínez on Thu Oct 02, 2003 at 01:12:33 AM EST

Lots of polls should be discounted as utter bullshit if it weren't for the fact that they have an effect on the voting population.

A classic example: Proposition 227 in California, banning "bilingual education". A lot of media, based on polls, claimed that Hispanic voters in CA were in favor of the measure. Exit polls on election day showed that this was wrong-- 63% voted against. Yet a few outlets still claimed afterwards that there was majoritary Hispanic support.

The point is simply that the pollsters don't accurately sample many ethnic populations.

--em

So? (none / 0) (#100)
by Brandybuck on Sun Oct 05, 2003 at 08:04:43 PM EST

So why shouldn't we discount that Prop 227 exit poll as well?

[ Parent ]
I always hated it (2.62 / 8) (#59)
by loorihc on Thu Oct 02, 2003 at 04:18:38 AM EST

I always hated the dewy decimal system. I always though the library should just put the books in aphabetical order.

Conan the Librarian reference? (none / 0) (#88)
by StingK on Fri Oct 03, 2003 at 09:11:39 AM EST

Is it?

[ Parent ]
Gut instinct (3.00 / 4) (#60)
by Eight Star on Thu Oct 02, 2003 at 07:51:48 AM EST

I think that the "Terminator" will "win" the "election". If you get mey drift...

You are a dork (3.50 / 6) (#80)
by benzapp on Thu Oct 02, 2003 at 02:57:51 PM EST

Is the inclusion of your stupid quotation marks supposed to indicate some level of sarcasm?

If so, all it makes you look like is a fucking tool.

[ Parent ]

Is english your second language? (none / 0) (#89)
by nxor on Fri Oct 03, 2003 at 10:26:54 AM EST

Because that's not why the quotes are there.

[ Parent ]
Ok I am an idiot (none / 0) (#95)
by gmol on Sat Oct 04, 2003 at 05:04:45 AM EST

I don't get the quote thing if it doesn't stand for sarcasm...

[ Parent ]
What does "registered" mean? (5.00 / 1) (#62)
by am3nhot3p on Thu Oct 02, 2003 at 09:11:49 AM EST

As a non-American, I'm always puzzled by the concept of registered Republicans and Democrats. It always comes up in US political discussions, but is never explained. Is this analogous to party membership in other countries? What does it mean?



Registered voter (3.50 / 2) (#64)
by cestmoi on Thu Oct 02, 2003 at 10:02:48 AM EST

To vote, you have to fill out a form that gives your current address and verifies that you are a U.S. citizen. Filling out the form puts you in a group called "registered voters."

One of the questions on the form is whether you care to state an affiliation with any political party. If you choose a party, say Democrat, then you're a "registered Democrat." The question makes it easier for the political parties to target mail and phone campaigns. It also binds your choice in the primary election.

Primary elections are the rather backward way we go about winnowing a large field of candidates. Each party nominates several candidates for an office and the primary election serves to choose the party's single nominee for the general election. If you've registered as a Republican, you can only vote for Republican candidates in the primary. Independents, such as me, don't get to vote for anyone until the general election which is held after the primary.

The party labels don't mean much except for the extreme members. A moderate Democrat may vote for the upcoming recall in California as a moderate Republican could vote against it. An independent, like me, may end up feeling very strongly that the recall is an excellent opportunity to rid the state of a corrupt governor.

[ Parent ]

The system exists (5.00 / 1) (#71)
by aphrael on Thu Oct 02, 2003 at 01:01:56 PM EST

The system of registration exists in part for pragmatic reasons. It allows us to say "go *here* to vote" and the people at that place to have a list of who is supposed to vote there; makes it easier to prevent double-voting, etc.

This wouldn't be required if we had some form of state-required registration of residence, as a lot of European countries do.

[ Parent ]

UK too (none / 0) (#81)
by am3nhot3p on Thu Oct 02, 2003 at 03:31:48 PM EST

My question wasn't about registration per se, but with the concept of registration for a particular party.

In the UK, too, you have to register to vote. They send a form to your house in the autumn, and you fill it in with the names of people who are eligible to vote and live at that address. (You can also update your registration at any time in the year now, I believe.)

When I was at university in student accommodation, the university dealt with voter registration, and did a very bad job of it. My Malaysian flatmate, who wasn't even allowed to vote, was given a polling card for the general (i.e. national) election with which he could have turned up at the polling station and voted, albeit illegally.



[ Parent ]
party registration (4.00 / 1) (#83)
by aphrael on Thu Oct 02, 2003 at 04:01:44 PM EST

It used to be, back in the day, that the leaders of the political parties got together and selected a candidate, behind closed doors, with no input from the voters. This was an irritating practice, and helped maintain widespread corruption, and was particularly objectionable in states like California where it was next to impossible (due to political emotions) for members of another party to get elected.

As part of a series of reforms adopted in California in the early 20th century, and elsewhere at various different times from the late 19th century through the middle of the 20th century, a system was set up whereby the nominee of the political party automatically got on the ballot in exchange for the party agreeing that its nominees would be chosen through a public vote, administered by the state, of people affiliated with that party. That system is called a 'primary election'; in order for it to work, the state has to know what party you're affiliated with. There have been some experiments with tweaks to this system (for example: a blanket primary, in which everyone gets the same ballot on primary day regardless of their party, and can vote regardless of party, and the top vote getter in each party goes to the general election; also, cross-filing, in which the same candidate could run in the primaries of multiple parties), but most have either failed or been invalidated by the courts.

[ Parent ]

It's different in New Jersey (none / 0) (#85)
by landtuna on Thu Oct 02, 2003 at 05:12:07 PM EST

I think this might vary by state. In New Jersey, you don't affiliate yourself with a party when you register to vote.

You're only affiliated with a party when you first vote in that party's primary or when you submit a special affiliation form.

But not many people are affiliated here since our primaries are so late they're pointless.

[ Parent ]

That doesn't sound right. (none / 0) (#93)
by molo on Fri Oct 03, 2003 at 09:21:27 PM EST

I think this might vary by state. In New Jersey, you don't affiliate yourself with a party when you register to vote. When I registered to vote in New Jersey, I went down to the town hall and had a choice of party affiliation or Independant. -molo

--
Whenever you walk by a computer and see someone using pico, be kind. Pause for a second and remind yourself that: "There, but for the grace of God, go I." -- Harley Hahn
[ Parent ]
You check a box when you register to vote (5.00 / 1) (#76)
by epepke on Thu Oct 02, 2003 at 01:44:52 PM EST

You have to register beforehand to vote. When you do, you get to check a box for one of the major parties. This means that you can vote in primary elections, that is, elections where members of one party are running against each other.


The truth may be out there, but lies are inside your head.--Terry Pratchett


[ Parent ]
youi don't have to check the box of a party.. (none / 0) (#92)
by molo on Fri Oct 03, 2003 at 09:15:01 PM EST

You can register as an Independent, beloning to no party.  However, this precludes you from voting in primary elections in most states.

-molo

--
Whenever you walk by a computer and see someone using pico, be kind. Pause for a second and remind yourself that: "There, but for the grace of God, go I." -- Harley Hahn
[ Parent ]

True, you don't have to (none / 0) (#96)
by epepke on Sat Oct 04, 2003 at 11:23:25 AM EST

But what's the fun in that?


The truth may be out there, but lies are inside your head.--Terry Pratchett


[ Parent ]
Registration (5.00 / 2) (#90)
by Brandybuck on Fri Oct 03, 2003 at 03:44:10 PM EST

You register to vote so that you can vote. A tautology almost. It prevents Oregonians from voting in California, or even worse, Californians voting in Oregon.

As for registering for a party, that's an anachronism that should be done away with, in my never humble opinion. Its purpose is to determine what ballot you get in primary elections. For those of you outside the US, a primary election determines which candidate a particular party puts forth for the general election. Everyone registered Republican, for instance, gets to vote to see who the Republicans will run for governor, senator, president, etc.

But political parties are in essence still private organizations. Only the members of a party should have any say-so in who their candidate will be. But long ago the Democrat and Republican parties co-opted the electoral system to make these decisions for them. But it's not the law, though most people think it is. For example, some smaller third parties don't use the primary election to decide their candidates.

In California it gets even weirder. Because of a recent initiative process, anyone registered with any party can vote for candidates in any other party. Thus Democrats can vote for who they want the Republicans to run, and vice versa. Imagine the Sun board of directors getting to decide IBM's CEO, and you see the absurdity of it.

The current California election didn't have a primary, which is why there are so many candidates. Whether this is good or bad is a matter of opinion.

[ Parent ]

Should this story exist? (none / 0) (#65)
by cestmoi on Thu Oct 02, 2003 at 10:06:52 AM EST

Color me surprised when this story posted. The 36 hour window closed and it didn't have the requisite 90 votes. Have K5 posting policies changed or did Scoop poop?

95 votes sufficient, but not necessary (none / 0) (#66)
by truth versus death on Thu Oct 02, 2003 at 11:34:30 AM EST

You should probably post this question in an editorial comment, not topic.

Anyways, read this.

"any erection implies consent"-fae
[ Trim your Bush ]
[ Parent ]
How? (none / 0) (#78)
by cestmoi on Thu Oct 02, 2003 at 02:35:12 PM EST

Thanks for the link - I didn't know the rules had changed.

BTW, how do you post as an editorial comment once an article is posted?

[ Parent ]

Oops (none / 0) (#79)
by truth versus death on Thu Oct 02, 2003 at 02:46:02 PM EST

Heh. I dunno. :-) I guess you have to reply to one already there.

"any erection implies consent"-fae
[ Trim your Bush ]
[ Parent ]
Not sure about your Dewey reference... (none / 1) (#97)
by leviramsey on Sat Oct 04, 2003 at 11:34:43 AM EST

Dewey won among households that had telephones in 1948. Unfortunately for Dewey, in 1948, not all households had phones.

Have a link to back that assertion up? In the 1936 election, Literary Digest ran a telephone poll of their subscribers which caused them to predict a massive defeat of Roosevelt by the Republican candidate (whose name escapes me). In the aftermath of that election, pollsters realized that telephone polls might not be the most reliable polls out there.



Dewey Wins Again? | 102 comments (78 topical, 24 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest © 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!