Nate Silver and the rest of the 538.com crew have just finished thoroughly eviscerating the Republican polling firm Strategic Vision. There have been a number of less serious anomalies with Rasmussen as well (though Rasmussen had an excellent record with the 2006 and 2008 races). And not too long ago, Stu Rothenberg was caught mouthing GOP talking points about the race in NY-20. Rothenberg has also made a number of other comments (here; here) that make it clear he leans Republican personally.
So here’s something interesting: Rothenberg is currently predicting a small number of losses for Democrats in the House (it looks like a dozen or less from this) while Charlie Cook says there’s a 50-50 chance that Democrats will lose 40 or more (Rothenberg, by contrast, shows only a total of 31 Democratic seats as being at all in play and most of these he ranks as relatively safe).
I find the intersection of political prognosticating and political messaging to be a fascinating place. I think that most of the major prognosticators (Cook, Rothenberg, Sabato — with obvious local exceptions, previously Chuck Todd) play it straight with their predictions (if not their other comments) for the simple reason that to do otherwise would hurt them professionally. But politics is full of self-fulfilling prophecies, people like to back a winner, fundraising depends on the perception of how likely a candidate is to win a race, and so on. There’s an interesting tension between trying to accurately predict things (as, say, Carville/Greenberg’s Democracy Corps group does or, be bipartisan here, Mike Murphy generally does) and saying crazy stuff about having “the math” that you think will help your side win.
I often wonder what someone like Michael Barone thinks he’s accomplishing by spewing right-wing nonsense when he could be using his former respectability to further his political agenda.
Cat Lady
I’ll take your word on Nate Silver. They lost me at Fourier analysis.
The Village has never cared about right and wrong which makes the line between prediction and opinion all the more meaningless. Pollsters are just part of the cable TV clown show now. If I were a pollster, I wouldn’t want Nate Silver mad at me though.
gizmo
I don’t see any value in polling House seats this far out from the election. By the time we vote, healthcare may or may not have passed, we have no idea what the economy will look like, Afghanistan is still a question mark, etc. Polls taken now are useless, IMHO.
Just Some Fuckhead
Predictacating is hard work!
r€nato
@Cat Lady:
yeah I am not afraid of math and I got solid A’s in the two statistics classes I took in college…
but my brain fuzzed out around there. I could probably grok it if I spent a half-hour or so re-reading it… but I think I’ll take his word for it.
Just how does one use something that’s so deep into the weeds, to prove one’s point to other, non-statistics geeks?
ThatLeftTurnInABQ
What I find more interesting is the role played by relatively anonymous polling firms as hired hit men for push polling. Pundits may have some small degree of reputational risk to worry about but I don’t think too many polling firms are going to be burned in a lasting way for what is essentially contract work slinging propaganda. Yet another way that our political environment today is toxic and will continue to be that way until the peasants don’t feel like dancing to the tune dictated by their betters.
GambitRF
OT but I have to vent about this
hotair.com/archives/2009/10/06/the-white-house-department-of-white-coats/
New wingnut meme: Actual doctors wearing doctor’s coats is bad for some reason.
Hot Air calls it a “masquerade party” and on Malkin’s site the headline is “Halloween came early to the White House.” Nothing says Halloween like doctors dressing like… uh… doctors.
Now the Wall Street Journal has one of those whining “why did the liberal media cover up this important angle to the story?!” pieces.
No one ever questions that all of the doctors present are actually doctors, they just somehow find it objectionable that the ones that came in business clothes were given doctor’s coats, that they presumably have all worn while being practicing doctors. If you’re outraged at this, don’t you have to be outraged at like, basketball teams taking pictures for yearbooks. “What?! They’re not playing a game, but their in their uniforms!! What visual trickery is taking place here! Halloween has come early!! My God, the fascism!!”
stickler
I agree with Gizmo. The November 2010 elections will be decided on issues that haven’t been settled yet, here in October 2009. If Congress passes a piece of crap healthcare bill, and joblessness is still north of 10%, it’s going to be an ugly election for the Democrats. If Congress passes a decent healthcare bill and unemployment is coming down, it’s going to be ugly for the GOP.
Too. Soon. To. Tell.
geg6
Wow. Been a long time since doing a Fourier analysis, but this looks pretty damning. Essentially, to get those results, SV pretty much has to be cooking the books. Which doesn’t surprise me since they refused any sort of transparency when asked. Nate’s been like a bulldog on this. If I were the other pollsters, I’d be watching my ps and qs now that Nate has taken such an interest in politics.
ricky
A solid majority of my friends think pollster have motive.
Others say, when prompted, aspersions are easy to cast, but often tricky to sign.
General Winfield Stuck
Scott Rasmussen is a good pollster. But he has two modes. One, like now, in between elections, and the other when the election campaign begins.
The in-between mode is nothing less that to provide faux grist for the RWNM, mostly to buck up the troops and give good news to Fox News and RW bloggers to flog to maintain the image of win.
But when the chips are down in the months leading up to elections, Rasmussen switches to reality polling which is all about self interest keeping up the reputation of reputable.
Strategic Vision has always been a slimy outfit astroturf like push pollster for wingnuts to pretend they are doing well.
We are now in between the Bush disaster and a shiny new presnit and big dem majority and the public is in a wait and see mode, especially on job creation. I would say nobody knows which way it will break until spring of 2010, and Obama doesn’t have to bring a state of economic nirvana. He just has to show things are getting better, which will mean some months of minor but increasing job growth.
But even if the job picture doesn’t measure up to that standard, the dark shadow of Bush will temper any landslide possibilities for the right, when the ad wars start.
The only way, I think, for the dems to lose big is if we hit a double dip recession next year. And then there is the true wild card that wingnuts go full Wolverine and scare all the children into voting dem regardless.
timb
@GambitRF: Gambit, it’s one thing to use doctors as willing props for a photo op. That’s a bad thing.
It’s another to use men and women on an aircraft carrier as a photo op against their will and without their permission, to stage manage the arrival of the carrier, to place political messages on the carrier, to have the president play dress up (instead of the audience), and then to use the photo op to fundraise to the wingnut crowd who imagined the President is a heroic leader. That’s a good thing.
Add in the fact that one was for a war fought with borrowed money with disastrous results and the other was for a public good, which ideally pays for itself, and you once again understand why the war photo op is good and the healthcare photo op is bad.
GambitRF
@timb:
Well, yeah. The Bush photo-op only would’ve been outrageous if some of the military personnel were in coat and ties instead of uniforms and they were handed out uniforms before hand. That would’ve been halloween coming early! Bush in a flight suit is nothing of the sort, though.
bago
So what algorithms and datasets are being used? I know how to apply a FFT to a massive data set, thread it, and farm jobs out to multiple servers, although thanks to a brilliant engineer at DoubleClick, I only had to deal with 32 bit integers, and not all of that floaty crap.
bago
@Just Some Fuckhead: Predicate.
Zifnab
That only works so far. You can come in and declare yourself the new Oracle of Congressional Politics, but if your predictions are completely off base, you discredit yourself irregardless of partisanship.
I remember hearing how John McCain was totally going to win in a landslide, right before Mitt Romney was going to sweep the primary, which was – in turn – right before Rush Limbaugh and Bill O’Reily assured us that Nancy Pelosi was going to cost the Dem House dozens of seats.
Nate Silver got on the map specifically because he was uncanny in predicting voting trends. Did any of his trend lines change because of the “Nate Silver Effect”? I mean, I think it really just boils down to money. The GOP isn’t going to pay for a poll that shows its candidates are chronic losers. They pick their candidates in smoke filled back rooms and worry about winning after the fact. By contrast, the netroots isn’t interested in backing loser candidates. They have to support their candidates very publicly from the ground up. Phony polls showing a netroots candidate in the lead, when the candidate doesn’t have a snowball’s chance in hell, doesn’t benefit DKos or FireDogLake or any of the others in the long run.
Sentient Puddle
@bago: The data set is the last digit of all the poll numbers Strategic Vision had. You’d have to read the 538 thread to see the algorithm, because I only know what Fourier transforms are at a very high level.
He also provided the source code. Thing is, he also wrote it in BASIC. So your eyes might bleed.
geg6
OT, but the GOP rolls on in its mission to alienate every constituency except white, male southerners:
nrcc.org/news/read.aspx?id=863
Money quote:
If Nancy Pelosi’s failed economic policies are any indicator of the effect she may have on Afghanistan, taxpayers can only hope McChrystal is able to put her in her place.
geg6
@geg6:
Block quote fail. Sorry.
bago
Yeah. Using ‘let’ to allocate memory hurts my syntactic synapse. It’s a proposition, not a declarative!
bago
Alright. Last number in an arbitrary enumeration scheme scares me. Reading the thread now.
joe from Lowell
Rasmussen didn’t have an excellent record in the 2006 and 2008 races. In the 2008 election, he ran absurdly pro-Republican numbers throughout the campaign, at odds with everyone else except the Zogby LOL/Internet poll, and then at the last minute, when there wasn’t any more benefit to making McCain look stronger than he actually was, he “discovered” a huge swing to the Democrats that no one else picked up, and viola, he ended up with accurate final numbers.
I suppose it’s possible that the extreme pro-Republican outlier was actually the most accurate pollster throughout the campaign; and I suppose it’s possible that there really was a huge swing towards the Democrats that Rasmussen, and only Rasmussen, picked up in the last five days; and I suppose it’s possible that it’s merely a coincidence that these two events had the effect of bringing the extreme outlier poll in line with what every other pollster had been saying for weeks; and I suppose it’s possible that every other pollster was wrong in stating that Obama had a 3-5 point lead, and Rasmussen was the solitary voice of reason saying McCain had a slight lead or was tied throughout October – but it’s a lot more likely that Rasmussen was just skewing his numbers to boost McCain for most of the campaign, and then at the last minute posted accurate numbers to maintain his reputation.
General Winfield Stuck
@bago:
Not for Word Explorers.. We come from a dictionary far far away.
Zach
Hah, Weissman was a professor of mine some time ago. Not all that surprised to see his name pop up in a quasipolitical context.
Martin
You should see their countertops…
Sentient Puddle
Well, as far as Rasmussen goes, we know that a big part of the tilt of his polls now is that he’s using a likely voters model. That obviously doesn’t make any sense (unless he were polling, say, Virginia or New Jersey), and I recall reading somewhere that his likely voters model skews fairly conservative. All this goes to explain his off-year skew.
My curiosity is whether or not he rejiggered the model towards the end of the campaign last year. Nate seems to have a pretty good opinion of his polls, so I would be tempted to chalk up weird numbers to methodology rather than just making shit up.
Punchy
You just listed about 20 names in that post, and besides Nate, I know absolutely none of them. Your posts are becoming greater and greater inside-baseball-ish. Boo.
General Winfield Stuck
@Sentient Puddle:
If you recall, during the years of Bush’s poll descent to become cast in cement at the bottom, Rasmussen was pretty tracking with other polls. He then announced he was using a different weighting formula, but didn’t say what it was, and overnight his Bush numbers jumped up 6 or 7 points and stayed their well above any other polling, at least until 2008.
bago
Wait, after reading the 538 thread this. Wow. Math geeking time. And yes, seeing dim hurts my brain. Line numbers too. What is this, 1982? This is some Bill Gates personally coded this stuff type of deal.
Man, I wasn’t even born when most of this stuff was happening.
Augustine
@bago:
predicatepreteritejharp
I highly recommend the gamblers visit intrade.
The odds don’t seem to match up. I believe there is some real deals being offered.
Asking price is 90 for a 20 seat gain. Or 75 for a 25 seat gain.
Something Fabulous
@timb: I feel foolish; my mind went immediately to Oliver North, back in the day, and the observation that in his day to day job at the Pentagon he wore a suit, but once it was time to testify, out came the uniform. It made me wonder what moment they could be attempting to avenge with this, “look they do it too!” tone.
And here is this much more recent, much more obvious (and damning) comparison I totally overlooked. I must be getting old.
Sentient Puddle
@General Winfield Stuck: I wasn’t a polling geek back then, so I wouldn’t have been able to tell your Rasmussens from your Research 2000s from your Zogby Interactives (OK, that last one is a lie). So I wasn’t seeing the discrepancy and couldn’t say why there was that much of a difference.
I guess what I’m getting at here is that you can’t just assume the pollsters have their fingers on the scale. The methodology may be defensible somehow (even if it looks silly for whatever reason). In these cases, it’s better just to make a mental adjustment rather than cry foul.
(And yes, everything I’m saying here is basically a crib of Nate’s house effect post. Honestly, I think that interpretation makes the most sense.)
bago
@Punchy: : So “Good Will Hunting” was about Fourier Transforms. Knowing these things is like being handed the keys to the universe. I would advise studying in your free time. It will be quite elucidating.
Cain
I don’t quite understand why you would vote republican for an alternative? I mean it’s not like the republicans have a plan all they’ve been saying is FAIL! FAIL! FAIL! OMG!!! Sadly, if democrats lose seats it is because they fail to uniformly state this. Keep challenging them to come up with a plan. Make them cough something up.
Now, if there were some good independents then definitely I would vote them for alternative especially if they are going to get things like healthcare. More liberal politicians please and less two faced democrats.
cain
General Winfield Stuck
@Sentient Puddle:
Man, was I poll geek back then during Bush. Bordering on obsession. I don’t read them much any more, just a little to get a basic trend. As far as I can tell, Obama’s polling is pretty much standard fare for a new presnit. First the honeymoon and then back to basically something close to the election vote percentage. I would note that along with a press, that in general is looking to create news these days, instead of report it, it has surprised me that pros like Cook are apparently being assimilated into that circus in trying to make some drama when it is way too early to start predicting what will happen next election in 010.
General Winfield Stuck
@bago:
Outfuckingstanding!
bago
@General Winfield Stuck: From here I could take the Lucent route(SATAN!), or I could say BrilFuckingTastic.
Mark S.
This has got to be my favorite prediction so far:
Using MichelleMalkinMath, I’m predicting that the Democrats will lose 10,000 seats, and the new Republican majority will impeach Obama 1,100 times.
General Winfield Stuck
@bago:
The word world is a magic realm of free floating syllables.
catclub
Fourier transforms – pish!
How about some operations on Galois Fields?
Maybe some Hilbert transforms?
Remember John Cleese speaking Russian as an aphrodisiac
in “A Fish Called Wanda”?
General Winfield Stuck
@Mark S.:
I don’t want to even think about what will happen to ACORN. Nobodies counter top will be safe.
Paul L.
The Democratic will gain seats in Congress in 2010.
Keep believing that and having the mainstream media push that narrative.
It will make for some amusing nutroots outrage when the Democrats get teabagged in the 2010 elections (like 1994) .
Sentient Puddle
@General Winfield Stuck: Oh yeah, polling knowledge or not, I can agree that the media (the press and the horse-watchers) aren’t doing us any favors by throwing around this “Democrats could lose the House!” forecasting shit. If they’re going to bother forecasting, they should at least add some obvious context (like, say, the fact that a large part of health care will be considered resolved well before November ’10)…
smiley
@r€nato:
Knowing about the central limit theorem from my statistics course for my masters degree got me exempted from 2 semesters of statistics in my PhD training. (Damn, I wonder if the PhD students at that university were even exposed to it. Oddly, I’ve never asked any of my graduate school friends.) Anyway, I shy away from statistics because my work only requires basic stuff but sometimes I wish that instructor had made me take the courses. All that said, there are lies, damn lies, and statistics. I came up with that on my own 8^).
LD50
@Paul L.: Paul L seems a little more pathetic than usual today…
General Winfield Stuck
@LD50:
A cry for help
Bobby Thomson
@Zifnab:
There was nothing at all “uncanny” about it. Poblano’s great at interpreting data, but then, so is Chuck Todd. What people forget is that 538 didn’t actually do any of its own polling. It interpreted the polls that other people did in a reality-based way. Nate deserves a lot of credit for not mindlessly following the conventional wisdom and for basing his observations on empirical facts, rather than hunches. He didn’t do anything that other smart people weren’t doing at the same time, though.
Calouste
@Mark S.:
Considering how wrong Dick Morris was about everything during the last election cycle (IIRC there were probably 20 states where his prediction was seriously off from the actual result, as in predicted toss ups going 10-15% to one side etc), that means that it more likely that the Dems will win 100 seats. Ok, realistically speaking 20-30 for the Dems if Morris says 100 for the Reps.
Cain
Sorry, OT, but has anybody seen this at Wonkette? Basically she’s blogging about a painter who painted something titled “One Nation Under God” and has all the fun wingnutty visuals one could have. It’s quite a hoot watching Jesus holding the constitution and the Devil looking like Senator Palpatine/Darth Sidious.
wonkette
cain
Svensker
@Paul L.:
Making up a quote and then ridiculing it isn’t a great strategy for winning debate points.
As far as Dems getting “teabagged” (ew) in the upcoming elections… As former Repubs who were driven from the party by the insanity that became manifest during Bush II, my husband and I are split on the election. I’m going to hold my nose and vote for the stinky Big Money Dem, my hub either won’t vote or will vote for an indie. Neither of us will EVER vote Repub again, ever. And people like you, Paul L., abundant in the remaining ranks of Repubs, are the reason.
freelancer
@Cain:
Check the Bitsy open thread, about 3 or 4 of us have been riffing on it all day. I even posted a bit about it.
Mark S.
@Cain:
Wow, there is so much insanity in that painting I really encourage everyone to go take a look! On the right side of the picture, it looks like John Adams and some other Founder are dancing a jig.
Paul L.
@Svensker:
I am not quoting anyone. I am just want it embraced by the Media/Nutroots.
Rachel Maddow , Janeane Garofalo and the Nutroots call the tea party protesters “tea-baggers”. So it is fair game to throw it right back in their faces.
Notorious P.A.T.
@timb:
Don’t forget the sheriff who stumped for McCain, off duty, in his uniform, or the general who evangelized in his uniform. That’s okay, too.
Cain
Yes, and please note the two figures who are hiding their face in shame is 1) a union soldier from civil war (notice, it wasn’t the traitors who rebelled) and 2) a judge. Yeah, that’s some pretty fucked up shit. The guy needs some therapy (or maybe that is what the painting was)
@freelancer – I’ll check it out.
The painting is definitely worth a post. :)
cain
ThatLeftTurnInABQ
@Bobby Thomson:
Speaking as a recovering 538 junkie (from early on in the primaries), I’d say that’s not really accurate. What Nate did was to datamine the cross-tabs of the polls other folks were producing (supplemented by state and congressional district level demographic info) and run regressions against that info and most importantly against the actual results of the state by state primaries, to try to isolate the statistically most significant demographic characteristics of the voter population.
He kept honing the details of his regression model to the point where by the end of the primary season and going into the general election it could and did hold up extremely well as a predictive tool. He wasn’t just averaging or aggregating polling results, instead he was building a model for what factors were most important in understanding the voting behavior of the electorate – how much race, income level, etc. factored in to voters leaning one way or the other. Nobody else was doing that at anything remotely like the level of detail he was working.
And this was something that Nate really needed the long drawn out nominating contest to do, because his model had to be repeatedly tweaked against the primary results as they came in – something that nobody could do against a single-shot event, because by the time the results came in a predictive model would no longer be needed, and there is no guarantee that the demographic factors chosen would have the same weights or the same predictive power by the time the next election cycle came around 2-4 years later.
Nate probably couldn’t have done what he did as well as he did if the Dem nominating contest had been short and boring. Which means that in 2012 he may have a better shot at recalibrating his regression models on the GOP side, unless Obama faces some real challenges from both the left and the right side of the Dem field and the Dem primaries are contested well past the point when a sitting POTUS is normally confirmed as the nominee.
freelancer
@Cain:
That painting deserves a Lexicon entry.
It’s got Activist Judges, The Constitution being divinely inspired, Reagan Worship, Beck’s crazy NWO conspiracy theorist inspiration, Real ‘Merka, the Moranic inclusion of Thomas “Payne”, to name just 6 things.
Mark S.
@freelancer:
It’s also got a college student holding a copy of “The 5,000 Year Leap,” so the painter is obviously a Beckster. The college professor, who looks like Jerry Springer, is holding “The Origin of Species.”
Mr Hollywood does look like a prick, though. I wouldn’t go see any of his movies.
Notorious P.A.T.
LOL that’s hilarious. The soldier is black and named “King” as a tribue to Martin Luther King Jr. Because he was all about the wars!
Alex
I think referring to Thomas Payne counts as an EPIC FAYLE.
CalD
Charlie Cook was certainly the least accurate oracle of the 2006 mid-terms by a pretty large margin. In ’06 kept a spreadsheet of pundit predictions which I still happen to have.
In terms of their final calls on the week of the election, Sabato and the National Journal hotline tied for first place in my rankings, correctly leaning 60 of 78 house districts considered to be the most competitive. Rothenberg was a close second with 59. Cook got 47 out of 78, which is to say he could probably have done just about as well by flipping a coin or picking the petals off a daisy.
Something interesting though, was that when I went back to do the same analysis for three weeks before the election (which was the oldest revision of the spreadsheet I thought to save), neither of the two top finishers in the final analysis had done nearly as well. For the week of 10/16/06, out of 80 races considered most competitive the rankings for correct leans were as follows:
Evans/Novak 55
Rothenberg 53
National Journal 53
Sabato 51.5*
CQ 49
Cook 45.5*
(I awarded half-points for districts rated toss-ups that really did end up being tight races on election day.)
Another interesting point was that on the 10/16 sheet, a meta-average of all of the above outscored any individual pundit with 57 out of 80, whereas in the final analysis it was the third place finisher with 58. The really obvious conclusion in all this though is that if the best that the best of them could do a few days before the election was about 4 out of 5, and a mere three weeks out that dropped to down to more like 4 or 5 out of 8, why are we even having this conversation about an election 13 months away?
CalD
I @CalD:
I meant:
“…if the best that the best of them could do a few days before the election was about
4 out of 53 out of 4, and a mere three weeks out that dropped to down to more like 4 or 5 out of 8, why are we even having this conversation about an election 13 months away?”(Where’s that darned edit button when you need it?)