By Jordan Ellenberg
Posted Tuesday, Sept. 19, 2006, at 1:17 PM ET
QUOTE
It's an anxious season for the GOP. With midterm elections eight weeks away, two dozen Republican-held seats in the House of Representatives are too close to call. Republican strategists interviewed by the Washington Post suggested that the Democrats had a 75 percent or even 90 percent chance of seizing control. Charlie Cook, a respected nonpartisan analyst, told the New York Times, "If nothing changes, I think the House will turn."
It's easy to see why Republicans are nervous. In the latest Rothenberg Political Report, 24 seats are listed as "tossups"—10 "pure tossups" and seven each "tilting R" and "tilting D." Every one of these tossup seats is held by a Republican. Democrats currently control 203 of the House's 435 seats. To reach a majority of 218, they need to gain 15 seats. Given so many opportunities to flip seats, it's easy to see that happening.
But before Democrats get too optimistic, they should think about the Baltimore Orioles. Like the Democratic congressional delegation, the Orioles have put up losing records for many years. And Orioles fans, like Democratic voters, can see a path to victory—if a few starting pitchers improve, if a promising rookie develops, if the scrappy second baseman regains his slugging form. ... None of these hopes are unreasonable. But it's too much to expect that they'll all come to pass. More likely, some things will go the O's way and some won't; they may end up improved, but they probably won't end up on top.
FPRIVATE "TYPE=PICT;ALT=Click Here!"
The Democrats, too, need a lot of things to go right if they're to take over. Let's assume that the races with one side clearly ahead will come out as predicted. That means the Dems need to win 15 of the 24 tossup races to take control. What's the chance of this happening?
The simplest model is to assume that the Democrats have a 50/50 chance of winning each race and that the races are mutually independent. That is, if we knew that the Democrat won in North Carolina, it's no more or less likely that the Democrat won in New Mexico. (You can find a more thorough discussion of independence, not to mention a stiff dose of 2001 nostalgia, in my article on Gary Condit's love affairs.) The theorem you need to know here is the Law of Large Numbers: If you have a bunch of independent 50/50 chances, it's very likely that just about half of them will go one way and half the other. In other words: Not only can you not expect everything to go your way, you can't even expect most things to go your way. Chances are that Democrats will win very close to half of the 24 tossups. Half is 12, and 12 is not enough.
The Law of Large Numbers suggests that it's unlikely Democrats will win as many as 15 of the 24 tossups. To guess exactly how unlikely requires a more powerful tool: the Central Limit Theorem. Imagine sketching a graph of the probability of various outcomes of the midterm election. The graph would have a big hump in the center, representing the most likely scenarios—the Democrats winning around 12 tossups—and would rapidly tail off in both directions as the outcomes became more extreme. What you just drew is the normal distribution, or bell curve—the foundation of all applied statistics. The Central Limit Theorem says that a random variable (like the midterm election) made of many small, identical, independent parts (like the individual House races) always obeys a law approximating the bell-shaped curve. The more individual events involved, the better the obedience to the bell curve.
If you flip a coin—or a congressional district—N times, you expect to get about N/2 heads. But it's too much to expect to hit N/2 on the nose—it's standard to see some deviation from dead center. But how much? The answer, naturally, is called the standard deviation, which in this case comes to half the square root of N. So, for our 24 tossup elections, the standard deviation is about 2.45. We shouldn't be too surprised, then, to see between nine and 15 seats go to the Dems. But the Central Limit Theorem does this one better; it says that the chance of beating expectations by one standard deviation or more is approximately 16 percent. To win control, the Democrats have to beat expectations by three seats, a bit more than one standard deviation; their chance of doing so should thus be a bit less than 16 percent. In fact, it's about 15.4 percent. If you'd like to see this computation, and a more detailed one, which takes Rothenberg's "leaning R" and "leaning D" races into account, check out this PDF file.
That still looks bad for the Democrats. But let's reconsider one of our assumptions—that the results of House races are independent. A Dem victory in the tossup race in the 11th district of North Carolina could mean any number of things—that national events are benefiting Democrats or that undecided voters are breaking the Dems' way. Either way, if we knew the Democrats had won in North Carolina we'd think it was more likely that they'd also won in New Mexico. In other words, the two races aren't independent but correlated. And correlated events don't yield a neat bell curve. Think of the most extreme example—if one Democratic victory forced all the other districts to go Democratic, then the Dems would win either 24 seats or none. In that case, their chances of taking the House would be 50/50—much better than the 15 percent the CLT suggests. The more correlation between races, the better for Democrats.
So, how much correlation is there? It's hard to be sure, but we can make a quick and dirty estimate by looking at the past. Nathan Gonzales, the political editor at the Rothenberg Report, was kind enough to send me a summary of the races Rothenberg rated "pure tossup" in early September of the last six election years.
Year Tossup Seats Tossup Seats Won By Democrats
2004 6 4
2002 5 2
2000 12 5
1998 13 9
1996 20 13
1994 25 5
One thing to take note of is the shrinking number of tossups, a symptom of our steadily less competitive legislative elections. But what we're really interested in is how the Democrats did compared to our expected outcome—winning half the seats—if House races were independent events. In 1998 and 1996, the Democrats did better than one standard deviation above the expectation. In 1994, the Republicans did much better. So, based on history, it wouldn't be weird to see some correlation between the races. How much correlation? With only these six data points it would be brutal malpractice even to throw out a number. But it's fair to say that the Democrats' chances of flipping the House are somewhere between 15 percent (the scenario in which the races are independent) and 50 percent (where the races are as correlated as possible).
But remember that Republican strategists told the Washington Post that the Dems have as much as a 90 percent chance of taking the House. Fifty percent isn't close to 90 percent. Why all the predictions of triumph for the Democrats if the party's still running behind?
One explanation is that the tossup races might really be leaning Democratic. Polling in House races is pretty erratic and pretty bad. The data we do have doesn't show a clear swing toward Democrats, but it's possible insiders know something we don't. Pros like Cook and Rothenberg are not just counting seats but evaluating the political climate. Rothenberg believes that national forces will make this a "wave election" like 1994, in which the large majority of tossups and some Republican-leaning seats will turn blue in November.
There's one more explanation for Dem-friendly projections. Everyone has an incentive to overestimate the chances of a change of hands: A Congress in the balance is a great fund-raising tool for both parties. Plus, for Republicans, selling the impending Democratic victory resets expectations—in the current environment, the Democrats could pick up 14 seats and still come off as losers. Even nonpartisan commentators like Rothenberg and Cook might be inclined to oversell the Democrats. It's more fun to write about a photo finish than a foregone conclusion, and prognosticators get a lot more credit for getting spectacular predictions right than you get blame for getting them wrong.
If you're a Democrat, you've got every right to be hopeful this fall. But if somebody gives you the chance to bet on the GOP, get your wallet out
It's easy to see why Republicans are nervous. In the latest Rothenberg Political Report, 24 seats are listed as "tossups"—10 "pure tossups" and seven each "tilting R" and "tilting D." Every one of these tossup seats is held by a Republican. Democrats currently control 203 of the House's 435 seats. To reach a majority of 218, they need to gain 15 seats. Given so many opportunities to flip seats, it's easy to see that happening.
But before Democrats get too optimistic, they should think about the Baltimore Orioles. Like the Democratic congressional delegation, the Orioles have put up losing records for many years. And Orioles fans, like Democratic voters, can see a path to victory—if a few starting pitchers improve, if a promising rookie develops, if the scrappy second baseman regains his slugging form. ... None of these hopes are unreasonable. But it's too much to expect that they'll all come to pass. More likely, some things will go the O's way and some won't; they may end up improved, but they probably won't end up on top.
FPRIVATE "TYPE=PICT;ALT=Click Here!"
The Democrats, too, need a lot of things to go right if they're to take over. Let's assume that the races with one side clearly ahead will come out as predicted. That means the Dems need to win 15 of the 24 tossup races to take control. What's the chance of this happening?
The simplest model is to assume that the Democrats have a 50/50 chance of winning each race and that the races are mutually independent. That is, if we knew that the Democrat won in North Carolina, it's no more or less likely that the Democrat won in New Mexico. (You can find a more thorough discussion of independence, not to mention a stiff dose of 2001 nostalgia, in my article on Gary Condit's love affairs.) The theorem you need to know here is the Law of Large Numbers: If you have a bunch of independent 50/50 chances, it's very likely that just about half of them will go one way and half the other. In other words: Not only can you not expect everything to go your way, you can't even expect most things to go your way. Chances are that Democrats will win very close to half of the 24 tossups. Half is 12, and 12 is not enough.
The Law of Large Numbers suggests that it's unlikely Democrats will win as many as 15 of the 24 tossups. To guess exactly how unlikely requires a more powerful tool: the Central Limit Theorem. Imagine sketching a graph of the probability of various outcomes of the midterm election. The graph would have a big hump in the center, representing the most likely scenarios—the Democrats winning around 12 tossups—and would rapidly tail off in both directions as the outcomes became more extreme. What you just drew is the normal distribution, or bell curve—the foundation of all applied statistics. The Central Limit Theorem says that a random variable (like the midterm election) made of many small, identical, independent parts (like the individual House races) always obeys a law approximating the bell-shaped curve. The more individual events involved, the better the obedience to the bell curve.
If you flip a coin—or a congressional district—N times, you expect to get about N/2 heads. But it's too much to expect to hit N/2 on the nose—it's standard to see some deviation from dead center. But how much? The answer, naturally, is called the standard deviation, which in this case comes to half the square root of N. So, for our 24 tossup elections, the standard deviation is about 2.45. We shouldn't be too surprised, then, to see between nine and 15 seats go to the Dems. But the Central Limit Theorem does this one better; it says that the chance of beating expectations by one standard deviation or more is approximately 16 percent. To win control, the Democrats have to beat expectations by three seats, a bit more than one standard deviation; their chance of doing so should thus be a bit less than 16 percent. In fact, it's about 15.4 percent. If you'd like to see this computation, and a more detailed one, which takes Rothenberg's "leaning R" and "leaning D" races into account, check out this PDF file.
That still looks bad for the Democrats. But let's reconsider one of our assumptions—that the results of House races are independent. A Dem victory in the tossup race in the 11th district of North Carolina could mean any number of things—that national events are benefiting Democrats or that undecided voters are breaking the Dems' way. Either way, if we knew the Democrats had won in North Carolina we'd think it was more likely that they'd also won in New Mexico. In other words, the two races aren't independent but correlated. And correlated events don't yield a neat bell curve. Think of the most extreme example—if one Democratic victory forced all the other districts to go Democratic, then the Dems would win either 24 seats or none. In that case, their chances of taking the House would be 50/50—much better than the 15 percent the CLT suggests. The more correlation between races, the better for Democrats.
So, how much correlation is there? It's hard to be sure, but we can make a quick and dirty estimate by looking at the past. Nathan Gonzales, the political editor at the Rothenberg Report, was kind enough to send me a summary of the races Rothenberg rated "pure tossup" in early September of the last six election years.
Year Tossup Seats Tossup Seats Won By Democrats
2004 6 4
2002 5 2
2000 12 5
1998 13 9
1996 20 13
1994 25 5
One thing to take note of is the shrinking number of tossups, a symptom of our steadily less competitive legislative elections. But what we're really interested in is how the Democrats did compared to our expected outcome—winning half the seats—if House races were independent events. In 1998 and 1996, the Democrats did better than one standard deviation above the expectation. In 1994, the Republicans did much better. So, based on history, it wouldn't be weird to see some correlation between the races. How much correlation? With only these six data points it would be brutal malpractice even to throw out a number. But it's fair to say that the Democrats' chances of flipping the House are somewhere between 15 percent (the scenario in which the races are independent) and 50 percent (where the races are as correlated as possible).
But remember that Republican strategists told the Washington Post that the Dems have as much as a 90 percent chance of taking the House. Fifty percent isn't close to 90 percent. Why all the predictions of triumph for the Democrats if the party's still running behind?
One explanation is that the tossup races might really be leaning Democratic. Polling in House races is pretty erratic and pretty bad. The data we do have doesn't show a clear swing toward Democrats, but it's possible insiders know something we don't. Pros like Cook and Rothenberg are not just counting seats but evaluating the political climate. Rothenberg believes that national forces will make this a "wave election" like 1994, in which the large majority of tossups and some Republican-leaning seats will turn blue in November.
There's one more explanation for Dem-friendly projections. Everyone has an incentive to overestimate the chances of a change of hands: A Congress in the balance is a great fund-raising tool for both parties. Plus, for Republicans, selling the impending Democratic victory resets expectations—in the current environment, the Democrats could pick up 14 seats and still come off as losers. Even nonpartisan commentators like Rothenberg and Cook might be inclined to oversell the Democrats. It's more fun to write about a photo finish than a foregone conclusion, and prognosticators get a lot more credit for getting spectacular predictions right than you get blame for getting them wrong.
If you're a Democrat, you've got every right to be hopeful this fall. But if somebody gives you the chance to bet on the GOP, get your wallet out
http://www.slate.com/id/2149839