All Analytics Academy
The Internet of Things Joins the Enterprise

Jun 9 - Jun 23
Join 5 interactive classes & chat with peers
 
John Barnes

The Landslide That Wasn't: A Lesson in Distributions

NO RATINGS
1 saves
View Comments: Newest First | Oldest First | Threaded View
Page 1 / 2   >   >>
John Barnes
User Rank
Blogger
Re: one more thing
John Barnes   11/20/2012 9:09:25 AM
NO RATINGS
Louis, I'd say they saw the ground but not the implications. Kind of like recognizing that "well it's a high scoring game, we are only down by a touchdown and a field goal, and there is still five minutes on the clock" but then not getting to the conclusion "we have to play it out but we are almost certain to lose." Not so much ignoring the facts -- the polls were very accurate this time -- but refusing to see what the facts meant, and instead insisting on just repeating whichever facts made you happiest.

Louis Watson
User Rank
Blogger
Re: one more thing
Louis Watson   11/20/2012 1:47:42 AM
NO RATINGS
I see what you are saying John, so is it safe to say republican pollsters simply did not understand what the underlying meaning of the distribution was in reality ?   And if this was the case, how can seasoned campaign managers make such a colossal blunder ?  Objectivity lost to partisan politics ?   Well, of course it was.

But I think it goes to what you and @rbaz were discussing earlier in this thread, the fact that the media has skew reality to such a degree, couple that with a Media pool that is at best passive and non-confrontational  produces outcomes  such as this past election.  Am I the only one who thought this (the election) was over by half-time ?   

I have always held a heathly disdain for polls (especially national elections) because they tend to repeat themselves in flow ( meaning regardless of all the other polls before the one just before voting will most often be deemed a "close race").   I have yet to see one in my lifetime where this pattern veered too far from this formula, which is a major reason I have no use for polls. As far as I am concerned yet again Polls and Polling  did not reflect what is really going on " on the ground'.

I just can't believe this simple fact was missed by many so called experts. 

John Barnes
User Rank
Blogger
Re: one more thing
John Barnes   11/20/2012 1:01:17 AM
NO RATINGS
Louis, well, if you understand the thing being represented, that's a pretty good guard against many kinds of folly. And my purpose here is not to teach people how to do the math. There isn't space, time, or interest for that here. The idea is more to get people comfortable with asking for the math and having an idea of what it says when they get it. Kind of like the wine columnist doesn't teach you how to make wine, but what to order when and what to look for.

John Barnes
User Rank
Blogger
Re: Vote 2012 and the Inquisition that failed
John Barnes   11/20/2012 12:54:17 AM
NO RATINGS
Lyndon, I think Krugman did a pretty solid job of explaining too. Another way to look at distributions is to think of them as functions that convert local and specific margins into overall probabilities. But a key point not to be lost is that distributions also apply to forecasting markets, liability, crime, war, sports, any large scale wide participation human activity. I guarantee that someone who is chuckling "silly Republicans" right now will make the same mathematical error themselves within a day. (I hope to reduce the number but I don't think it can be eliminated).

Louis Watson
User Rank
Blogger
Re: one more thing
Louis Watson   11/19/2012 11:26:23 PM
NO RATINGS
Thanks John for explaining in part what happen to republican pollsters with regard to understanding or the lack thereof with respect to distributions.  The method of analysis seems easy enough however many make this kind of mistake whenever this tool is in use.  

I am not sure I understand it completely either,  but I take pride in practicing your 5th tip - Knowing the ground.  This alone can make up for numerous statistical shortcomings IMO.

Lyndon_Henry
User Rank
Blogger
Vote 2012 and the Inquisition that failed
Lyndon_Henry   11/19/2012 10:22:26 PM
NO RATINGS
..

John Barnes writes


Wishful thinking was only a minor factor in the massive, obvious, embarrassing error by conservative pundits who predicted that the 2012 presidential election would be a dead heat or even a Mitt Romney landslide. A profound misunderstanding of statistical distributions caused the humiliation of so many conservative bloggers, journalists, and campaign managers.


 

 

In a sense, the profound failure of GOP election prediction reflects a case of getting caught by their own petard.  Carl Rove's vehement disbelief, witnessed by millions on live TV when Fox News analysts called Ohio for Obama, is iconic, and it seems to reflect a situation of believing the fantasies in the whacko reality you have constructed and led others into.

In another sense, the GOP prediction failure represents a failure of a kind of a 21st-century Inquisition.  The GOP targeted venomous anger against both polls and analysts who dared to use math objectively and read the results that suggested a rather solid Obama victory.  This level of disbelief and rejection of science (math) reminds me of the pressure brought to bear on Galileo, forcing him to deny what his own scientific research and observations were telling him.  Fortunately, for this election, the rightwing Inquisition simply fizzled.

Nate Silver of the NYT's 538 blog, a platform mainly for the presentation of the results of his own political analytics, has been widely hailed for the accuracy of his math-based predictions.  For example, see:

Nate Silver Takes A Victory Lap After Obama Re-election

Here are some interesting quotes:


Silver came through with flying colors, as Obama performed nearly exactly the way he said he would. The public recognition was immediate.

"You know who won the election tonight? Nate Silver," Rachel Maddow said on MSNBC. Even Fox News tipped its cap to Silver.

Others said that the results could force a bit of a sea change in political journalism.

"What does this victory mean?" Mashable's Chris Taylor wrote. "That mathematical models can no longer be derided by "gut-feeling" pundits. That Silver's contention -- TV pundits are generally no more accurate than a coin toss -- must now be given wider credence."


 

Silver, of course, became a particularly hated target of the rightwing anti-science blitz that attempted to portray some kind of mysterious Romney "surge" till the bitter end.

Economist and NYT columnist Paul Krugman discussed much of this (somewhat along the lines of John Barnes's explanation) in a Nov. 4th blog entry:

Math Is Hard

Some of Krugman's interesting points:


First of all, from what I can see a lot of people have trouble with the distinction between probabilities and vote margins. ...

Second, people clearly have a problem with randomness — with the fact that any poll, no matter how carefully conducted, has a margin of error. (And the true margins of error are surely larger than the statistical measure always reported, since sampling error isn't the only way a poll can go wrong). ...

What this means is that if you look at all the polls, you're very likely to find one or two that tell you what you want to hear... even good pollsters will produce an occasional off result, and you really, really don't want to start picking and choosing those off results to make yourself feel good.

...Oh, and a third point: those margins of error are for any one poll. An average of many polls will have a much smaller standard error.


 

 

Callmebob
User Rank
Master Analyst
Back to school.
Callmebob   11/19/2012 3:09:27 PM
NO RATINGS
Thanks for the lesson, Professor Barnes! Probably the only ones at the Romney victory party were hermaphrodites that skew Normalia.

SethBreedlove
User Rank
Data Doctor
Re: one more thing
SethBreedlove   11/17/2012 9:26:40 PM
NO RATINGS
I like that saying. :)   

John Barnes
User Rank
Blogger
Re: one more thing
John Barnes   11/17/2012 9:07:20 PM
NO RATINGS
Seth, accuracy wasn't really an issue here; it's just that when you have successive close-numbers events and one side needs fewer wins than the other, the side that needs fewer wins has a massive advantage. As the IRA communicated to the Queen after a failed assassination attempt, "You have to be lucky every time. We only have to be lucky once."

SethBreedlove
User Rank
Data Doctor
Re: one more thing
SethBreedlove   11/17/2012 7:25:46 PM
NO RATINGS
I saw the articles explaining an 80% chance of winning.  It always amazes me how just a couple of percentage points here and there can cause major events to go in one direction. One state polls may have a large margin of error, but the margin of error is much reduced, however, when you aggregate different polls together, since that creates a much larger sample size.

Page 1 / 2   >   >>
Information Resources
More Blogs from John Barnes
Analysts would do well to get out of the rut of using linear regressions by default.
Sometimes your results require accuracy and sometimes precision. Knowing the difference matters.
Rule-based behavior models offer a good alternative to guesswork and folk wisdom.
Keeping these three words, often jumbled in business discourse, separate and precise can help you be a better decision maker.
Always keep in mind that regression to the means happens and watch those outliers.
Radio Show
Radio Shows
UPCOMING
James M. Connolly
Finding Answers Through Prescriptive Analytics


7/21/2015   REGISTER   0
UPCOMING
James M. Connolly
Health Analytics: Find Data Beyond the Hospital Doors


7/28/2015   REGISTER   0
ARCHIVE
James M. Connolly
Visualization: How to Bring Data to Life


6/22/2015  LISTEN   55
ARCHIVE
James M. Connolly
Learn Why Analytics Are at Home in the Cloud


6/15/2015  LISTEN   26
ARCHIVE
James M. Connolly
Analytics: Your Defense Against Cyber Threats


5/27/2015  LISTEN   60
ARCHIVE
James M. Connolly
Big Data & Big Pharma: How Analytics Might Save Your Life


5/19/2015  LISTEN   37
ARCHIVE
James M. Connolly
Live Interviews From SAS Global Forum


4/28/2015  LISTEN   11
ARCHIVE
James M. Connolly
How to Hire Great Analytics Talent


4/23/2015  LISTEN   51
ARCHIVE
James M. Connolly
Sports Analytics Mean Fun and Business


3/24/2015  LISTEN   3
ARCHIVE
James M. Connolly
Secure Your Big Data in the Cloud


2/26/2015  LISTEN   114
ARCHIVE
James M. Connolly
Make It Big As a Data Scientist in 2015


2/11/2015  LISTEN   106
ARCHIVE
James M. Connolly
Big Data, Decisions & the Simulated Experience


2/3/2015  LISTEN   87
Information Resources
Quick Poll
Quick Poll
Infographic
Infographic
It Pays to Keep Insurance Fraud in Check
While 97% of insurers say that insurance fraud has increased or remained the same in the past two years, most of those companies report benefits from anti-fraud technology in limiting the impact of fraud, including higher quality referrals, the ability to uncover organized fraud, and improve efficiency for investigators.
Follow us on Twitter
Follow us on Twitter
Like us on Facebook
Like us on Facebook
About Us  |  Contact Us  |  Help  |  Register  |  Twitter  |  Facebook  |  RSS