One of the great things about my work is that I get to travel and meet really smart people the world over. On Election Day, for example, I happened to be attending an analytics event and had the opportunity to do plenty of offline conversing on who would win.
Just imagine a bunch of quants sitting around a lunch table talking about all the various, possible election outcomes, their related probabilities, the evils of the Electoral College, the possibility of an electoral tie, and on and on. Aside from the obvious danger of waking up nose-deep in a garden salad, that's a truly fascinating conversation, right? Well, for me, not so much.
Why? Because who would win had been obvious to me for quite some time. The data were pretty clear.
Unfortunately for me, I hadn't quite gotten around to starting that blog I've been threatening to write for the last couple of years, and so I hadn't published my erudite, quantitatively sound, and ultimately correct prediction. Some guy named Nate Silver did.
Nate who?
That was my reaction, anyway. Apparently I've somehow managed to avoid hearing about the predictive wunderkind who took the political prognostication realm by storm back in 2008. Fortunately, the event's host had the foresight to give each attendee a copy of Silver's new book, The Signal and the Noise: Why So Many Predictions Fail -- But Some Don't. It had been waiting for me on my place setting that very morning. That was great, because I love books, but too bad, because I just don't read them anymore. Don't get me wrong -- I'm a voracious reader -- but the only time I read something that's printed on actual paper is during that time on the airplane when I have to turn off my e-reader between the ground and 10,000 feet. The only notable exception is the output of one tree-killing colleague of mine who insists on printing absolutely everything. You know who you are.
Anyway, not wanting to spring for the e-reader version with a perfectly good -- if somewhat anachronistic -- actual book in my hand, I decided to wade into the pulp. It turns out that Silver worked as an economic consultant for a big accounting firm, got bored with that (unimaginable, I know), wrote some software to predict baseball player performance à la sabermetrics, and then founded a political blog, now on The New York Times, called FiveThirtyEight. It was on that blog that he correctly predicted the outcome of the 2012 presidential election in all 50 states. Four years ago, he only got 49 out of 50 correct (darn that pesky Indiana). It might be easy to argue that 2008 was a fluke, but repeated success is somewhat more difficult to discount.
Because of that success, Silver has received a lot of attention. The question I ask is not how he did it, but rather why his success seems to be such a singularity (setting aside the fact that I wasn't in the game yet). Why aren't lots of other people coming up with the similar predictions and similar rates of accuracy? Perhaps others are, but there are certainly a lot of people trying who aren't, and most of them, sadly for us, seem to have found employment as TV talking heads. I believe one possible answer to my question is foundational to the big-data challenge, and it's embodied in my new favorite quote. From the introduction to Silver's book:
We face danger whenever information growth outpaces our understanding of how to process it.
@Bryan_Engle, Thanks for your comment. I find that one of the chief obstacles to driving adoption of analytics-based practices is that most business people don't understand what is possible. One of the key roles I play is educating people about the possibilities. The good news is that with all the buzz around analytics, people are eager to listen these days.
True, leadership is still a small minority. As more enterprise organizations begin to invest in big data, adapting and creating data driven cultures should be a key strategic involvment in day-to-day decision-making.
@hospice, thanks for your comment. I also believe we will figure it out, and I think we will make great impact that may one day be taken for granted. Mother Earth didn't figure out how to feed us on her own. Tremendous efforts in science and agricultural engineering went into creating the food supply that supports our population. There are many great stories, innovations, and unsung heroes in that history. In the same way, it will take the focused effort and remarkable creativity to tame the data deluge, but the rewards will be great.
May also want to check out the Princeton Election Consortium, which Deadspin (not your usual source for analytics) calls "criminally underrated." Agree with your point though - why aren't we using this type of methodology used more in other areas. We hear a lot about data driven decision making, but seems that we (businesspeople in general) have a hard time moving away from using our expertise/gut to make decisions.
"We face danger whenever information growth outpaces our understanding of how to process it."
It reminds me the Malthusian catastrophe that predicted that "the power of population would be superior to the power of the earth to produce subsistence for man". But was Thomas Malthus' observation correct? The earth has ever since learnt how to porduce subsistence to its growing population. I think we will be able to do the same with regard to big-data challenges.
Beth - More important are Silver's frequent stops on Comedy Central with Jon Stewart and Stephen Colbert. He's everything one imagines a geeky quant should look and act like. Sweet.
Mark -- certainly having the Times affiliation is great exposure. I think others of the spot-on presidential predictors are well enough known in their own domains, academics, for instance. -- like Election Analytics.
With an Android app, cellphone data, and analytics, the Navy hopes to better understand, monitor, and react during a crisis like severe flooding, earthquakes, and disease outbreaks.
Analysis of -omics data -- genomics, proteomics, transcriptomics, etc. -- can take tremendous computing power and data storage. Big-data can provide the fuel.
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
To save this item to your list of favorite AllAnalytics content so you can find it later in your Profile page, click the "Save It" button next to the item.
If you found this interesting or useful, please use the links to the services below to share it with other readers. You will need a free account with each service to share an item via that service.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.
Elizabeth Barth-Thacker, a BI and informatics technology manager at Humana, tells us how her team is creating data transparency and building engagement with the business – with the help of an internal collaboration portal called Humanalytics.
Speaking at SAS Global Forum Executive Conference, Rajeev Kaul, SVP of pricing at OfficeMax, uses a Chinese proverb to explain one of the reasons he's deploying SAS Visual Analytics.
In an All Analytics interview, Mike Cavaretta, technical leader, predictive analytics at Ford Research & Advanced Engineering, shares how big-data is fueling vehicle decisions.
Analytics professionals and SAS executives share how organizations can get on with their work so much faster when working in a high-performance and visual analytics environment.
Analytics professionals who attended SAS's recent Executive Briefing in New York share how they think visual analytics might help their organizations get better value from data.
At Boeing, effective decision making comes down to this simple formula: QxA=E, as executive Jerry Allyne explained at the recent INFORMS analytics conference.
Whether working in major league sports, financial services, or healthcare, analytics, and data, professionals are checking out how visual analytics and high-performance technologies can help them optimize their environments, shrink their cycle times, and improve decision making, as attendees at the recent SAS Executive Briefing in New York share with us.
SAS CEO Jim Goodnight speaks with us at a recent SAS Executive Briefing about getting a feel for what's in your big-data and other new realities powered by advanced analytics.
Jim Davis, SVP and CMO at SAS, talks with us at a recent SAS Executive Briefing about how high-performance analytics and visual analytics take away the concerns over big-data and let companies get down to business with their data.
Tweet This
14 
8 
