Many people make it through statistics class without a clear understanding of the common process used for all statistical hypothesis tests. In fact, it's not unusual to complete the class without realizing there is a common process. That's not the students' fault -- most professors don't emphasize it.
Executives need accurate, relevant information that is presented simply, quickly, and clearly. They need answers to their questions, and they need to have confidence, not just in the data's message, but also in the messenger.
What happens to analysts who don't know the process and don't use it to clarify their thinking? Most often, executives just don't find those analysts persuasive. Their presentations sound too esoteric, and their answers to questions do not satisfy.
Here's a confession: I love data analysts who are full of hot air. It is so easy to disarm these windbags. All I have to do is smile sweetly, look them straight in the eye, and ask a little question -- What were your assumptions here? -- or make a simple point -- The method you've used was designed for ratio measures, but this metric is ordinal -- and it's all over. Everybody in the room understands. Don't let that happen to you. The five-step process common to every statistical hypothesis test will make your work bulletproof.
State a null and alternative hypothesis. If this doesn't seem familiar, or you are not sure of how it's done, read any introductory statistics textbook. It's in all of them.
State the assumptions that are reasonable for the application. These are conditions we believe to be true: simple random sampling, normally distributed population, and so on.
Select a test type. The type of test to use depends on what you are measuring and the assumptions you can reasonably make. Each test has its own assumptions, and you may have to comb the books carefully to find them. Make sure the assumptions fit your situation. If they don't, you've selected the wrong test.
Calculate a p value. This can be done for any type of statistical hypothesis test. The p value is interpreted the same way for all tests. Very low p values imply strong evidence against a null hypothesis. High p values represent weaker evidence against the null hypothesis -- or no such evidence at all.
State your conclusion in a simple, plain-language sentence. This one is the downfall of many a mathematical genius. State the results in terms any business person can understand, such as "Our tests show no evidence that the test coupon will outperform the coupon we are using today." Avoid sentences like "The p value was 0.12, so we failed to reject the null hypothesis."
Do you use these five steps in your work? Please share your experience.
The points were well sumarized. I'm thinking the last one might be a neglected one though. Being able to summarize the results so the executives can easily see what's going on and the recommendation can turn into a overblown recitation that goes over the heads of many and leaves confusion if not stated simply and with as few words as possible.
Michaeljackson, very true on eliminating technojargon - I do think to employ simplicity in practice takes time to digest the results and think about what expression best represents what needs to be conveyed. I wonder about the degree of difficulty analysts struggle with delivering information simply while meeting deadlines. Simplicity and refinement do not mix with urgency at times.
Nice points, Meta, about how basic stat concepts can be understandable, and how the last point, describing uncertainty, can send a misunderstood meassage about the opportunity possible from the result. Professional discussions around the topic could inspire more discussion of about displaying technical results to support data visualization, I imagine.
Sounds like the real problem is Stupid (and probably overpaid) Executives. Executives should at minimum understand the business they are in. Data analysis is usually done in order to answer a question (solve a specific problem.) Your procedural is ineffective when "here's a bunch of data we got off of Facebook, tell us what it means" scenario.
"Garbage in, Garbage out". Wisconsin has the highest per capita consumption of cheese and also the highest per capita rate of rectal cancer. The numbers show high statistical correlation and the numbers pass your test. Still wrong though.
"State your conclusion in a simple, plain-language sentence. This one is the downfall of many a mathematical genius. State the results in terms any business person can understand, such as "Our tests show no evidence that the test coupon will outperform the coupon we are using today." Avoid sentences like "The p value was 0.12, so we failed to reject the null hypothesis."
This point, in my opinion is the most valid. If you can communicate your point across to me without all of the techno jargon, then I can better understand you and feel more comfortable in doing business with you.
I did not, and have never, recommended stating a p value in a presentation to an executive. I recommended calculating the p value as a part of a process that leads to a plain English statement of the conclusion. However, it is important that the analyst know and understand the p value, and be prepared to discuss it if asked, as some executives, and certainly some staff members, are aware of the concept and will ask.
The use of confidence intervals in place of hypothesis tests is a variant arising out of the same statistical theory, not a revolutionary alternative. As a presentation technique, it's a good idea. However, this should be regarded as a supplement, not an alternative, to hypothesis testing.
I never provide p-values. This concept is too esoteric and most executives i work with don't know what it is and they like when i speak their language, rather than mine. indeed, i never perform statistical tests, but instead I provide confidence intervals based on model-free inference. Google "Analyticbridge first theorem" for details.
2015 Visual Analytics Interactive RoadshowSAS(r) experts are coming to a city near you in a series of live, interactive workshops focused on SAS Visual Analytics, including how to prepare your data for VA, the integration of VA with Office Analytics and a Visual Statistics demo.
January 22: King of Prussia, PA
February 24: Austin, TX
March 26: Redwood City, CA
April 22: NYC, NY (1st of 2 stops)
May 13: Seattle, WA
June 18: Minneapolis, MN
July 21: Rockville, MD
August 18: Chicago, IL
September 24: Irvine, CA
October 9: Cary, NC (during SAS Championship)
October 21: NYC, NY (2nd of 2 stops)
November 17: Orlando, FL
December 8: Atlanta, GA
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
The hospitality industry gathers massive amounts of customer data, and mining that data effectively can yield tremendous results in terms of improved CRM, better-targeted marketing spend, and more efficient back-end processes. Roger Ares, vice president of analytics at Hyatt Corp., discusses the ways he and his staff use big data.
Charged with keeping track of travel assets, including employees, iJET International relies on data management best-practices and advanced analytics to keep its clients in the know on current and potential world events affecting travel, Rich Murnane, Director of Enterprise Data Operations & Data Architect, told All Analytics in an interview from the 2014 SAS Global Forum Executive Conference.
Jason Dorsey, chief strategy officer for the Center for Generational Kinetics and keynote speaker at last month's SAS Global Forum 2014, describes how Gen Y professionals are enhancing the makeup of multigenerational analytics organizations.
From analytics talent development to the power of visual analytics, All Analytics found a variety of common themes circulating throughout the exhibition floor and session discussions at the 2014 SAS Global Forum and SAS Global Forum Executive Conference events held last month in Washington, DC.
Talking with All Analytics live from the 2014 SAS Global Forum Executive Conference, Eric Helmer, senior manager of campaign design and execution for T-Mobile, discussed the importance of customer data -- starting internally -- in devising the mobile operator's marketing plans.
The big-data analytics market can be a confusing place. Among the vendors vying for your dollars are traditional database management providers, Hadoop startup services, and IT giants. In this video, All Analytics editors Beth Schultz and Michael Steinhart sit down in a Google+ Hangout on Air with Doug Henschen, executive editor of InformationWeek. Henschen discusses use cases for big-data analytics, purchase considerations, and his recent roundup of the top 16 big-data analytics platforms.
At the National Retail Federation BIG Show last month, All Analytics executive editor Michael Steinhart noted a host of solutions for tracking and analyzing customer activity in retail stores. From Bluetooth beacons to RFID tags to NFC connections to video analytics, retailers must find the right combination of tools to help optimize the shopper experience, streamline operations, and boost revenues.
The days when historical shipment trends and gut feelings were enough to forecast retail demand accurately are long over. SAS chief industry consultant Charles Chase outlines the benefits of pulling real-time sales information from point-of-sale and product scanner systems, then flowing that data into dynamic forecasting tools from SAS.
With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.