We love to extol the virtues of data-driven decision making here at All Analytics, but new research tells us we'd be foolish to think business executives give as much credence to analytical insight as they might.
Now, before you get all huffy, let me point out that the research doesn't say business executives aren't appreciative of the depth of insight provided via advanced analytics -- because they are. It's just that they're giving intuition the edge over the rational -- i.e., the data analysis -- when the time comes to make business decisions.
This is the key finding of a survey conducted last month of 720 senior executives, 88% of whom are in director-level positions or higher, said the study's collaborators, Fortune Knowledge Group and global ad agency Gyro. The study's intent was to explore the role of emotions and other subjective factors in business decision making.
Gut Decision Making by the Numbers
65% of executives believe subjective factors that can't be quantified (including company culture and corporate values) increasingly make a difference when evaluating competing proposals; only 16% disagree
62% of executives say relying on gut feelings and soft factors is often necessary
61% of executives say human insights should come before hard analytics when making decisions
My own gut tells me that analytics professionals should be neither discouraged nor disquieted by these findings... yet. Let's not forget that, despite the incessant chatter about big data's pervasiveness, many companies are in but the earliest stages of their use of advanced analytics. We can't expect miracles to happen overnight and the C-suite or other top managers to wake up one morning and cast all their years of experience aside and call only for the data. What we should expect, however, is that the analytical insight proves of such high value that decision by decision the data takes on greater and greater weight till one day the balance tips to its side.
The study, you might say, even validates the need for greater analytical depth. About one third of respondents indicated they believe a more analytical approach to decision making is hampered by insufficient analytical capacity (37%), excessive data volume (34%), and rapid growth in the types of information available (31%). The executive summary, "Only Human: The Emotional Logic of
Business Decisions," explained:
As a result, factors that can't be quantified -- such as a company's values, reputation, and corporate culture -- increasingly make a difference when executives choose among competing proposals. Many respondents (62%) believe that it's often necessary or even preferable to rely on "gut feelings." Respondents also indicate that unquantifiable factors should be given at least the same weight as quantifiable factors. Only a minority (38%) feel that executives and managers should disregard these factors in favor of a strictly analytical approach when making decisions.
I would argue, then, that as analytical expertise and experience deepens, the volume and variety of data will become less and less overwhelming to business executives. Big data will simply be data from which decision-influencing insights come, not some big, hairy monster hiding behind a server array.
And here's another noteworthy point from the study: Analytics cannot stand alone -- and certainly not sit above the business. "A solid majority of respondents (61 percent) say that in order to uncover the most effective, actionable insights, people who know the business should filter data to frame analysis before using predictive analytics." And only 39% said "advanced analytics should be used before human analysis. "
I don't find anything wrong with that. Analytics teams, we've long said, must work hand in hand with business users. Doing so brings them a necessary understanding of the challenges the business faces and helps them guide the types of questions the business should be asking of the data.
So today, given where we are in the evolving use of analytics, I reiterate: Don't be disheartened by this report's findings. But take heed from them and figure out ways to gain trust from the business. Little by little, the result will show in the shifting balance between gut and the data when it comes to business decision making. If you don't see this start to happen at your company over time, then -- yes -- start worrying.
What's your gut telling you? Am I giving business executives too much leeway here or is my logic flawed? Let's talk!
Tracy, glad you mentioned a decision support system, as that's what had come to mind as I began reading through your comment. I'd like to see more companies take advantage of automated decision management systems -- as long as, like you suggest, they enable exceptions (with explanation) and allow for continous feedback and improvement. Do you think enough companies incorporate decision managment systems into their advanced analytics strategies?
"...The data and resulting insights aren't supposed to replace the decision makekr's gut. They are supposed to inform decision makers and help them narrow the unknowns. But in the end decision makers are paid to interpret those unknowns and make a call. In other words, they use their gut, and rightfully so."
@David Schimitt Agreed. Somehow the discussion has morphed into Data vs. Gut , when as you and others mention these two factors supplement the other. No question this is a difficult task because most are not fluent in the effective practice of analytics. Many enterprises rely on the "cash cow" nature of their business unfortunately.
Of course the argument is that analytics will improve margins, squeeze every little bit out of every corner. Well there are more than a couple of barriers to this, but for starters a company really needs a strong foundation in the use of Analytics as it relates ( and that is the most important part IMO ) to their business.
Beth, Really good question. We can never know what's happening inside someone else's head, or how they used data -- unless they're willing to tell us. After years of studying how people explain decisions, I've learned that we're all tempted to blame someone for 'ignoring the evidence' when we disagree with them.
Without explicit capture of decision processes, we can't measure 'data vs. gut' -- or 'data + gut'. My work focuses on creating transparent (but not too painful) ways for decision makers to find relevant data and acknowledge how they weighed it. We can improve substantially on the old model of citing references, or compiling lists/matrices of factors. So accomplishing what you suggest (measuring 'data vs. gut') is possible if we have a practical way to see what the evidence says, and compare it to decisions being made.
Example: In evidence-based medicine, a clinical decision support system can make recommendations based on patient data and available evidence. Then the physician can accept that, or suggest a different action/treatment (some workflows require the care provider to explain exceptions). After that healthcare cycle plays out, we can perform analytics on electronic health records, outcomes, and decision support systems to measure the quality of a decision process, and how successfully the evidence was incorporated.
Unfortunately, in areas such as business strategy, comprehensive evidence isn't so available. I'm hoping that will change. Decision makers do get useful data from tools such as BI and data visualization. But it's typically not incorporated into end-to-end decision processes that connect actions to outcomes, supported by evidence-based recommendations. I look forward to the day when a marketing manager can easily see a systematic summary of evidence from multiple sources on, for instance, paid search marketing. Then we could begin to measure whether his/her marketing plans were based on data or gut.
Tracy, thanks for jumping in here. I haven't had a chance to read your piece yet, but in the meantime let me ask: Do you think the balance we see now between data and gut is appropriate? My sense is that when the data is counter to gut, the decision maker can be all too willing to disregard the data and favor the gut. Maybe that turns out OK sometimes, maybe not. But how do we measure that?
I totally agree with this. The data and resulting insights aren't supposed to replace the decision makekr's gut. They are supposed to inform decision makers and help them narrow the unknowns. But in the end decision makers are paid to interpret those unknowns and make a call. In other words, they use their gut, and rightfully so.
I wrote about this some time ago on this site -- glad to see the topic resurface!
'Data-driven' vs. 'Gut feel' isn't the problem that needs solving. Technology should supplement human intuition, rather than try to replace it. For instance: Medical diagnosis is messy, requiring hard evidence *and* the physician's decision-making.
@Maryam, as one wise businessman I know likes to say, business is about emotions. And trust is a very strong emotion. That's why relationships (with vendors for instance) will remain key decision drivers, despite what numbers may say.
Hi Maryam. Yes, you're right --- when a new leader comes in, nothing is guaranteed. But with increased use of business analytics, I'd hope that change just for the sake of change (or ego) gets harder to accomplish.
Beth I have witnessed it firsthand sometimes previous relationships weigh heavy on project awards. I have seen cases where all vendors were replaced because the new leader preferred vendors from previous workplaces. While the numbers should be a key driver at times they just hold a back seat to old habits and comfort levels.
Diego Klabjan, chair of the INFORMS University Analytics Program Committee and program director for Northwestern University's Master of Science in Analytics program, gives his advice for figuring out where to get an advanced analytics degree.
What Works: Open Source Analytics Software International Institute for Analytics WebinarOn Wednesday, Sept. 24, join IIA CEO and Co-Founder Jack Phillips, along with featured guest Gary Spakes, as we explore the five modernization stages that analytics hardware/software have experienced. We will discuss the considerations when calculating total cost of ownership of the analytics ecosystem.
2014 VA Interactive Roadshow -- Cary, NCThe 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Essential Practice Skills for Analytics Professionals Drawing on best practices from the field, this INFORMS course helps analytics professionals add value from beginning to end: listening to clients, framing the central problem, scoping a project, defining metrics for success, creating a work plan, assembling data and expert sources, selecting modeling approaches, validating and verifying analytical results, communicating and presenting results to clients, driving organizational change, and assessing impact.
Analytics 2014 The Analytics 2014 Conference is a two-day, educational event for anyone who is serious about analytics. This annual event brings together hundreds of professionals, industry experts and leading researchers in the field of analytics. All Analytics members save $500 on conference fees by using promo code ACAA.
Premier Business Leadership Series 2014 The Premier Business Leadership Series is an exclusive event for senior executives and decision makers that focuses on solving the current issues that affect governments and businesses globally. The Series is a unique learning and networking experience focused on the most innovative leadership strategies and analytic solutions for competing in todayâs global economy.
2014 VA Interactive Roadshow -- BostonThe 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Data Exploration & Visualization Get hands-on training that focuses on the critical steps in the process of analyzing data: accessing and extracting data, cleaning and preparing data, exploring and visualizing data. This INFORMS course will use several of the most popular software tools intensively, and provide an overview of the range of software options.
Foundations of Modern Predictive Analytics In this INFORMS course, learn about modern predictive analytics, the science of discovering and exploiting complex data relationships. This course will give participants hands-on practice in handling real data types, real business problems and practical methods for delivering business-useful results.
2014 VA Interactive Roadshow -- AtlantaThe 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
The hospitality industry gathers massive amounts of customer data, and mining that data effectively can yield tremendous results in terms of improved CRM, better-targeted marketing spend, and more efficient back-end processes. Roger Ares, vice president of analytics at Hyatt Corp., discusses the ways he and his staff use big data.
Charged with keeping track of travel assets, including employees, iJET International relies on data management best-practices and advanced analytics to keep its clients in the know on current and potential world events affecting travel, Rich Murnane, Director of Enterprise Data Operations & Data Architect, told All Analytics in an interview from the 2014 SAS Global Forum Executive Conference.
Jason Dorsey, chief strategy officer for the Center for Generational Kinetics and keynote speaker at last month's SAS Global Forum 2014, describes how Gen Y professionals are enhancing the makeup of multigenerational analytics organizations.
From analytics talent development to the power of visual analytics, All Analytics found a variety of common themes circulating throughout the exhibition floor and session discussions at the 2014 SAS Global Forum and SAS Global Forum Executive Conference events held last month in Washington, DC.
Talking with All Analytics live from the 2014 SAS Global Forum Executive Conference, Eric Helmer, senior manager of campaign design and execution for T-Mobile, discussed the importance of customer data -- starting internally -- in devising the mobile operator's marketing plans.
The big-data analytics market can be a confusing place. Among the vendors vying for your dollars are traditional database management providers, Hadoop startup services, and IT giants. In this video, All Analytics editors Beth Schultz and Michael Steinhart sit down in a Google+ Hangout on Air with Doug Henschen, executive editor of InformationWeek. Henschen discusses use cases for big-data analytics, purchase considerations, and his recent roundup of the top 16 big-data analytics platforms.
At the National Retail Federation BIG Show last month, All Analytics executive editor Michael Steinhart noted a host of solutions for tracking and analyzing customer activity in retail stores. From Bluetooth beacons to RFID tags to NFC connections to video analytics, retailers must find the right combination of tools to help optimize the shopper experience, streamline operations, and boost revenues.
The days when historical shipment trends and gut feelings were enough to forecast retail demand accurately are long over. SAS chief industry consultant Charles Chase outlines the benefits of pulling real-time sales information from point-of-sale and product scanner systems, then flowing that data into dynamic forecasting tools from SAS.
With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.