Situation: You have read a report that describes the objective, methods, results, and significance of an analytic inquiry. The author’s objective is not 100% clear to you, but you attribute that to the fact that highly intelligent people communicate at a pitch that only Mensa members can hear.
The literature review references doctoral dissertations written in German. Every fifth sentence uses a Latin phrase (like Ceteris paribus -- all things being equal). There is enough linear notation to require a degree in quantum physics to understand. The explained variance in the regression model is less than 3%, but the author frames the results in language that suggests the results are ground-breaking.
There are even several charts and graphs that tell a story in pictures, but you are too ashamed to tell someone that they do not make sense to you. When you Google the author, you find that he or she hosts YouTube tutorials and authors several blogs.
Question: As a professional analyst, should you readily accept the contents of the report?
Answer: Not on your life! As an analytics professional, especially if you are just getting started in the field, it is important to know that everything that is presented or published is not always correct or unchallengeable. One of the skills you will need to be successful in analytics is the ability to respectfully question and push back on statements that do not make sense.
The analytics profession bears the burden of producing results. When the results are more mind-numbing than awe-inspiring (i.e., studies show that being alive is the largest risk factor of experiencing death), authors can be tempted to "put lipstick on the pig" to justify the project's funding. Here are some embellishment techniques that you should look for in the analytic report:
Attempting to establish a big truth with a small sample size. Nine out of ten doctors may recommend a particular glucose drug for diabetics, but how many doctors (in the US, Brazil, China -- location is important) does this constitute? It could be nine out of ten doctors in the same HMO group in Rhode Island or Wyoming.
Distorting details of tests. Stating that rats who ate chocolate developed cancer does not state the proportion of chocolate consumed. In some of the animal studies, the test subjects consume twice their body weight of a substance for a period of time, and then develop a morbid condition. But any human who consumes twice his/her body weight of anything will also develop a morbid condition.
Understating or misstating units of measurement. One bad habit I have seen is when an author uses charts/graphs from other studies as points of comparison, but the units of measurement are not the same. This becomes more problematic when trying to overlay two graphs that are scaled differently.
Presenting misleading comparisons. This is one of my favorites: "Software product A is faster than software product B." The question to ask here is, do the two products execute in the same manner? Some products write to disk while others write to RAM. If you purchase the RAM-based product without also buying more RAM, the speed differential may be non-existent.
Changing the hypothesis to fit the results. This is one is self-explanatory. It's unethical and only done by those who feel that it is in their best interests to do so.
In short, part of being an analytics professional is assessing the work of other analysts. While we seek to leverage our curiosity and tenacity for learning, a healthy dose of skepticism is in order. We are a self-governing body and have to keep each other accountable in advancing the profession. Respectfully pushing back helps to push us forward.
I would add a couple additional "leadership" animals --- the snake, the passive-aggressive types who will destroy you when you turn your back on them, and the vultures, who wait out all the foxes and lions and pick their carcasses after the fight.
I greatly admire your optimism; I used to think just like that when I younger. It's wonderful to be a middle-aged person, but the downside is that time has a way of making one a skeptic. I agree that over time, work places will be less combat-oriented. I think that the gradual increase in women in the white-collar labor force is slowly forcing work place culture to be less Spartan-like (or at least the lawyers are doing that). But your hopes for a better day remind me of Vilfredo Pareto's theory of the circulation of elites. He said that leadership styles cycle through periods of lions (traditional male brute force and intimidation) and foxes (people who can covertly manipulate, massage and convince people to do their bidding). I can see a day when cultures are kinder and gentler. But I would warn you that whether you work with lions or foxes, the outcome for the slow and weak will be the same. Foxes are just as predatory as lions, they just are kinder and gentler in their approach. But ultimately, the goal is to 'do you in'; a soft approach but just as deadly. So keep your whether combative or diplomatic, keep your skepticism radar up - trust but verify!
@bkbeverly, you are right. Some "old dogs" still like to fight like young pups. And there are still plenty of dogs that will wait till your not looking and will nip at your heels. One day all company cultures will be kinder and gentler but until then ...
Could not agree with you more. Bias is a chronic problem in sampling. As in the case you mentioned, it was a random sample of people with at least a BA/BS degree. Bias is also a problem in survey design and reporting. People with advanced degrees tend to write to the level of their peers. Some of these survey results represent a 'privileged class' and not the thoughts of Joe or Jane Lunchbucket. Nothing wrong with that as long as you state that the external validity applies to white collar professionals. When your colleagues, neighbors and Facebook friends are in the same socioeconomic status groups, then one's perception of applicable analytical truths is skewed. Not intentional, not malicious, but innate and habitual. When the only people in your circle of associations are like you, then the assumptions or assertions you make should be 'viewed with a jaundiced eye'. Good observation (as always).
@Broadway - Not a specific event. Just a gradual realization that it was a hollow victory to attack a person and not constructively work toward refining ideas. Particularly since many environments are charged with testosterone, challenging a coworker or a superior that you wanted to knock off was expected. When you were a young pup, you wanted to prove that you were ready to be a big dog. But when you are a young pup, you do not know that every big dog one day becomes an old dog. No not a specific incident in this case - just a gradual maturing. Every one grows old, but some never grow up; there are some gray haired babies whose insecurities find it better to reaffirm themselves by attacking others rather than preserving the dignity of others and helping them to shape their ideas. It is nothing wrong with being a skeptic, but it can turn out wrong if it is done the wrong way for the wrong reason. So all that to say is that these are the lessons learned from an old dog. I can't learn any new tricks, but I know when someone is trying to play them.
I love this post - these are terrific points, and all should be taken into account!
May I also add that we should examine methods for signs of plain old ordinary bias? For example, I've seen a number of posts recently about a data science salary survey. Most of the posts push the value of certain skills, based on the survey results.
A quick review of the survey report reveals that all the respondents were attendees of one particular conference. That conference focuses on certain specific skills, and certain types of applications. It's by no means a representative sample of the analytics community, or even the "data science" community as a whole. It's a biased sample, and it's obvious that nobody even tried to eliminate bias when planning the survey.
I used to be one of those persons! In an academic environment, that is the way of life. It feeds the ego when you think you have won a round of one-upsmanship. But over the years, I learned that it is better to make colleagues than enemies. In fact, "there are no permanent friends and there are no permanent enemies, but there are permanent interests". The person you intellectually attack today may be a needed friend tomorrow. And since all of us have strengths and weaknesses, you really gain nothing by seeking to show up someone because what goes around comes around. So over the years, I found it more constructive to question the range of options rather than attack a person. Oh I am quite tempted sometimes when someone else makes a first strike at me; the natural response is to hit back. And even then I have to quickly assess if the statement is personal or professional. Cannot say that I always respond like a grown up. But I do try not to initiate conflict. So yes, I try these days to take a softer approach, but oh no, that is not how I started out.
@bkbeverly, granted I was in grad school nearly 15 years ago and it was for a liberal arts degree, but back then students and teachers didn't question -- they always accused and attacked. Why is this relevant? Etc. etc. perhaps it was the culture of that school.
Diego Klabjan, chair of the INFORMS University Analytics Program Committee and program director for Northwestern University's Master of Science in Analytics program, gives his advice for figuring out where to get an advanced analytics degree.
What Works: Open Source Analytics Software International Institute for Analytics WebinarOn Wednesday, Sept. 24, join IIA CEO and Co-Founder Jack Phillips, along with featured guest Gary Spakes, as we explore the five modernization stages that analytics hardware/software have experienced. We will discuss the considerations when calculating total cost of ownership of the analytics ecosystem.
2014 VA Interactive Roadshow -- Cary, NCThe 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Essential Practice Skills for Analytics Professionals Drawing on best practices from the field, this INFORMS course helps analytics professionals add value from beginning to end: listening to clients, framing the central problem, scoping a project, defining metrics for success, creating a work plan, assembling data and expert sources, selecting modeling approaches, validating and verifying analytical results, communicating and presenting results to clients, driving organizational change, and assessing impact.
Analytics 2014 The Analytics 2014 Conference is a two-day, educational event for anyone who is serious about analytics. This annual event brings together hundreds of professionals, industry experts and leading researchers in the field of analytics. All Analytics members save $500 on conference fees by using promo code ACAA.
Premier Business Leadership Series 2014 The Premier Business Leadership Series is an exclusive event for senior executives and decision makers that focuses on solving the current issues that affect governments and businesses globally. The Series is a unique learning and networking experience focused on the most innovative leadership strategies and analytic solutions for competing in todayâ€™s global economy.
2014 VA Interactive Roadshow -- BostonThe 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Data Exploration & Visualization Get hands-on training that focuses on the critical steps in the process of analyzing data: accessing and extracting data, cleaning and preparing data, exploring and visualizing data. This INFORMS course will use several of the most popular software tools intensively, and provide an overview of the range of software options.
Foundations of Modern Predictive Analytics In this INFORMS course, learn about modern predictive analytics, the science of discovering and exploiting complex data relationships. This course will give participants hands-on practice in handling real data types, real business problems and practical methods for delivering business-useful results.
2014 VA Interactive Roadshow -- AtlantaThe 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
The hospitality industry gathers massive amounts of customer data, and mining that data effectively can yield tremendous results in terms of improved CRM, better-targeted marketing spend, and more efficient back-end processes. Roger Ares, vice president of analytics at Hyatt Corp., discusses the ways he and his staff use big data.
Charged with keeping track of travel assets, including employees, iJET International relies on data management best-practices and advanced analytics to keep its clients in the know on current and potential world events affecting travel, Rich Murnane, Director of Enterprise Data Operations & Data Architect, told All Analytics in an interview from the 2014 SAS Global Forum Executive Conference.
Jason Dorsey, chief strategy officer for the Center for Generational Kinetics and keynote speaker at last month's SAS Global Forum 2014, describes how Gen Y professionals are enhancing the makeup of multigenerational analytics organizations.
From analytics talent development to the power of visual analytics, All Analytics found a variety of common themes circulating throughout the exhibition floor and session discussions at the 2014 SAS Global Forum and SAS Global Forum Executive Conference events held last month in Washington, DC.
Talking with All Analytics live from the 2014 SAS Global Forum Executive Conference, Eric Helmer, senior manager of campaign design and execution for T-Mobile, discussed the importance of customer data -- starting internally -- in devising the mobile operator's marketing plans.
The big-data analytics market can be a confusing place. Among the vendors vying for your dollars are traditional database management providers, Hadoop startup services, and IT giants. In this video, All Analytics editors Beth Schultz and Michael Steinhart sit down in a Google+ Hangout on Air with Doug Henschen, executive editor of InformationWeek. Henschen discusses use cases for big-data analytics, purchase considerations, and his recent roundup of the top 16 big-data analytics platforms.
At the National Retail Federation BIG Show last month, All Analytics executive editor Michael Steinhart noted a host of solutions for tracking and analyzing customer activity in retail stores. From Bluetooth beacons to RFID tags to NFC connections to video analytics, retailers must find the right combination of tools to help optimize the shopper experience, streamline operations, and boost revenues.
The days when historical shipment trends and gut feelings were enough to forecast retail demand accurately are long over. SAS chief industry consultant Charles Chase outlines the benefits of pulling real-time sales information from point-of-sale and product scanner systems, then flowing that data into dynamic forecasting tools from SAS.
With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.