Nobody wants to be in the position of trading speed for accuracy of analytical work, but sometimes, you don't have much choice. The business has questions it wants answered, and you need to deliver -- the sooner the better.
Problem is, "as you're enabling more and more data to become usable, there are more and more questions being asked," Chris Gifford, senior vice president of customer analytics at JPMorgan Chase, recounted during yesterday's "The Big Reveal: What's Your Data Telling You?" session at SAS Global Forum Executive Conference 2013. Maybe the organization no longer wants to know about customer behavior in aggregate, for example, but on an individualized basis -- taking into account all touch points.
And so, you may well find yourself waiting... and waiting... and waiting for your answers as your models chug through all those bits and bytes. And then you may find yourself doing things that go against best-practices wisdom.
You might, as did the Chase customer analytics group Gifford heads, decide to select only some rows in your database to model rather than using all rows. Or, you might only look at some attributes and restrict others, or settle on one algorithm rather than test multiple algorithms to figure out the best solution to a problem. And maybe you wouldn't do quite as much testing of your model as you really ought, explained Gifford, who also told some of the Chase story during Sunday night's combined opening session for the SAS Global Forum 2013 and the executive conference, both taking place in San Francisco this week (watch the session here on demand).
At Chase, Gifford said he knew he needed to figure out a way that would let his analytics team stop cutting corners.
With a goal of increasing the speed and accuracy of the customer analytics models, Gifford said he began investigating high-performance computing solutions, like SAS High-Performance Analytics (HPA), that would streamline the time required for modeling runs. When first seeing SAS HPA in action, Gifford admitted he was a bit skeptical. "Could it really run fast with our models and our data?" he recalled thinking.
The proof-of-concept testing answered with a definitive, "Yes."
During that testing, his team ran a number of its traditional models. One, a risk model for its mortgage business, had been taking about 160 hours -- nearly one full week -- to complete. HPA testing showed the same model, with the same data, running in about 84 seconds. A credit risk model, which took 14 hours using the traditional SAS approach, ran in 180 seconds on HPA, Gifford said.
Needless to say -- although he did -- "those kinds of improvements are fantastic." And with them, we suddenly have options again, he added.
You can continue to chase the efficiencies. You can do more models per statistician per day or week. But you can improve your accuracy if you're willing to trade off some of that newfound speed. You can test additional algorithms or increase the sample counts -- more rows, more columns, perform more tests.
HPA has significantly improved the speed -- 200 times and 100 times performance improvements in the case of the two examples Gifford cited -- while improving accuracy. That's of no small significance for a financial institution the size of Chase. As he noted, "as we reduce the type one and type two errors for this kind of use case, it turns us into being able to say yes to more of the good guys and no to more of the bad guys -- so, yes, there are very big impacts."
At the end of the day, bringing data down to all these servers gives modelers the chance to rethink how they do things, too, he added. "The speed gets them ahead."
In fact, traditional business analytics speeds aren't going to cut it in the future. "As you keep moving more and more data to the edge of real-time decision making, you have to engage in this process."
I totally agree that trade offs between speed and accuracy are not acceptable these days. With constant research going on this field, it is highly unlikely that a solution isn't available to fulfill the business's need for high performance analytics and an application developer is the one who looses out if it isn't meeting the user's needs because there is someone out there who has a solution.
@waqasaltaf, while I agree solutions are available I don't think it's that simple. High-performance analytics/computing doesn't necessarily come cheap, for example, so there is budgeting to consider, for one.
Beth, true. Budgeting is not a factor to be ignored esp for a SME however large corporations may agree to pay as much as a high quality solution costs because of the economies of large scale they benefit from.
@Waqas, while waiting for a SAS Global Forum session on HPA to begin, i did overhear a couple of gentleman from a satellite TV company say they thought HPA was great, and would be useful, but at "a half a million bucks" unreasonable for its environment. I asked SAS CTO Keith Collins about this when I met with him the next day. He agreed that, yes, the initial offering was not affordable at large. But a June release changes that, eliminating the need for the pricey, dedicated appliance plus slicing up the functionality so you can buy only what you need and grow from there. Plus, companies will be able to take the SAS procedures they have running on a single server and move them over to an HPA cluster as needed.
One more thing I would like to add is that if ever a compromise needs to be made, it should be made on speed and not accuracy. What if you get an answer that is misleading but you receive it an hour earlier; it wouldn't help.
Faced with the alternatives between your position and one which suggests the risks to get some indication quickly trumps other concerns - my considered opinion aligns with yours.
Curiously, I have to admit to giving the issue as presented only a cursory review (haven't looked at any of the links provided, or even taken the time to read the post carefully); so I've opted for speed over accuracy. Yet, I've given the base arguments (as encountered in a number of presentations and discussions), considerable thought - enough to establish in my mind the value of the principle you and I share, in this regard.
What I just said may seem a contradiction; but I don't think it is. What it suggests to me is that there are principles which can serve us well, even in instances where arguments to the contrary are posed in a novel form. If we sense that a proposal suggests that speed is more important than accuracy, our principle keeps us from being taken in. At the same time, we can hold to that principle, yet be open to persuasion that the argument is actually suggesting a means to improve speed without risking misinformation - the world can always use a better mousetrap.
In support of our principle, we have last week's Wall St. flash-crash, as an example of the what can go wrong when speed trumps accuracy.
Avinash Kaushik, a well known web analytics practioner, noted the trade off between accuracy and precision. The comparison complements your comment about the principles that serve well even if augments to the contrary are posed in a novel form. Precision is preferred over accuracy, even if the intent is not what was expected. The reason is because of repeatability, being able to at least repeat an experience, and allowing our minds to organize the process and analyze it further. Speed and accuracy, however, introduces some interesting scenarios in either case. Is it better to faster know a problem vs zeroing in on an error even if the time arc differs.
In order to be fair to the points made in the post - I'll really have to read it more critically. I also have to think through your comments (though the question you raise about speed of awareness is obviously important; and it brings to mind the related issue of false positives).
It does seem, though, that there is a healthy regard for prudence in leveraging inferences drawn from analytics, expressed by most of those frequenting this site. I don't see that same level of discretion among other sample populations - where arguments about adverse consequences are either less well understood, or are trivialized as not relevant to new realities.
There is certainly truth in that statement, as it could be applied to many considerations. As I read your implication, you hold that the context of modeling and analytics are essential to determining what is acceptable, in terms of both the nature of the results, and the means of achieving them - the principles to which I adhere are in agreement.
The problem with such an endorsement, in many cases (though I can't say it applies in this case), is that the very context which the principle requires has been stripped from the data on which the processes operate, or even the processes themselves. The effect ends up being like some food being deemed both beneficial and detrimental to good health; without an understanding of the particulars of how these contradictory conclusions were derived, how can anyone make an informed decision as to what to eat or not eat? I think a question similar in nature is faced by those considering analytics strategies. With mushrooms, the safe assumption is to assume they are unsafe, unless proved otherwise - pretty much the same with speed vs. accuracy.
waqasaltaf, - If this was a debate i would be on the other side saying compromise on report if you have to. Speed is important..as they say late information is uselsss. I like to hope that if a report has to suffer from having been quick it would be on the completeness as opposed tp the accuracy.
Kicheko, you have raised a good point. Completeness is the thing to compromise on rather than the accuracy and that is what practically happens in practical life. Misleading information and delayed information; both are not acceptable.
Diego Klabjan, chair of the INFORMS University Analytics Program Committee and program director for Northwestern University's Master of Science in Analytics program, gives his advice for figuring out where to get an advanced analytics degree.
What Works: Open Source Analytics Software International Institute for Analytics WebinarOn Wednesday, Sept. 24, join IIA CEO and Co-Founder Jack Phillips, along with featured guest Gary Spakes, as we explore the five modernization stages that analytics hardware/software have experienced. We will discuss the considerations when calculating total cost of ownership of the analytics ecosystem.
2014 VA Interactive Roadshow -- Cary, NCThe 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Essential Practice Skills for Analytics Professionals Drawing on best practices from the field, this INFORMS course helps analytics professionals add value from beginning to end: listening to clients, framing the central problem, scoping a project, defining metrics for success, creating a work plan, assembling data and expert sources, selecting modeling approaches, validating and verifying analytical results, communicating and presenting results to clients, driving organizational change, and assessing impact.
Analytics 2014 The Analytics 2014 Conference is a two-day, educational event for anyone who is serious about analytics. This annual event brings together hundreds of professionals, industry experts and leading researchers in the field of analytics. All Analytics members save $500 on conference fees by using promo code ACAA.
Premier Business Leadership Series 2014 The Premier Business Leadership Series is an exclusive event for senior executives and decision makers that focuses on solving the current issues that affect governments and businesses globally. The Series is a unique learning and networking experience focused on the most innovative leadership strategies and analytic solutions for competing in todayâs global economy.
2014 VA Interactive Roadshow -- BostonThe 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Data Exploration & Visualization Get hands-on training that focuses on the critical steps in the process of analyzing data: accessing and extracting data, cleaning and preparing data, exploring and visualizing data. This INFORMS course will use several of the most popular software tools intensively, and provide an overview of the range of software options.
Foundations of Modern Predictive Analytics In this INFORMS course, learn about modern predictive analytics, the science of discovering and exploiting complex data relationships. This course will give participants hands-on practice in handling real data types, real business problems and practical methods for delivering business-useful results.
2014 VA Interactive Roadshow -- AtlantaThe 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
The hospitality industry gathers massive amounts of customer data, and mining that data effectively can yield tremendous results in terms of improved CRM, better-targeted marketing spend, and more efficient back-end processes. Roger Ares, vice president of analytics at Hyatt Corp., discusses the ways he and his staff use big data.
Charged with keeping track of travel assets, including employees, iJET International relies on data management best-practices and advanced analytics to keep its clients in the know on current and potential world events affecting travel, Rich Murnane, Director of Enterprise Data Operations & Data Architect, told All Analytics in an interview from the 2014 SAS Global Forum Executive Conference.
Jason Dorsey, chief strategy officer for the Center for Generational Kinetics and keynote speaker at last month's SAS Global Forum 2014, describes how Gen Y professionals are enhancing the makeup of multigenerational analytics organizations.
From analytics talent development to the power of visual analytics, All Analytics found a variety of common themes circulating throughout the exhibition floor and session discussions at the 2014 SAS Global Forum and SAS Global Forum Executive Conference events held last month in Washington, DC.
Talking with All Analytics live from the 2014 SAS Global Forum Executive Conference, Eric Helmer, senior manager of campaign design and execution for T-Mobile, discussed the importance of customer data -- starting internally -- in devising the mobile operator's marketing plans.
The big-data analytics market can be a confusing place. Among the vendors vying for your dollars are traditional database management providers, Hadoop startup services, and IT giants. In this video, All Analytics editors Beth Schultz and Michael Steinhart sit down in a Google+ Hangout on Air with Doug Henschen, executive editor of InformationWeek. Henschen discusses use cases for big-data analytics, purchase considerations, and his recent roundup of the top 16 big-data analytics platforms.
At the National Retail Federation BIG Show last month, All Analytics executive editor Michael Steinhart noted a host of solutions for tracking and analyzing customer activity in retail stores. From Bluetooth beacons to RFID tags to NFC connections to video analytics, retailers must find the right combination of tools to help optimize the shopper experience, streamline operations, and boost revenues.
The days when historical shipment trends and gut feelings were enough to forecast retail demand accurately are long over. SAS chief industry consultant Charles Chase outlines the benefits of pulling real-time sales information from point-of-sale and product scanner systems, then flowing that data into dynamic forecasting tools from SAS.
With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.