Well Jeanne, I think we're just about out of time. I want to thank you for sharing your fantastic insight. I also want to thank all of our participants, vocal or not. And, I'd like to invite folks to take an MIT SMR survey on how they're dealing with the big-data deluge. You can do that here
If we have an app that can help us address very specific questions from massive amounts of unstructured data, we have something that could be valuable. I've been imagining more general data mining efforts.
This is more the kind of "killer app" thing I've had in mind. The app developer would need to look over samples of the data, detect possible patterns for value, agree on a strategy with the organization, and then write an app that would yield valuable results. Hopefully, the results would not just add to the growing mountain of junk data...
Top companies know how to determine what has business value. They demand a strong business case for every project and they check at every milepost whether that business value case is still solid and then they check afterwards if they got the value. It helps them decide what is and isn't worth pursuing. This is a learning process that will be valuable in the analytics space.
MicroStrategy has been doing a lot of work in the Social Analytics space. Their "Wisdom" app allow you to mine Facebook data and analyze social trends. They have some interesting examples, such as determining marketing opportunities based on "Likes" across many people, pages, and products.
@BethSchultz, there's room for competitors to leapfrog the top firms just by implementing a more comprehensive data function company-wide. There's no limit on tech advances and just one misstep can change the leaderboard. Do RIM & Nokia have the best analytics in their industry? I think not.
@CemChase I think IT units are taking more of a leadership role in identifying what is and isn't valuable. As they do so, they are less run by business requests and they learn a lot about whawt drives business value--and what doesn't.
We have a nice case study on how Trinity Health (a hospital conglomerate) has been building an analytics capability. It's been a long journey. If you're interested, send me an email (firstname.lastname@example.org), and i'll send you the case.
I think the companies that are findings lots of value in analytics have lots of transaction data, because it affords so many opportunities for analysis. So i expect that retailers like Wal-Mart, Amazon and Target are ahead of most companies in most industries.
Nice example Lyndon. But different from some of the data mining that we were talking about, I think. If we have an app that can help us address very specific questions from massive amounts of unstructured data, we have something that could be valuable. I've been imagining more general data mining efforts.
Jeanne, a lot of companies seem to be spending lots of effort in understanding what's being said on the social networks about them and their brands, so a particular type of unstructured data. Given your description of sacred data and what you've said about unstructured data, I have to ask how you feel about trying to glean sentiments and what-not from social data ....
I think the top companies in every sector have gone well beyond simple analytics and obvious transactional reporting. They'll have dozens of analytical models covering all aspects of their business and will have initiatives to analyze new data sets, regardless of their (un)structure, source or size.
Beth asks about my experience with old data. Well, yes. Transit agency historical records from the founding of the agency. Needed to solve problems, answer critics. Unfortunately, almost entirely paper (yuk). But absolutely essential in answering key questions 15-20 years later. Hard to sort through, some of it still in my brain. Nice killer app would be great, one that could sift through boxes of paper...
But, the same thing could happen with Digital Age records. You know, all those docs entered in WordStar and Lotus 1-2-3?
There has been a lot of discussion of structured versus unstructured data, and the consensus is out on the value of optimizing toward heavy undirected analysis of unstructured data. So what does it mean to get better at structuring data? That seems far more subtle and dynamically proactive than simply defining tags /labels / object models extensible to future data sets. What will real progress look like there? Will it be tied more closely to enterprise needs?
Analytics models ARE strategic assets!. A few weeks ago the NYTimes had a great article about Target using their vast data to pre-market to pregnant women...some of whom hadn't even acknowledge publicly that they were pregnant yet. That's a powerful capability, and one that Target was most interested in protecting.
Sacred data is the transaction data that helps you know your customers, products, suppliers, and that has obvious value when analyzed. I think if we start there instead of looking for big, glitzy things, we'll learn how to drive value and then we can get a bit fancier.
@PredictableChaos, I think that "strong IT person" who knows analytics needs to be embedded in the business and report to the business. I've seen a lot of talent locked up on the IT side that could be generating a ton more business value if they worked directly for the front-line business leaders.
Jeanne, of all that stockpiled, out-of-control data ... is it mainly internal to the organizations studied, or does some include Big Data gathered via data-minining techniques, e.g., Facebook skimming, etc.?
In the article, we talk about sacred data and unstructure data that is critical to structured processes. I think we can lose that very valuable data if we believe that all data needs to be safed somewhere.
@egerter - For data analytics to work requires a strong IT person together with someone from the buisiness side who is willing to work at explaining and re-explaining what reports, what information would be most helpful. Where i've seen this work, there is a lot of back and forth to get to the summary reports that are actually helpful.
@Jeanne -- so, not to be trite, but analyzing unstructured data is sort of like putting the cart before the horse? (Which really applies to analyzing structured data without understanding the purpose, too, of course).
I questions whether storing data until we figure out how to use it means we are storing data that will be irrelevant by the time we analyze it. Someone once told me that NASA hadn't kept some old records on how they build early rockets--they thought that was a problem. I can't imagine we lost anything valuable there.
Jeanne, it sourt of sounds like you're suggesting organizations implement a strategy for evaluating and storing their data. Good idea. But a lot of the time, the data are pouring in, and it's maybe years before somebody looks are figures out how it should have been organized.
@Beth, I would try to get value from data analytics and accompanying technologies by identifying one business process that my company thought was critical to long-term success. Then i'd ask waht data it depends on, and then i'd get good operational data, and then i'd get analysts of all kinds to understand what is going on.
IT needs data curators, people who understand how to capture, store, document and disseminate data sets. Getting these data sets in front of business analysts is key to getting value from them. Having IT do analysis themselves is a losing scenario; they don't know the business as it stands today and their analytics tend toward simple counts and correlations. The business side (at the department level) has the field of vision to understand current challenges and will need data (properly analyzed) to support their budget requests.
We have been thinking that things like semantic webs etc were going to solve the lack of business focus and discipline for a long time. I can't imagine an app that simply "finds" the value in all our data that is lying around. We still need to know what we're trying to do.
Jeanne, given all the problems IT/enterprise data architects/analysts, etc. have in dealing with the data companies have today, what's your advice for how to approach the data of tomorrow -- the more complex and unstructured data increasingly gathered. Do they start from scratch? Bring in new technologies like Hadoop? Or build off of what they have in place today?
Oh, OK. So i think there's a gradual building of capabilities. Companies aren't likely to use predictive analytics well until they have good "habits" around business processes and customer relationships. You can analyze anything but whether you get good output or can make changes based on that output assumes some level of organizational maturity around business processes.
Jeanne, well, I can't find James' "official" definition of a DMS but in essence he's talking about a next-generation information management system that combines and applies predictive analytics, optimization and intelligent business rules to help companies automate certain types of decisions.
I have studied enterprise architecture for a long time and i noted the huge potential for business value from business process standardization. But many companies don't get nearly as much value as they think they will after they put in their ERP or CRM because they don't reassign responsibilities. So often the problem is that they never clean up or use the data that these systems make possible. So i've started studying data.
Safe, meaning they dispose of old data on a regular basis due to regulatory or legal concerns, or limit access to data due to concerns about theft or disclosure. Companies that can navigate these concerns and keep their historical record intact have a higher potential for learning and competitive advantage.
I'm trying to imagine a formalized data management system. There are some decision rules that we can build into automated systems--and it's good to keep people simply as oversight for when quirky things happen. Then there are people decisions that rely on rules, but i'm not sure they are formalized. Am i understanding you?
@egerter: re your point, "companies that value "safe" over innovation are risking being blind-sided by more technology-adept competitors that do keep all of their data and find ways to extract value from it" -- I'd argue that they risk being blindsided period, even by those companies that aren't keeping all their data!
Ah, the ideal analytics employee is a number of people. For the Big Data issues like engineering control systems or supply chains, get experts who understand statistics, logic, and biological systems. You also need data architects, and domain experts. It takes a team. They all need communications skills.
I don't think we "accidentally" get value fromd data that happens to be around. We get value from knowing what customer intimacy or efficiency or whatever goals will make a difference and then finding specific data that helps us understand what we've got.
The thing about data is that we tend to think the big opportunity is data mining and analytics. I really think it's more operational than that. Get data into the hands of people who make daily operating decisions, arm them with good business rules, and you can see a big business impact.
IT tends to have a service-provider mentality and isn't often willing to step up to the table to be an equal partner in the business. Unfortunate, as this results in much waste and many lost opportunities.
Storage is cheap, but finding stored data can be a nightmare today unless you know where the data is stored. We need a better object tagging system to help us find years-olds (or months) hidden nuggets. Jeanne and others, would you agree?
IT governance bodies should drive entriprise data retention policies by setting the templates and working with individual application domains to ensure compliance just like IT Risk etc. But data usually gets left out somehow.
I don't think that an analytics team can reside exclusively in IT unless you move domain experts into IT. The boundaries between IT and business are fading and this is one area where that will lead to huge benefits.
Jeanne, in the article you talk about IT being a good data steward and that business needs to take a leadership position here. Where does the analytics team — sometimes positioned within IT, sometimes in the business, and sometimes independent — fit into all of this?
Re: differences between public and private sector ... Public sector agencies are under more public and political pressure to reveal organizational info. They tend to get into more trouble if they trash Emails from 25 years ago, than a private business. So, just wondering about aspects like that...
I think @danmeier and @egerter are spot on. And cheaper storage will just make it worse, as big-data companies keep promising that you will discover the long-lost secret to instant customer satisfaction if you just have enough data and ask the right question. Kind of astrology.
The great companies are focused on strategic priorities and the data and processes that will help them achieve those priorities. We are seeing that at companies like USAA, P&G, Commonwealth Bank of Australia, etc.
Yes, i think that's right. It's not clear that the packrat mentality is a problem except for potential legal liabilities. So i think we need good direction on that and we probably shouldn't worry about the rest.
Working for a financial industry, data retention policies have been very clear. So it has been easy to decide when to indefinitely store, 7 yrs, 10yrs, etc. However, in other I worked for "just in case" was killing system performance if not creating liability.
Difficult for an IT department somewhat divorced from the business side to make the determination that a particular data set is not "worth keeping". With cheap storage (1 PB = $100K), it's easier to just store everything and maybe someone someday will find something of value.
IT doesn't have the business knowledge to decide what data is relevant, and the business users don't have the time or inclination to analyze what they really need. The result: "store everything just in case." They figure it out as they go along. The problem is that the business users never go back to review the data they're storing that they've found they don't need.
You would think companies would establish schedules for storing and deleting data -- for these very reasons. And some do. But how do you catch every version of a document? People tend to make copies of copies -- for safekeeping.
We are starting to see the creation of data management departments but their mission isn't yet clear. I would recommend starting with a workflow department. First figure out who is using what data for what and then you can manage the data and the flows.
Yes, i think people increasingly believe that storing too much creates a legal risk, but getting rid of anything that a company is legally required to keep is also an issue. Most lawyers know what is necessary; i don't think most people in their companies do.
A "just in case" item would be extensive transaction records--just in case we want to analyze who is buying what (10 years ago) or massive amounts of email because we can't figure out legally whether we're better off with too much or too little.
Right, Rodney. I think we're not sure that cloud will be cheaper for data that we may never need. I personally pay monthly for storage of stuff that i probably should have thrown away. That would certainly be the cheaper alternative.
@Jeanne, when you say IT has "little interest in taking the time or risk to identify data that is expendable" that smacks of the age-old problem of having a disconnect between IT and the business. Does the arrival of "big-data" exacerbate that problem, in your opinion?
That seems to be a selling point for the cloud vendors Jeanne - cheaper IT than do-it-yourself. That should lead to a value return to the companies, but maybe it really is just the first inning, and no one is really sure how the game will play out. But they still have to play it.
The management practice most often mentioned was just better tiering and archiving. There is little interest in taking the time or risk to identify data that is expendable. So IT units are just trying to minimize the cost without losing anything.
Jeanne -- that is a big issue -- meaning, saving data "just in case" -- especially as the volumes keep swelling so. I can't imagine coming to an answer is easy to do. What did some of the folks you studied have to say about how to come to a resolution?
I love the question about the cloud impact. I don't actually think the cloud will have an impact on the value question. I think that companies manage and use data well--or not. The cloud is another tool for storing and accessing. Would you expect an impact, Rodney?
We have done a number of studies on the topic of data, analytics, and what we call "working smarter." The study on which our SMR article was base involved interviews at 26 companies who we thought would have big data needs.
Mark your calendars for an e-chat here on Thursday, June 28, at 4 p.m. ET. We'll be talking with Jeanne Ross, director of MIT Sloan School of Management's Center for Information Systems Research, on how to find value in the information explosion.
At the CISR, Jeanne directs and conducts academic research that targets the challenges of senior-level executives at the center's more than 80 global sponsor companies. Besides her work at the CISR, Jeanne also teaches in MIT Sloan's Transforming Your Business Through IT Executive Education Course.
Preserving documents, emails, and personnel records is smart for businesses and critical for government entities. Advanced text mining technologies make the process easier and smoother by automating routine classification tasks.
Tech Marketing 360 The only event dedicated to technology marketers. Discover the most current and cutting-edge innovations and strategies to drive tech marketing success. Hear from and engage with companies like Mashable, SAS, Dun & Bradstreet, ExactTarget, Google+, IDC, Microsoft, LinkedIn, Oracle Eloqua, Leo Burnett, Young & Rubicam, Juniper Networks and more – all in an intimate, upscale setting. Register at http://www.techmarketing360.com with priority code CMANALYTICS14 to save $100.
SAS Global Forum Executive Conference 2014 The Executive Conference is held in conjunction with SAS Global Forum, a SAS users technology event. Investing in thought leadership and technical training are two of the best moves a successful company can make so take advantage of the world-class speakers, sessions and discussions around Analytics, Big data, Risk, Fraud and Data management.
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
David Tishgart, senior director of marketing and alliances at security provider Gazzang, explains the importance of data encryption for companies that are rolling out Hadoop environments to leverage big data analytics.
At the Strata Conference / Hadoop World 2013, Samuel Kommu, technical marketing engineer at Cisco Systems, shares some of the benefits that Hadoop brings to analytics platforms that leverage next-generation hardware. Kommu looks at big data operations that required 3,500 nodes in 2009, 2,000 in 2011, and now require only 64 nodes.
Wayne Thompson, manager of SAS Data Sciences Technologies, delivers a fascinating preview demonstration of SAS Visual Statistics, a tool that enables fast and flexible modeling against massive datasets on the fly. Visual Statistics will be made generally available in March, but you can see it here first.
At Strata/Hadoop World 2013, Cloudera CEO Tom Reilly discusses the new Enterprise Data Hub offering, explaining how it works with Hadoop, how it creates a single repository of full-history and full-fidelity data, and how it exposes that data to all users interested in exploratory analytics.
At this year's Strata Conference/Hadoop World 2013, SAS big data vice president Paul Kent presented a session on setting up Hadoop clusters for advanced analytics. We caught up with several audience members and recorded their impressions of the presentation.
In hearing directly from a doctorate-level Hadoop specialist, a healthcare data analyst, and a marketing executive, it's clear that big data analytics is a burgeoning field that cutting-edge companies are eager to explore.
At this year's Strata Conference/Hadoop World 2013 event, SAS VP of Big Data Paul Kent presented several sessions about modernizing and deploying advanced data analytics infrastructures based on Hadoop. In this video, he talks about the state of Hadoop adoption among enterprises today and looks out to the big data-driven applications of the future.
Companies that use SAS analytics tools for their traditional databases are looking to derive even more value by mining unstructured data. Data management platforms like Hortonworks enable that relationship by delivering an enterprise-ready Hadoop framework.
In this video, Shaun Connolly, vice president of corporate strategy at Hortonworks, explains how companies can incorporate Hadoop into their data analytics streams.
At the SAS Premier Business Leadership Series in Orlando, Manuel Sanchez, CRM Manager for Club Premier Aeromexico, explains the challenges and opportunities of transaction data. Using dozens of data sources among participating airlines and merchants, Club Premier creates robust customer profiles and works to maximize benefits for members and business partners alike while protecting individual privacy.
At SAS's October Premier Business Leadership Series (PBLS) in Orlando, attendees from the corporate and academic worlds joined thought leaders and analytics professionals to share insights and strategies around big data.
Will Hakes, CEO and co-founder of Link Analytics and keynote speaker at the SAS Analytics 2013 conference in Orlando, Fla., last month, talks candidly about the challenges that large enterprises face as they explore advanced analytics solutions. He also shares some practical tips for smoothing the transition.
At the SAS Analytics 2013 conference in Orlando, Bob Gladden, vice president for decision support and informatics at the Ohio nonprofit health insurance provider CareSource, explains how his company uses advanced analytics to keep administrative costs down and to identify at-risk patients for targeted healthcare initiatives.
At the Analytics 2013 conference in Orlando, Fla., two analytics experts from Dell -- global decision sciences manager Natalie Kortum and senior credit risk consultant Jack Chen -- share their real-world advice for analysts who want to sell their project ideas to business executives.
At the SAS Premier Business Leadership Series in Orlando, Fla., Lousiana State Representative Chris Broadwater outlined the state's success with analytics-driven fraud detection and shared his vision for streamlined processes at the DMV, the healthcare system, and even the department of corrections -- all delivered via a centralized repository of rich customer data.
Organizations that are ready to leverage big data need to move beyond buzzwords and approach the challenges with a business focus. Peter Guerra, principal at Booz Allen Hamilton, shares his insight and experience in helping clients transition to Hadoop and embrace new decision support platforms.
At this year's Strata Conference / Hadoop World 2013, Michael Steinhart chats with Rackspace Product Marketing Manager Sean Anderson about Hadoop, cloud computing, and how the two come together for companies that want to undertake a "proof of value" project.
With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.