As an analyst working with newer forms of communication and emerging media, I've seen a proliferation of analytics platforms aimed at these areas. I don't get it. Why, I wonder, do we need new analytics platforms when plenty of excellent ones are already available?
Could it be that, as new data formats arise and we face the constant need to come up with new analytics capabilities, we're falling behind? Instead of finding a more persistent and long-term solution to the data fragmentation dilemma, are we generating more and more one-off solutions?
The problem lies in the fact that most of today's analytics platforms are "static," as Ravi Kalakota, a managing director of the global professional services firm Alvarez & Marshal, discusses in this blog post.
The dominant design of such platforms is dependent on a specific set of questions and dimensions. However, as businesses and individuals are evolving, they face a mushrooming cloud of fragmented information they don't know how to use, along with a shortage of qualified personnel to handle it. Their in-house or third-party platforms aren't flexible enough to meet their needs, so they buy or develop different analytics platforms -- platforms that often overlap in data and purpose (particularly in the social media space). In particular, platforms fail to allow ad hoc exploration using real-time data feeds. If they do allow for such exploration, the process is time consuming and costly.
Beginning in 2010, we've seen many acquisitions in the social analytics and cloud data space, but there hasn't been much in the way of creating a more dynamic approach to analytics platforms, a la the analytics-as-a-service concept discussed in this Sand Hill blog.
In the emerging media space where I'm most focused (through client work and the social media measurement and social media for the arts courses I teach at University of California-Irvine Extension and at Rutgers University, respectively), I've come across three constructs that explain the analytics fragmentation we face and suggest possible solutions we can explore.
Dark data: This is the data organizations generate but don't understand how to optimize or use, so they aren't drawing much insight from it. As Wired Magazine wrote several years ago, "Freeing up dark data could represent one of the biggest boons to research in decades, fueling advances in genetics, neuroscience, and biotech." But unless we have questions to apply against the data, it may make no difference what the data is. Rather than focusing on the data, maybe our focus really belongs on the nature of our questions and how our data can help answer them. Dark data might be worth exploring.
Dark social: This aspect of dark data is related to social media activity that isn't easily traceable to the actual origin point, as the Atlantic recently discussed. Web analytics is limited in what it can capture, and it has to be instrumented deliberately to capture much of anything specific, which leaves out much of social data activity. In addition, the privacy settings in Facebook, LinkedIn, most forums, and some photo-sharing and blogging sites make sharing social data across analytics applications difficult. Attempts to solve the problem have simply generated more analytics fragmentation.
Ultraviolet data: I coined the term "UV data" in 2010, when I was among the first analysts to recognize a problem that analytics platforms weren't designed to solve. That problem: Businesses aren't equipped to capture much of the data they need, but data is present nonetheless, and it is waiting to be collected. It's quite possible the uncollected data is more valuable than the collected data. I've devised an audit process for discovering UV data, and I am figuring out how best to collect and organize it.
No doubt, all the forms of dark data, dark social, and UV data boil down to the same thing -- what we don't know might be as important as what we do know, if not more. It also explains why many analytics platform fail to drive the needed insights: They're based on static dimensions and haven't been built to answer the questions that we must increasingly solve in order to succeed.
I like the terminology and will love to float the term around and see people's reaction.
The thing with many old systems are they are not flexible, scalable and tend to be products of departments silos , with one or two purposes that need to become more connected. Hence, giving employees access to the whole brain and not just one part.
I think the right question would be contextually correct to the data you have to work with on one hand and your core needs, on the other other. I think it takes some perspective (ha, the title of my post!) in order to come up with the best questions, and I think some give and take, which a team of trusted advisors, and perhaps one or two new people who can introduce new perspectives, might be the best way to go.
I often worry that by trying to come up with the questions myself, without really hashing them out in group discussion, a too near sighted formulation comes up that ends up missing the point, and miss the insights - the very thing that leads to Dark Data, perhaps Dark Social, and certainly UV Data.
We have a lot of information in front of us constantly, but unless we have a reason (need) and wonder if the data has a solution for us, we may never do anything.
People are need based - until we have a need (which generates a series of questions) to understand and solve a problem (often one that is thrust upon us) we may never compose the questions that lead us to examine the data in forint of us.
That's exactly the situation I posited we had at my Rutgers University class that I teach on Social Media for The Arts. The first temptation was to give students the fundamentals of self promotion so they could take the Art and Humanities trading they get and apply it in Social Media.
But from my perspective ... Simply getting someone to apply what they know on a new medium (dark social in this context) is not as useful as if we taught them more important questions to ask in the first place.
That required exposing a new framework and that leads to new questions, which in turn leads us to examine the data around us, figure out if it might be useful, and then wonder how and what.
Hi Marshall, I'm trying to get my mind around this point you make about dark data: "But unless we have questions to apply against the data, it may make no difference what the data is. Rather than focusing on the data, maybe our focus really belongs on the nature of our questions and how our data can help answer them." Don't you have to know the type of data you have to know the type of questions you can answer? Unless you're also including external data in this dark data category?
Dark data either way is a benefit. Whether data is determined as "useless" it can easily be deleted and written off as "no value." Integrating dark data with existing data can prove to be significant data for business intelligence further adding good value to measuring. It is a good point in this article and to assess data accordingly
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
To save this item to your list of favorite AllAnalytics content so you can find it later in your Profile page, click the "Save It" button next to the item.
If you found this interesting or useful, please use the links to the services below to share it with other readers. You will need a free account with each service to share an item via that service.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.
Elizabeth Barth-Thacker, a BI and informatics technology manager at Humana, tells us how her team is creating data transparency and building engagement with the business – with the help of an internal collaboration portal called Humanalytics.
Speaking at SAS Global Forum Executive Conference, Rajeev Kaul, SVP of pricing at OfficeMax, uses a Chinese proverb to explain one of the reasons he's deploying SAS Visual Analytics.
In an All Analytics interview, Mike Cavaretta, technical leader, predictive analytics at Ford Research & Advanced Engineering, shares how big-data is fueling vehicle decisions.
Analytics professionals and SAS executives share how organizations can get on with their work so much faster when working in a high-performance and visual analytics environment.
Analytics professionals who attended SAS's recent Executive Briefing in New York share how they think visual analytics might help their organizations get better value from data.
At Boeing, effective decision making comes down to this simple formula: QxA=E, as executive Jerry Allyne explained at the recent INFORMS analytics conference.
Whether working in major league sports, financial services, or healthcare, analytics, and data, professionals are checking out how visual analytics and high-performance technologies can help them optimize their environments, shrink their cycle times, and improve decision making, as attendees at the recent SAS Executive Briefing in New York share with us.
SAS CEO Jim Goodnight speaks with us at a recent SAS Executive Briefing about getting a feel for what's in your big-data and other new realities powered by advanced analytics.
Jim Davis, SVP and CMO at SAS, talks with us at a recent SAS Executive Briefing about how high-performance analytics and visual analytics take away the concerns over big-data and let companies get down to business with their data.
Tweet This
7 
24 
