People who want to understand the voice of the customer, brand sentiment, and other measures important to market research are often drawn to social media monitoring as a viable replacement for traditional approaches. But the methodologies associated with and the data samples collected from online and offline panels are much different. Treating them the same way would be unwise.
For one thing, traditional market research uses carefully chosen panels to find out customer sentiment about brands, issues, or unmet needs. The same usually cannot be said of online panels drawn from social media, where control is lost and much less is known about who is actually speaking.
Online surveys
The use of social media to survey Website visitors is subject to many biases almost impossible to detect or eliminate. For example, many sites use online advertising to drive traffic to their online properties. Changes in these programs impact who comes to a site and fills out the surveys. Comparing survey results across competitors may be meaningless, because the population can't be normalized, as Gary Angel, Semphonic's president, discusses in this post.
Even the Net Promoter Score, an online survey many businesses use to evaluate customer mindset toward their brands, may be subject to bias, due to the uncontrollable and unstandardized differences in how businesses drive site visits. SparkScore, the recently released social media version of the Net Promoter Score, is largely speculative and of unproven quality, because we don't know if the comments left across social media represent a valid survey sample.
Online opinion panels
Online listening systems from companies such as Radian6 and Sysomos do a decent job of collecting a large number of mentions based on a series of keyword searches and presenting them in a streamlined manner. As a result, companies can mine online opinions of many people in almost real-time to supplement or perhaps replace some forms of market research entirely.
However, getting a statistically valid sample to use for an online panel has been challenging. Companies need to be able to discount people who have insider knowledge on an issue, industry, survey subject, and so on. (Angel discusses this point in another post.)
Cost, ethics, and privacy concerns
What's more, platforms that depend on Twitter data must contend with limitations set by the social network (such as 500,000 mentions/tweets per query search via data aggregators such as DataSift and Gnip). A search costs a few cents per thousand results, but these costs can mount up when information is extracted from large data sets, as I discussed recently.
And Facebook has its own challenges with forming online panels. Most of the social network's content (85 percent) is still private and not usable for online opinion surveys or any other data mining, except for when Facebook chooses to allow it. And that circumstance is somewhat new and controversial.
Harris Interactive's FanConnect offers an alternative for Facebook online panel building. About 60,000 Facebook accounts have given Harris Interactive permission to Friend them and collect their "private" postings. From that, the company creates intelligence by attaching the information to about 100 data points it has built to understand online behavior and brand sentiment.
Ethics, privacy, and legal concerns also are factors in building online panels. They could discourage many organizations from building panels, particularly in light of Google's move to push Google+ to the forefront of searches and its new privacy policy that links up user behavior from across of its properties.
Should businesses use online panels at all for opinion monitoring? In general, I'd say we need more research to find out how online panels reflect the offline opinions of target audiences, and vice versa.
Using keyword searches is one of the worst ways to build an online panel. Behavioral targeting via InfiniGraph and other social intelligence platforms that build Facebook and Twitter panels by behavior and interest is far more effective.
What do you think? Share on the message board below.
Just some quick food for thought to add to this fascinating and lively discussion. I wonder if you're aware of the work of either Jure Leskovec, an assistant professor of computer science at Stanford University, whose research seems to have much to say about the degree to which behavior on the social Web mimics behavior in the real world, and Dmitri Williams, an associate professor at the University of Southern California Annenberg School for Communication & Journalism, whose research seems to indicate that behavior in virtual gaming environments does not differ from behavior in the real world. I would think the work of both these researchers might shed some light upon the question of the reliability of social media surveys.
Webmetricsguru writes, " the online data isn't structured like a survey would, so people could be more self expressive."
Hmm ... how are these different? Surveys usually have highly controlled response choices, to facilitate fast and simple processing. Are you saying the questions and response choices would be different, or the online surveys tend to allow open-ended responses? Of course, it would seem that the latter would be rather hard to digest and tally, especially from a really large population of respondents. So we surveyors have traditionally avoided them like something very treacherous.
Well, I don't know (yet) if there's been enough or any research on the differences of how people behave offline (in general) vs. online, esp with a panel.
Do online opinions reflect a truer opinion than offline - it depends. For one thing, the online data isn't structured like a survey would, so people could be more self expressive. On the other hand, what they means, and how to structure the information is much, much harder.
Agreed -and while there are issues with collecting online opinions, many think there is value in collecting the data, including me. The main thing I think, though is that we need a lot more researhc and standards work being done before we can sell this stuff in (and that hasn't been done yet, for the most part).
I used to moderate an online forum with anonymous users, the things people would write could be vile, racist, sexist, misogynist, cruel, and eternally damned if spoken in a confession booth. These people would never say or write these foul things about other people if their real identity was known. I think the same issue of honesty would apply for online panels with anonymous participants hiding behind their firewall, much different than in typical controlled surveys done in-person or a focus group.
In her first response to this thread, Beth asked what I thought was a very good question:
======================
I think one of the biggest challenges you bring to attention here is the need to determine how online panels reflect the offline opinions of target audiences, and vice versa. Do people react differently online then they do offline and, if so, what does that mean for the opinions they express about their attitudes toward a particular product, brand, company, political candidate, and so on? Might we come to learn that offline opinion polls provide a much truer picture of a particular target audience's (even in a one-to-one comparison) inclinations than online polling? If so, what value does social polling provide that offline polling can't -- and how do we take best advantage of each?
======================
"Do people react differently online then they do offline...?" Indeed. I didn't see that issue addressed in subsequent responses (unless I missed one — while just skimming). Based on various experiences online and offline with the same people, I tend to think there is a fairly significant difference in behavior, responses to surveys, etc. (For example, you have a lot less anonymity offline than online.)
I wonder if there's been any good, hard research on this.
@Seth -- I agree with your statement that the "model for measuring is different for each company" -- in fact I think it'd be foolish for companies to think there's a one-size-fits-all to work with! But I'm wondering if we can come to some basic rules of thumb or common best practices around all of this. What do you think?
The proof that the correlation between social media chit chat and word of mouth depends entirely upon the product is a migrane for companies to deal with.
The lesson to me would be that the model for measuring is different for each company and one can't assume that a vendor with a one-size fits all approach will benefit a comany.
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
To save this item to your list of favorite AllAnalytics content so you can find it later in your Profile page, click the "Save It" button next to the item.
If you found this interesting or useful, please use the links to the services below to share it with other readers. You will need a free account with each service to share an item via that service.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.
Elizabeth Barth-Thacker, a BI and informatics technology manager at Humana, tells us how her team is creating data transparency and building engagement with the business – with the help of an internal collaboration portal called Humanalytics.
Speaking at SAS Global Forum Executive Conference, Rajeev Kaul, SVP of pricing at OfficeMax, uses a Chinese proverb to explain one of the reasons he's deploying SAS Visual Analytics.
In an All Analytics interview, Mike Cavaretta, technical leader, predictive analytics at Ford Research & Advanced Engineering, shares how big-data is fueling vehicle decisions.
Analytics professionals and SAS executives share how organizations can get on with their work so much faster when working in a high-performance and visual analytics environment.
Analytics professionals who attended SAS's recent Executive Briefing in New York share how they think visual analytics might help their organizations get better value from data.
At Boeing, effective decision making comes down to this simple formula: QxA=E, as executive Jerry Allyne explained at the recent INFORMS analytics conference.
Whether working in major league sports, financial services, or healthcare, analytics, and data, professionals are checking out how visual analytics and high-performance technologies can help them optimize their environments, shrink their cycle times, and improve decision making, as attendees at the recent SAS Executive Briefing in New York share with us.
SAS CEO Jim Goodnight speaks with us at a recent SAS Executive Briefing about getting a feel for what's in your big-data and other new realities powered by advanced analytics.
Jim Davis, SVP and CMO at SAS, talks with us at a recent SAS Executive Briefing about how high-performance analytics and visual analytics take away the concerns over big-data and let companies get down to business with their data.
Tweet This
17 
24 
