People who want to understand the voice of the customer, brand sentiment, and other measures important to market research are often drawn to social media monitoring as a viable replacement for traditional approaches. But the methodologies associated with and the data samples collected from online and offline panels are much different. Treating them the same way would be unwise.
For one thing, traditional market research uses carefully chosen panels to find out customer sentiment about brands, issues, or unmet needs. The same usually cannot be said of online panels drawn from social media, where control is lost and much less is known about who is actually speaking.
The use of social media to survey Website visitors is subject to many biases almost impossible to detect or eliminate. For example, many sites use online advertising to drive traffic to their online properties. Changes in these programs impact who comes to a site and fills out the surveys. Comparing survey results across competitors may be meaningless, because the population can't be normalized, as Gary Angel, Semphonic's president, discusses in this post.
Even the Net Promoter Score, an online survey many businesses use to evaluate customer mindset toward their brands, may be subject to bias, due to the uncontrollable and unstandardized differences in how businesses drive site visits. SparkScore, the recently released social media version of the Net Promoter Score, is largely speculative and of unproven quality, because we don't know if the comments left across social media represent a valid survey sample.
Online opinion panels
Online listening systems from companies such as Radian6 and Sysomos do a decent job of collecting a large number of mentions based on a series of keyword searches and presenting them in a streamlined manner. As a result, companies can mine online opinions of many people in almost real-time to supplement or perhaps replace some forms of market research entirely.
However, getting a statistically valid sample to use for an online panel has been challenging. Companies need to be able to discount people who have insider knowledge on an issue, industry, survey subject, and so on. (Angel discusses this point in another post.)
Cost, ethics, and privacy concerns
What's more, platforms that depend on Twitter data must contend with limitations set by the social network (such as 500,000 mentions/tweets per query search via data aggregators such as DataSift and Gnip). A search costs a few cents per thousand results, but these costs can mount up when information is extracted from large data sets, as I discussed recently.
And Facebook has its own challenges with forming online panels. Most of the social network's content (85 percent) is still private and not usable for online opinion surveys or any other data mining, except for when Facebook chooses to allow it. And that circumstance is somewhat new and controversial.
Harris Interactive's FanConnect offers an alternative for Facebook online panel building. About 60,000 Facebook accounts have given Harris Interactive permission to Friend them and collect their "private" postings. From that, the company creates intelligence by attaching the information to about 100 data points it has built to understand online behavior and brand sentiment.
Should businesses use online panels at all for opinion monitoring? In general, I'd say we need more research to find out how online panels reflect the offline opinions of target audiences, and vice versa.
Using keyword searches is one of the worst ways to build an online panel. Behavioral targeting via InfiniGraph and other social intelligence platforms that build Facebook and Twitter panels by behavior and interest is far more effective.
What do you think? Share on the message board below.