I absolutely agree that this is not a broadly perfected application of the technology. It does have presently useful, valuable application, however, and the usefulness and value of this kind of application will only increase as we use it and play with it more.
Taking a cue from the study on Twitter predicting box office receipts (link: http://arxiv.org/pdf/1003.5699v1), you could do something similar to look to Tweet volume and sentiment pre-release and post-release. The more buzz and the more positive -- especially after release (once people have been able to see, buy, and try), the better that product -- in our case, the toy -- will sell.
Customer surveys are rife with problems. First, you have to convince people to take them. Then, as John Barnes points out in a great piece from earlier this week, you have to try to make sure they're answering them honestly. And even before all that, as I point out with an upcoming piece for A2 that should go up in the near future, you have to make sure that the survey is constructed so as to give you accurate and useful information. Many companies simply don't know how to do that.
Additionally, it's not being leveraged to its full predictive ability. Instead, consultants are telling executives like, "Oh, people like this!" "Oh, people DON'T like this." But you can actually use this data on a much more sophisticated level to predict future events AND make decisions based upon those predictions. If you know what toy is likely to sell best this holiday season, you can make intelligent decisions as to how to market your own competing toy, when to release it, and even what kind of competing toys to develop.
To your first point about Tweets being unrelated to stock tips, understood. But theoretically you could game sentiment, stir dissent, create alarm, stimulate lack of calm...or the reverse, couldn't you?
@Shawn: Although there was another study that specifically did look at stock-related Tweets. That study, too, found that there was the ability to predict -- in that case, based upon sentiment and volume.
@Shawn: That's irrelevant to what Derwent does. They don't look at stock-specific Tweets. They look at everyday Tweets by all sorts of people about all sorts of subject. It's simply a matter of looking at general global sentiment.
@Pierre: We can howl at the wind all day about how it shouldn't work, how stupid it is, and how there are all sorts of problems with it -- but time after time it's been proven by people who are way smarter than us. That's not to say we shouldn't still keep doing what we can to look for hammer out the kinks, but there really is something here that can change the world.
Derwent analyzes a million Tweets each day (which is something like 10% of all Tweets each day, I think), calculates a sentiment score for each, and plugs it into their algorithm -- analyzing for calmness.
@Joe I hear you, but it's the idea of making a impact that has spread. The concern is that while a good idea, there needs to be better ways to eliminate gaming that distorts. The measurement tools have not come along deep enough for the average business to discern well. It's gerat that the dialogue has gotten started, but measurment capability needs more development.
Well, there's the example we've talked a lot about, Derwent Capital Markets. It's a hedge fund that is based upon a study that found that Tweets could be successfully used to predict the Dow -- even as much as six days in advance.
@Pierre: There's a difference between buying a follow or Like, and buying advocacy. The follower-buying services are baloney because all they give you is a number -- nothing else. (In fact, social media consultant Elliott Volkman pointed out on Social Media Today (sorry, I don't have the link) how using follower-buying services can have a negative ROI.)
@Shawn: For one thing, the technology is pretty good at weeding out the fakes. There may be some concern about people posting from multiple accounts, but even if those don't all get weeded out, they're a drop in the bucket.
@Shawn: That's not an uncommon skepticism. People are concerned that some may try to skew results. At the same time, in the aggregate, considering the sheer volume of the millions and millions of Tweets, status updates, and what-have-you going up each day that are ripe for mining and analysis, any false indicators would likely be silenced.
@Shawn: I think its use as a communications tool is precisely what makes it so valuable as a source of data. It gets right into the hearts and minds of the public -- what they are saying and thinking when they're not interacting with you. (The Onion tongue-in-cheek reported on what would happen if Google could listen to what you were saying on the phone.)
Great question, Shawn. One is to discover what services/products are drawing customer interest. Another can be discovering content that a customer is searching for. A third can be determining which marketing is really eefctively drawing a good return on investment, be it through a brand lift or an actual sale.
I think it's organizational more than anything. There are companies that sense that their website is important, but haven't realized that a consistent periodic analysis can lead to new ideas, improved customer service, and a way to weather the economic storm we have now. It's not strategic to them, even though their data and analysis can be the ace in the deck that leads to innovation or new ways to market and brand a business.
Beth, great question -- it's still separate in many cases. There's been studies from Experian and eMarketer that suggests that online behavior and offline behavior from consumers (and clients) are linked, that what you offer online affects whether or not you gain a sale. Yet many businesses treat their web data as a seperate, even secondary consideration, when in fact it an be the canary in the coal mine that can warn you of an unmet customer need or sales influence.
Pierre, I'm curious -- from what you've seen out there, how successful are companies in integrating the data they're collecting from the web into some sort of master data repository? is this common, or do companies tend to keep their online data separate from data they collect from other sources?
I also like the point Cordell makesabout finding unexpected results or reactions. We might call this the "squeaky wheel" effect. But if not measured accurately this could cause knee jerk reactions to problems that might not be that serious being amplified by social media.
Beth I agree. And just because it's not reliably representative doesn't mean it's not valueable. Rather than looking for something that points to a broad sentiment maybe it's more useful for spotting things like oversights or gauging reactions you didn't expect. Outliers so to speak.
True, it can lead to a replacement, but again, a company must be careful. When Honda's fanbase express their dislike for the Crosstour, the moderates tried unsuccessfully to focus the group on the vehicle's benefits - the way they did it made the fans feel as if they were not being heard...
Sticking with that butter pecan example for a sec -- even if the company decides it's not financially in its best interest to relaunch the flavor, it at least can think about how to make those irked customers feel better about it -- sending a coupon for a new flavor, for example (and of course measuring the success of said campaign and modifying it as needed)
Well, the idea of predicting the strict likelihood of someone liking or not liking a site or linking to someone or not is a lot more about behavior (and measurable behavior) then trying to figure out how the user "feels."
Beth, that can be a great point about vocal minority. A lot of people may want Butter pecan, but if the margins are not profitable to produce it, it would be that a large number of people would economically justify its sale. A minority may feel it, but not buy in enough quantity.
Don't you have a selection bias problem as well? In the ice cream example, just because a bunch of people on FB launched a campaign to bring back Butter Pecan doesn't mean it's a widely held view. Bringing it back may not be in thier best interest.
The thing about comments is that you'll end up with a number of comments, but you need to sort the messages being given. For car, no one eer said give me a cupholder -- there had been comments of needing more room, I'm in my car all day, but no one ever said my car needs a cupholder. A cupholder is what the auto industry delivered after sorting through the comments...
Pierre- to your point about Honda. That's thing, right -- if you're unhappy or disgruntled, you're more likely to voice your dismay. That's not necessarily the case if you're happy. Your voice becomes less important, so to speak, in that latter case.
The social media universe just seems so arbitrary to me, for the most part! But even so, I must admit to being intrigued by what researchers are trying to do with all this data amassing. Good to look in on rather than buy into at this point, though.
Beth, It is. Many companies have run into it -- I recall Honda being noted for how FB commented on the Accord Crosstour were handle. It is something to be noted, but the concern comes back to context and what can be done. In the Honda example, the vehicle was already designed and being released, yet long-time Honda fans objected to it. Honda still sold the vehicle. It's in production, and despite Honda's poor use of FB, they still had sales. Sentiment may ahve been captured, but it did not lead to a significantly changed result.
Credibility can be a problem if people do not see authentic interaction. It become harder to attribute the context of a comment -- did it come from someone who had something to say or gaming the system?
Pierre, hmmm. I'm thinking of something along the lines of noting that your FB fans -- customers -- are angry over product quality or some such. Recently I wrote abOUT OBERWEIS Dairy, which has an active FB community of users. What if a large portion express anger over the company's decision to discontinue Butter Pecan at the local ice cream shops. Isn't that type of sentiment worth noting and making actionable?
Beth, I think the viability becomes a bit worse on a FB page -- to be analytic means to be actionable on an intent. FB is popular, but it is harder to determine what to do once you've determine a score for it. This is a main probvlem with FB Insights -- you have trending data, but it's not leading to a specific goal like you would see in a GA or Site Catalyst.
Shawn, yes, the claims can be misleading, though I also think the core is that we have to change how we consider quantifying sentiment. An all encompassing score is somewhat helpful but is a byproduct of being "number one" at something, when in fact the idea should be more nuanced.
I'm with you there. Of all the things to focus in on, and delegate dollars to, sentiment analysis would be love on priority list if I were a marketer. Too arbitrary in my mind -- at least when we think in terms of the general Twitter population. Does the viability change if you're dealing with a particular domain, though, such as your Facebook page for customers?
And measurement is still linked to very basic web architecture that is changing due to how we engage the web. It just makes measurement a tricky process that can be misinterpreted or over-emphasized without context.