The Beauty of Sentiment Analysis

What if you could be psychic?

Well, OK, if you're like me, you'd probably buy a winning lottery ticket or two, quit working for a living, and retire immediately.

Linguistic sentiment analysis won't make you psychic, but as this field develops, it promises to provide near-psychic predictive abilities.

Linguistic sentiment analysis studies the words used in various types of content and looks for indicators of certain sentiment, opinions, or emotions. It tests how the content fits into multiple classes (such as "positive" vs. "negative"), a polar range (such as how much someone likes a movie on a scale of 1 to 5), or a range of opinion strength. From there, the content can be classified as expressing a certain sentiment, being about a certain topic, coming from a certain kind of document, or even being written by a person from a certain demographic group.

Consequently, there's a lot more these days to social media analytics than measuring "Likes." Companies can derive business intelligence from the sentiment in social networking content. In this way, they can get perhaps the most effective BI of all, getting information directly from the horses' mouths -- the general populace.

Most of the social deployment of sentiment analysis has been limited to searching for positive and negative sentiment about particular brands and features. This is a bit like owning a Porsche when the only driving you do is to the 7-Eleven two blocks away.

There is an entire world of information that sentiment analysis can harness -- through subtle cues beyond mere positivity and negativity.

Research has shown automated sentiment analysis can tell us a lot from as little as a single Tweet or other social network post, with human-like or better-than-human accuracy. Here are some examples:

  • Basic facts about the author's identity, such as gender, political affiliation, and geographic region.
  • How the author uses social media, and what "type" of social media user the author is.
  • Whether the author is feeling calm, anxious, worried, fearful, happy, alert, sure, or numerous other emotions.

More excitingly, as researchers are able to determine correlations between sentiment expressions and particular events, linguistic sentiment analysis can predict the future. For instance, researchers have found that the sentiment in Tweets could be analyzed to predict such things as film box-office receipts and election outcomes. Additionally, linguistic sentiment analysis has repeatedly been shown to be able to predict stock market performance. One investment firm has even launched a hedge fund that bases investment decisions on linguistic sentiment analysis of Tweets. Its algorithm showed 87.6 percent accuracy in a study.

It would hardly be unreasonable to suggest that sentiment analysis of social media content could be used one day to predict such things as what toy will sell best in a particular Christmas season and which political party will be in control after an election.

As Community Editor Shawn Hessinger recently commented, "The kind of analytics we are discussing here goes beyond the garden variety and could perhaps be seen as a macro view of the social media ecology with very unique insights far beyond the reach of predicting, say, most people's favorite antacid."

Scientific developments in the past few years have only started to reveal the predictive potential of these insights. When it comes to the full breadth, depth, and value of the BI that linguistic sentiment analysis of social content has to offer, we are only limited by our imagination and our ingenuity.

Point / Counterpoint, Attorney & Marketer

Joe Stanganelli is founder and principal of Beacon Hill Law, a Boston-based general practice law firm.  His expertise on legal topics has been sought for several major publications, including U.S. News and World Report and Personal Real Estate Investor Magazine. 

Joe is also a communications consultant.  He has been working with social media for many years -- even in the days of local BBSs (one of which he served as Co-System Operator for), well before the term "social media" was invented.

From 2003 to 2005, Joe ran Grandpa George Productions, a New England entertainment and media production company. He has also worked as a professional actor, director, and producer.  Additionally, Joe is a produced playwright.

When he's not lawyering, marketing, or social-media-ing, Joe writes scripts, songs, and stories.

He also finds time to lose at bridge a couple of times a month.

Follow Joe on Twitter: @JoeStanganelli

Also, check out his blog .

Counterpoint: Train for Data Visualization Skills

Chances are you've already got good data visualization experts on staff, even if you don't know it yet.

Customer-Centric Banking Analytics Scares Me

Banks have tons of customer data at their disposal; unfortunately not all will use it scrupulously.

Hope everyone will join us for our live chat tomorrow...
  • 8/17/2011 11:43:19 PM

Thanks everybody for the great conversation on this topic. Hope everyone will join us for our live chat on social media analytics tomorrow Thursday Aug. 18 at 2 p.m. ET including a more detailed discussion with Joe and Pierre. See you then!

Re: Can we really read minds?
  • 8/11/2011 9:21:25 PM


I agree with what you're saying in theory, Joe, and all I would come back with is that the handful of samples I've seen are dealing with much simpler sentiments, calmness in the case of the hedge fund we talked about for example in our earlier conversation. It seems to me that when we get predictive about differentiation (which brand I prefer, what movie I will see, who I will elect president) we are getting into a much more complex area. I think I'll need to be sold on the idea that such preferences can be predicted by analyzing the sentiment of a 140 character tweet.

Re: Can we really read minds?
  • 8/11/2011 4:06:58 PM

Hi, Beth and Shawn.

The "near-psychic" element I'm talking about here isn't so much about reading minds (as Shawn puts it) as it is about predicting the future.

One example of a practical use (off the top of my head): Let's say you know (i.e., through linguistic sentiment analysis) which brand is probably going to do best in a particularly busy upcoming shopping season (say, the weekend after Thanksgiving), or which movie is probably going to perform best on an upcoming opening weekend.  That can tell you what people are interested in and where you should focus both your marketing and PR efforts as well as your development efforts (to stay competitive).

If you know, within a reasonable degree of likelihood, that a particular event is going to happen... well, there's almost always SOME way to make money off of it.  With enough ingenuity, it's not hard to capitalize on that information.

Re: Can we really read minds?
  • 8/11/2011 1:31:48 PM

Shawn, I'm with you on this one. I love the idea, but not convinced of the practical use of sentiment analysis yet -- good for the university research crowd today but not much beyond that for  a bit. 

Can we really read minds?
  • 8/11/2011 1:13:29 PM

I just don't know what to make of this, Joe, though I know you and I have e-mailed back and forth a bit on the topic. As a guy with social media experience and the person primarily responsible for building community here at, I'm a big believer in the power of social media. I've seen it work in communities I have helped build. On the other hand, knowing as I do the number of people out there with multiple accounts (not to mention ulterior motives for the things they post on those accounts), it's a bit hard to believe that what is being posted can be taken at what seems to be face value. If the idea here is to read the mood regardless of the intentions of the people doing the communicating (like a lie detector reads indicators the subject may not be able to control) then I suppose there may be merit to the technique. But even so, I question, as does Pierre, whether our current level of sophistication really permits it.