Voice Analytics: You Sound Emotional

Imagine you ask Siri to find a restaurant nearby. She replies, "You sound kind of down, so I'd recommend a warm and quiet spot a few blocks from here."

How did Siri know how you were feeling? Welcome to the world of intonation analytics.

Humans read emotion in verbal communication all the time: I don't like your tone! -- An edge crept into his voice. -- She spoke soothingly. But the science of sentiment analysis is still relatively new and focused primarily on text.

As explained by data scientist Brian Kolo on the Opera Solutions blog, algorithms have a hard enough time differentiating "that's hot" (as in, good) from "that's hot!" (as in, you'll burn yourself). It seems a stretch to suggest they can understand how we feel based on our intonations.

Or does it? At yesterday's Sentiment Analysis Symposium in New York, executives from Beyond Verbal demonstrated software that seems to pick up on these cues, too.

"It's not what you say, but how you say it," explained Dan Emodi, vice president of marketing at Beyond Verbal. "Vocal intonations convey your mood, attitude, even personality."

The research began 18 years ago, when a physicist and neuropsychologist began exploring the way that babies understand language even before they understand words, Emodi said. The team discovered that intonation transcends language and culture.

"Patterns of happiness, sadness, aggression, introversion, extroversion, apathy, and action-orientation, they're all the same across languages," he said.

On Beyond Verbal's site, vocal analytics are applied to the speech of President Barack Obama, Steve Jobs, and Princess Diana. The company also analyzed Edward Snowden's June interview with The Guardian, finding that his voice betrayed some egocentricity and pride in orchestrating his NSA document leak.

You can listen to me, but do you hear me?
Like most sentiment analysis projects, Beyond Verbal started with a set of samples -- 16,000 audio clips, in this case -- and trained its algorithms using crowdsourced human evaluators. Temper and anger were easiest to detect accurately, Emodi said. "Call centers have plenty of aggressive tones." Over the years, the training sample grew to 32,000 clips.

The potential applications are limited only by imagination, he added. Contact centers, market research firms, dating sites, search services, and many other businesses could benefit from this kind of technology.

I asked about law enforcement, and Emodi said the software has caught the attention of Homeland Security officials. Another audience member asked about healthcare applications, where the software could detect confusion in patients suffering from early stages of dementia, for example.

"We've been contacted by healthcare companies," Emodi said. "When we started this, we never even thought about healthcare."

Beyond Verbal's analytics happen in the cloud, so individuals can try it on the website or via the Moodies iOS app.

I gave it a shot this morning, and as you can see in the screenshot, it seems to have me pegged. "It helps us get a better understanding of our own selves," Emodi said.

What do you think, members? Do you see intonation analytics changing the way we interact with machines? What other applications might benefit from intonation analytics? Make your voice heard (but watch your tone) below.

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Related posts:

Michael Steinhart, Contributing Editor

Michael Steinhart has been covering IT and business computing for 15 years, tracking the rising popularity of virtualization, unified fabric, high-performance computing, and cloud infrastructures. He is editor of The Enterprise Cloud Site, which won the Least Imaginative Site Name award in 2012, and he managed TheITPro.com, a community of IT professionals taking their first steps into cloud computing. From 2006 to 2012, Steinhart worked as an executive editor at Ziff Davis Enterprise, writing and managing research reports, whitepapers, case studies, magazine features, e-newsletters, blog posts, online videos, and podcasts. He also moderated and presented in dozens of webinars and virtual tradeshows. He got his start in IT journalism at CMP Media back in 1998, then moved to PC Magazine, managing the popular Solutions section and then covering business technology and consumer software. He holds a Bachelor of Arts degree in communications/journalism from Ramapo College of New Jersey.

Biosensors, Robots Replacing Human Guards

In prisons and other security settings, electronic devices are predicting and detecting anomalous behavior with high accuracy and reducing the need for human personnel.

Questioning the Ethics of Learning Analytics

Schools around the country are weighing the benefits and ethical questions of learning analytics.

Re: Pretty accurate!
  • 3/22/2014 11:04:21 AM

As we're separating words from intonation, this technology might be great for travelers, who'll be able to gauge the emotions of people who speak another language. At the very least, it might help a tourist avoid getting ripped off.

Would do wonders for a marriage (or not)
  • 3/22/2014 10:57:36 AM

With this technology, millions of husbands around the world will finally be able to understand their wives. No longer shall we have insensitive men who can't hear what women are not saying. Simply slap this on a wearable and it'll sell like hotcakes, though remember to advertise in wedding fairs, just to be sure.

But it's a double-edged sword. It could totally backfire on the insincere, especially if wives also have this technology.

"I love you." [LYING]

"I'm sorry." [EGO]

"I feel sick; I can't take out the garbage." [LAZY]

Re: Pretty accurate!
  • 3/14/2014 2:08:45 PM

That's a rather substantial either/or, Michael. So much so that doesn't it more or less negate any value of voice analytics? 

Re: Pretty accurate!
  • 3/13/2014 8:43:09 PM

Thanks for the information Michael, I would tend to think there are just too many factors which could affect the results, I mean people fake "happiness" for instance - all the time.  : )

Re: Pretty accurate!
  • 3/13/2014 6:14:51 PM

Just to follow up -- the friend repeated the test and got 'stress' and 'vengeance' again. Then, on a third try, it came up with self-esteem, leadership, and charisma. All of which are true. So maybe it's not as fine-tuned as it seems?

Re: Pretty accurate!
  • 3/13/2014 6:09:36 PM

I agree, and I also think it's an interesting exercise regardless of how accurate it is. I tried it on an upbeat, cheerful friend, and it detected stress and resentment as primary emotions. That sort of thing can either be an uncomfortable revelation, or an off-base assumption. We were in a noisy environment, so it's possible the algorithm had a hard time detecting intonation.

Re: Emotions and meanings
  • 3/13/2014 3:06:51 PM

Michael while those are key areas, the emotional state can determine the meaning of a word or phrase so it could impact the overall interpretation of what is said at any time. Whenever analysis is performed it is worth considering.

Re: Pretty accurate!
  • 3/12/2014 8:59:17 AM

Beyond Verbal certainly thinks it can be!

Re: Pretty accurate!
  • 3/11/2014 6:26:44 PM

@Lyndon_Henry   Good point reqarding Lie Detectors - which are loosely based on this notion of measuring responses. I don't think anyone would consider them "fool-proof" and intonation analytics probably faces many of the same challenges.

Re: Pretty accurate!
  • 3/11/2014 6:19:51 PM

I do find verbal intonation intriquing, but I wonder if analytics can really be applied to this ?

Page 1 / 4   >   >>