Emotional Analytics and the Human Touch

Human beings have been collecting data on each other since time immemorial, but it's only in recent years that we've begun to truly leverage its effectiveness through analytics.

We're learning new ways all the time too, like recent breakthroughs by Realeyes that allows for tracking of a person's emotional state based on a number of metrics related to their facial expression.

It's amazing technology that opens up a lot of interesting doors for improving customer satisfaction, judging the effectiveness of a show or event, or finding a deeper understanding for kinesiology.

The technique is so cheap and easy now, we're told, that it will not only offer deeper insight, but also replace many traditional methods for gauging public interest in something, such as surveys and focus groups. People will still need to be taken aside and shown a product or service, but with emotional analytics, the suggestion is, that we don't need to ask a person what they thought, we can just track what they felt.

We're also told that the accuracy of this system for predicting long-term sales is around 75 percent, 10 percent more than traditional methods. Right now its main use is in gauging interest in online ads, with a webcam feed providing the necessary close up video to analyze, but the scope is there for much much more.

As we saw with our look at predictive policing though, there are some concerns that emotion tracking like this might be too cold. The danger with any data-driven versus human-driven analysis, is that it can skip over important factors that might otherwise be obvious.

For starters, there are the shortcomings in the technology. As it stands, it is based almost entirely on facial expressions, which are only a part of someone's emotional state. Full-body language may have its own tells and while the existing technology has done a good job in predicting sales, it's not perfect and has seen better results in predicting engagement on social media a notoriously low-effort, low-impact activity.

It is also a very skin deep analysis. While certainly there is a lot to be learned by the expressions we make in terms of predicting our mood in that moment, tracking facial expressions doesn't go very deep. It doesn't explore the reason for that emotional reaction, or what the person is thinking behind the autonomous movements of the facial muscles.

With that in mind, it would be interesting to see a study completed where tandem results of emotional analytics and traditional survey information is combined to provide a deeper insight into an individual. I wonder too though, whether having a real human with insight into human behavior could also analyze some of the footage to discover more intuitive meanings than the placement of certain facial features can in an algorithm.

Unfortunately this may not happen any time soon, as one of the big features of this automated facial analytics is the cost savings involved. Without having to hire someone to write questions, capture footage or look through it, the whole process is cheaper and easier.

There is certainly a place for it, as there is with all types of analytics, but do you think that this is too good a technology to resist using without companion studies to achieve more accurate results in a broader sense? Or will analytics firms take it for what it is: a tool in a varied arsenal?

Related posts

Jon Martindale, Technology Journalist

Jon Martindale is a technology journalist and hardware reviewer, having been covering new developments in the field for most of his professional career. In that time he's tested the latest and greatest releases from the big hardware companies of the world, as well as writing about new software releases, industry movements,and Internet activism.

Brexit Negotiations Drive Analytics Growth

Could Britain's exit from the EU drive a new wave of analytics investment and growth? Here's a closer look.

Vocal Commands Arrive for Analytics

Voice interfaces may give many more users access to self-service analytics. Here's a closer look.

Re: There is a place
  • 2/19/2016 1:21:58 PM

For a lot of things, I agree that the current state is not good enough. If you go by an ad and wince, I think it is pretty reliable that you didn't like it. If you smile, maybe you did. The issue is really what is a third party allowed to do with your exoressions?

Re: There is a place
  • 2/19/2016 11:26:16 AM

All the research and practical applications of the emotional analytics seems very interesting, but I'm ot entirely convinced that the predictions hoped for will soon be useful in any broad scale use. As noted,  "facial expressions, which are only a part of someone's emotional state." and what other factors would have to be measured is still not fully understood.

Re: There is a place
  • 2/6/2016 10:43:30 AM

Ha! Of course, jmyerson2! You make an excellent point, and remind us why we should always watch our assumptions and not presume to know what's going on for someone else.

Re: There is a place
  • 2/6/2016 8:43:48 AM

Terry, you mentioned that if a customer has tears streaming down his face, should we assume he's sad or just very moved? I would like to add that sometimes customer has allergies that cause tears to stream down his face.

Re: There is a place
  • 2/5/2016 4:31:47 PM

@Terry. I agree that looking at a single emotional signal (such as tears) might be inconclusive. I know that some of the services offered to marketers (one that comes to mind gets a lot of attention around this time because they test potential Super Bowl ads) rely on multiple factors and tend to get pretty good results. Those also seem to be higher end systems in terms of complexity and cost than what we're talking about in this blog.

Re: There is a place
  • 2/5/2016 11:56:59 AM

I agree there's a place for emotional analytics, tomsg... it just strikes me that that the assignment of emotion can be very subjective -- lots of gray areas here. For instance, if a customer has tears streaming down his face, should we assume he's sad or just very moved? Similarly, social psychologists frequently remind us that fear and excitement look nearly identical from a physiological perspective.

For marketing applications, maybe what we're really talking about here is a binary that's as simple interested/not interested. That way, there's no need to try to parse whatever emotions are going on concurrently.

There is a place
  • 2/5/2016 10:30:36 AM

I agree that there really is a place for this type of analysis. I wonder about issues like opt in etc? Do you own your own emotions and do you have to give permission for them to be shared? Since it is primarily based on facial expression have you already put it in the public domain? I can see a lot of issues to figure out with an implementation.