What is emotion analytics and why is it important? (Interview with Seth Grimes)

by Seth Grimes , Alta Plana Corporation

Link to article

Interactions, facts and feelings shape our relationships. A truism: It's not what you say, but how you say it. Expressions matter, as do the sentiment behind each encounter and the emotions raised. Emotion is entwined with the literal meaning of words used.

This fact/feeling principle applies to both inter-personal and business relationships. "Emotional and factual appeals cannot be easily separated," writes Nigel Hollis of Kantar Millward Brown in an analysis of advertising approaches. "[A] distinction between emotional and rational is one that exists only in the minds of marketers, not consumers," according to Hollis.

The fact/feeling equation is central to corporate customer experience (CX) initiatives. CX practitioners map customer journeys that are defined by both the what and the how-did-it-make-you-feel? of customer-brand interactions. "Emotion drives loyalty," according to CX visionary Bruce Temkin, and loyalty drives profit.

Another truism: You can't improve what you don't measure, not systematically, on a corporate scale.

Sentiment analysis and the varieties of emotion AI

Enter sentiment analysis, software technology that quantifies mood, attitude, opinion, and emotion in digital media, in images, video, audio, and text. One subspecies infers emotion via facial-expression analysis. Providers include AffectivaCrowdEmotionEyerisKairosnVisoNoldus Information TechnologyRealEyes, and Sightcorp. Another variety analyses emotion in speech. Check out audEERINGBeyond VerbalEMOspeechGood VibrationsNICEVerint, and Vokaturi. On the text front, natural language processing (NLP) techniques can identify and extract emotion in online, social, and enterprise sources, delivered by companies that include ClarabridgeCrimson Hexagon, Feedback Ferret, IBM (AlchemyAPI and Watson Tone Analyzer), indicoReceptiviti, and an advisee of mine, Heartbeat AI Technologies

This article aims to get a handle on the state of emotion analytics — specifically, emotion in text — via an interview with Heartbeat founder Lana Novikova. Lana describes herself as a marketer by training and a market researcher by trade, never satisfied with numbers and observations, always pushing to understand the "deep why" behind consumer (and her own) decisions. She'll be presenting at LT-Accelerate, a conference I organise, November 21-22 in Brussels, alongside Odile Jagsch, a consultant at global market research consultancy Kantar TNS, topic "The 'Why' Behind Customer Loyalty."

Seth Grimes. Heartbeat designs "emotionally intelligent technologies." OK, what's an "emotionally intelligent technology"?

Lana Novikova. Let's start with the concept of Emotional Intelligence (EQ), popularised by a psychologist Daniel Goleman in 1990s.

Imagine a newborn human who comes with a basic wiring for recognising and expressing key emotions, and with an enormous capacity to learn. She can cry or stay calm, smell and turn her head towards her mother’s breast. Once she can see faces, her mirror neutrons start learning and mimicking facial expressions; then she develops more and more capacity to read and express emotions — from touch, to tone & voice recognition, to basic language to more complex if-then scenarios. In a perfect world, she grows into a secure and happy person who can recognize and name a wide range of her own emotions, understands what other people feel from multiple expressions, and has a capacity to express and manage her emotions.

SG. So you apply the EQ concept to and via technology.

One day, this technology might surpass humans in understanding human emotions because it will tap into data that humans can not perceive on their own.

LN. Technology today has a super high IQ — it can beat the best human chess, Jeopardy, and Go players — yet it has a very low EQ. At Heartbeat, we want to be a part of an academic and business community that changes this. Emotionally intelligent technology is never going to feel emotions or express them like our baby can, but it will eventually become very good at perceiving and understanding human emotions from data.

One day, this technology might surpass humans in understanding human emotions because it will tap into data that humans can not perceive on their own: biometrics, brain waves, subtle cues from body language and facial expressions, and more.

SG. Relating the tech to Heartbeat...

LN. We are focusing on training the (metaphorical) technology infant to recognize explicit feelings from language, from text, and to guess the range of emotions it communicates. Just as some people can intuitively differentiate between many emotions, our growing algorithm can tell what kind of Joy or Anger is expressed in language. There could be as little as 2-3% and as much as 50% affect words and phrases in any given unstructured text. We find these words and assign them to a cluster of emotions. This process mimics how our brain deciphers emotions from language.

SG. What aspects of emotion does Heartbeat detect and measure? Do you adopt a particular emotion model?

LN. There are a few models and classifications of emotions developed by brilliant psychologists like Paul Ekman and Robert Plutchik, and even by Human-Machine Interaction Network on Emotion (HUMANE). I was inspired by a more intuitive model of W. G. Parrott (2001), originally described by Shaver in 1987. It has a tree structure and includes Primary, Secondary and Tertiary emotions. I also did a lot of reading about effective neuroscience, and tried to combine Parrott's model with what I took from the work of J. LeDoux, R. Davidson, and J. Panksepp. Then my "practical life-long quant researcher” side took over and asked, “How is this segmentation going to be useful to a brand of chocolate, or a bank, or a political party?"

The art of analysing good quality text data lies in understanding (a) how to ask a good question, and (b) how to infer meaning from people’s answers.

We ended with a 2-level clustering of 99 complex emotions and feelings into nine primary emotions: Joy, Love, Trust, Anger, Fear, Disgust, Sadness, Surprise and Void (which is explicit lack of emotion like in “I don’t care"). We also added Body Sense (positive, negative and neutral) as a way to analyse words and phrases that don’t point to a particular emotion, yet are useful for understanding human perception overall, especially for marketing food and body care products.

Many words and phrases are coded into multiple emotion clusters. Would you agree that there is a large overlap between Disgust and Anger, or Love and Joy, or even Anger and Fear? For example, we put the word "terrific" into both Joy and Fear, and let context decide which emotion it is more likely to represent. This is the most challenging part of our journey — understanding how different context colors "terrific" into happy or unhappy expression. It's the domain of Machine Learning that needs lots of training data. We are just scratching the service here, but this is also the most exciting part of my job.

SG. How does the tech work?

LN. Today, our tech is very simple yet very accurate. It’s called "bag of words": 8,000+ words and phrases (including negation, metaphor, and other multigrams) professionally coded into categories and validated by skilled psychologists and psycholinguists. Our software consumes unstructured text from survey responses and social media, and produces a set of visualisations and charts in a simple elegant dashboard.

We do our best work when we analyse data from survey responses that focus on people’s feelings. Data like that produces over 30% affect words, and has the context controlled by researcher. Once the report is ready (which is almost instantly), we curate results by removing some words that do not apply to the report. This is how we deal with another industry challenge, ambiguity. Finally, to prove that we are very good at what we do - we show all words and phrases for each Primary emotion. You can click on any word and see exactly how it appeared in the original text - no “black boxes” here. Since our taxonomy reached 5,000, our match rate — the percentage of affect words that we recognise — is over 95%. Heartbeat is committed to accuracy, depth, and transparency.

SG. How is Heartbeat different or better than the competition?

LN. Heartbeat is different because it was created by a market researcher (myself) who spent hundreds of hours coding open ended survey data. My team built our award winning app for researchers and marketers who appreciate the depth of consumer insight. I love working with good quality data, and would choose quality over quantity any time.

Survey data is under-used and often abused. The art of analysing good quality text data lies in understanding (a) how to ask a good question, and (b) how to infer meaning from people’s answers. I believe Heartbeat is better for distilling emotions from open-ended survey questions than any other company on the market today including IBM/Alchemy and other powerful APIs. They can do a lot of advanced text analytics with huge amounts of data. We made it simple, transparent, and fun. Just check out our dashboards — clients love it! Another big differentiator is that we are 100% focused on emotions — not sentiment or basic emotions, but fine-grained feelings. Our reports can be useful for anyone - from a CEO and CMO to a brand manager to a CX analyst to an agency creative director.

SG. How do Heartbeat emotion text analytics findings complement or compare with insights discovered via neuroscience and biometrics, via facial image recognition and brain-activity measurement?

LN. I strongly believe in cooperation over competition. I think putting our best technologies together will create a better future for our businesses and for this planet. A graphic will communicate how I see the future of emotion AI integration. (See the image below.)

How Heartbeat AI sees the progress of emotion analytics.

How Heartbeat AI sees the progress of emotion analytics.

SG. Could you please sketch out 2 or 3 use cases? What's your target market — clients and applications?

LN. Let’s start with the customer, the consumer or shopper. What do all customers have in common? They're human, they feel emotions all the time, and those emotions drive many (if not most) of their decisions.

Why not measure emotions on every step of your customer journey? It's not easy to put an eye tracking and EEG device on thousands of people, but it's easy to ask one simple question, "Please share a few words that best describe how you feel about X." This natural human question fits nicely into any survey and customer feedback tool, and it’s surprisingly powerful. It not only invites a wide range of un-aided and unbiased feelings on the subject, but it can also aid at predicting what people will do in the future.

We're a start-up, but we have already published two case studies to show the predictive power of emotions measured with HEARTBEAT emotion analytics: banking and political elections. The best application of HEARTBEAT is in on-going customer experience measurement and foresight.

SG. What's on your product roadmap?

LN. We are driving in the fast lane! We launched our first prototype last December and won an international startup competition (Insight Innovation Competition by GreenBook) in March 2016 in Amsterdam. All Spring and Summer, we tested our tool with some of the best research companies in the world. Finally, we launched enterprise SaaS this fall, which will enable anyone — brands, market research suppliers, consultancies, marketing agencies — to access the engine. No more manual coding: leave it to a machine, fast and accurate.

Next, we're venturing into a long complex journey of NLP and machine learning, to crack the challenge of context and meaning.

Here's a quote that resonates with me, especially when it comes to trying to solve one of the biggest puzzles in the Universe, the puzzle of human consciousness: "The most influential thinkers in our own era live at the nexus of the cognitive sciences, evolutionary psychology, and information technology." That's New York Times columnist David Brooks.

The mission of Heartbeat Ai is to design emotionally intelligent technologies and tools to help machines understand peoples' feelings and improve our emotional wellbeing. I don’t know exactly how we'll get there in the end, but I know that we are on the right track today.

SG. Thanks, Lana!

Readers, I've written up a couple of other emotion analytics interviews — an IBM Watson blog contribution, Sentiment, Emotion, Attitude, and Personality, via Natural Language Processing, based on a conversation with IBMer Rama Akkiraju and On Facial Coding, Emotion Analytics, and Emotion Aware Applications with Affectiva principal scientist Daniel McDuff — and of course you can learn more about Lana's Heartbeat work atLT-Accelerate in November. If you can swing a trip to Brussels, see you there!

(This article was originally published at www.mycustomer.com on 12 October, 2016)

What happens when you ask 10,000 people: "How do you feel about ....?"

It has been over a year since I started asking one simple open-ended survey question:"Please share a few words that best describe how you feel about ..." The topics were wildly different: "eating chocolate" vs. "craving chocolate", "shopping at your favourite store", "doing banking" vs. "being responsible for my family banking", "the future of the planet", Millennial's feelings about their career, and using Facebook, brand Uber, having a period, Donald Trump, Bernie Sanders, competition vs. collaboration, and many more. Here are the three things I learned.

First. I was surprised that the overwhelming majority of people choose to answer my open-ended questions. They don't skip them, don't write the typical "dk", or angry "get lost." They are very open and honest. Some responders share a couple of words, some write a sentence, a few write a whole paragraph. For a market research analyst turned psychologist, this text data is incredibly rich and multi-coloured. It also gives me exactly what I am looking for - clean text data within the context that I can control.

So what? I encourage market researchers to stop asking "Why" questions and start asking "How do you feel" questions if you really want to understand your customer in a time- and cost-effective way.

Then, I started wondering if people ACTUALLY LIKE ANSWERING my questions. Despite the common belief that panel responders are sick and tired of surveys, I felt I found one question that people actually enjoy.

But why? A 2012 study published in Psychological Science showed that putting feelings into words could help us cope with stress. Psychologists did an experiment to see if verbalizing current negative emotions might be effective in helping people with spider phobias. 

"Participants who put their negative feelings into words were most effective at lowering their levels of physiological arousal. They were also slightly more willing to approach the spider. The findings suggest that talking about your feelings - even if they're negative - may help you to cope with a scary situation."

On June 24th, right after I learned about Brexit news, I launched a HEARTBEAT Pulse survey in UK - a one-question survey on Google Survey platform asking 300 "gen.pop" responders across UK how they feel about the future of their country. People shared their emotions from their heart: "concerned and saddened,"  "Very bleak and old fashioned. Us young people have been neglected," "When i think about the future of our country, i feel that unless we all learn how to pull together, we'll be in for a rocky ride...until we Do learn how to!"

Finally, when I started sharing our Emotion Analytics maps during presentations or on social media, people often displayed a strong reaction of empathy and curiosity. Seeing other people's emotions - especially Sadness, Anger, and Fear - quantified in a way that does not explain or simplify them into positive/negative creates an opening, a possibility to learn something new, and wonder what other people feel. 

(To play with this dynamic map and see what people in UK feel about the future of their country this week, please click here (n=300). To compare it with US, Australia and Canada - check out this chart.)