AI Asia: Canadian Startup Heartbeat AI Finds Success in Japan and South Korea

AI Asia: Canadian Startup Heartbeat AI Finds Success in Japan and South Korea

Lana Novikova is the Founder & CEO of Heartbeat AI, a Toronto-based AI startup that has developed a platform that helps users better understand the emotions of their clients and employees to make the best decisions. In this interview, Lana shares her experiences and insights on the field of AI and Canadian opportunities for this sector in Asian markets.

Teaching computers empathy

Teaching computers empathy

Lana Novikova is on a mission to teach computers to “read” your emotions.
Why is this important? Her Toronto start-up, Heartbeat AI, is connecting the dots between artificial intelligence, cognitive sciences, consumer research and marketing to help organizations better know their customers, employees and patients – by understanding how they feel.

Is Your Company Emotionally Intelligent?

Emotional connections are at the heart of enduring or loyal customer relationships, as well as employee engagement. The difference between fleeting transactional customer interactions and experiences that leave a lasting impression – whether positive or negative – is whether the experience strikes an emotional chord with the customer. 

My Experience Working With Heartbeat

(by Deanna Danelon)
I began working for Heartbeat Ai as an intern through Cisco’s Women Entrepreneurs’ Circle program. This program connects University of Waterloo engineering students with female entrepreneurs to provide technical support and services so that women entrepreneurs can optimize their businesses. Though I haven’t been a part of the Heartbeat team for long, these past few months have provided some incredible experiences, and I have learned so much.

The greatest experience has been the opportunity to witness what incredible things women are capable of, thanks to founder Lana Novikova. Seeing Lana’s success and hard work as a women entrepreneur, despite working a tech field, has been incredibly inspiring. Working and studying in a male dominated field, such as engineering, it is easy to feel discouraged or inferior. Being a part of the heartbeat team has shown me that women are capable of absolutely anything. This team consists majorly of inspiring women who work hard to support each other, and reach success, defying all odds. This has provided me new found confidence that being a woman is not a setback, all we have to do is support one another.

One of the most important things I’ve learned since working here has been how important empathy is. Having a technology that can understand emotions is a complete game changer. We are not human without our emotions, and having a technology that we can work with to help better understand emotions is a huge step towards greater human potential.

I am so lucky to have become a part of Lana’s team and to have been able to gain the experiences I did. I work with incredible, inspiring, and hardworking people, who are working to help make the world more empathetic.

The Future of Emotionally Intelligent AI

Society will get the AI it deserves.” 
- Joscha Bach, What to Think About Machines That Think

Today, AI-powered machines can defeat the most skilled players in chess, a game of Go, Jeopardy, and poker. They can outperform best doctors in diagnosing most complex health conditions. Yet when it comes to understanding a wide range of human emotions, AI is not very intuitive. In his 2016 Google talk, Professor David Gelernter said: “ Understanding emotion as an encoding and simulating function is one of the most important unsolved problems - in fact, ignored problems and missed opportunities in AI today.”

Why do we need to teach thinking machines to understand human emotions? While applications for emotionally intelligent AI are too many to list here, let me mention three areas that we, at Heartbeat AI, are most fascinated about.

Deeper understanding and better prediction of irrational human behavior

We don’t need to try hard to convince anyone today that human beings are emotional first and rational second. Decisions that we make every minute - from what flavor of Starbucks to buy to who to vote for - are made deep inside our emotional minds influenced by many conscious reasons and sub-conscious “primes.”

Last Summer, we decided to run an experiment and see if we could build a predictive algorithm for national elections using unstructured text data from surveys. Traditional election polling asks questions like “If elections were held today, who would you vote for?” or “Who do you think will become the next Prime Minister of Australia?”. We picked 3 traditional questions and added two open-ended questions asking how people feel about each candidate winning the elections. The survey ran on Google Survey Platform for 6 weeks prior to the June 2nd Australian elections and November 8th US elections. Then we used a predictive model to call the results. In both cases, emotional measurements helped improve the level of predictability, and therefore made correct predictions for Australia (predicted for Bill Shorten vs. Malcolm Turnbull: 44% vs 56%, actual: 48% vs. 52%) and US (Donald Trump vs. Hillary Clinton):

  • Florida: predicted 52% versus 48% actual: 51% vs. 49%; 
  • Iowa: predicted 58% vs. 42%, actual 55% vs. 45%; 
  • Ohio: predicted - 63% vs. 37%, actual - 54% vs. 46%;
  • Pennsylvania: predicted - 54% vs. 46%, actual - 51% vs. 49%

What was even more interesting is that the models showed what particular emotions drove voters’ behavior, which explained the deep “why” behind people’s decisions.

Emotion text analytics is a new, possibly groundbreaking, area of research and analytics. What if we could rely on text data to be the “secret sauce” for accurate prediction of future events that are based on human decisions, such as elections, consumer behavior, public opinion and social movements.

Apps that will help people deal with stress, change, and suffering

Eight years ago, I took a break from my career in market research and went to study psychotherapy. I had a small private practice for a couple of years, helping people deal with loss, trauma, distractive anger, and debilitative fear and anxiety. Inside my own family and friends, I witnessed hard struggles my loved ones fought with alcohol and drug addictions. At the end, I did not find the stamina to be a psychotherapist, but I never stopped wondering if good therapy could be scaled, free and available when and where people need it the most. Then two years ago, an idea of an algorithm that could recognize a wide range of emotions from language - written and spoken - came alive. That’s how Heartbeat was born.

This Spring, we are starting a pilot with a small residential drug and alcohol addiction rehab in Ontario. We will give people a chance to share how they feel in the most difficult moments of their lives, recognize and name their emotions, and track their progress. We hope to build predictive algorithms that could help prevent relapse and foster resilience.

This is just the first step in a long journey. With 6 billion people with access to mobile phones by 2020, apps can help us not only find a good restaurant or a Pokemon, but peace of mind too. In 2016, the Guardian stated that “AI could leave half of world unemployed”. Imagine the stress and anxiety this would trigger! AI-powered apps could emotionally support millions of people transition from losing a job to retraining for new careers, and building new lives.

Building the foundation for benevolent AI

“Artificial minds will be faster, more accurate, more alert, more aware and comprehensive than their human counterparts. AI will replace human decision makers, administrators, inventors, engineers, scientists, military strategists, designers, advertisers, and of course AI programmers,” writes Joscha Bach of MIT Media Labs. He continues: ” The motives of our artificial minds will (at least initially) be those of the organizations, corporations, groups and individuals that make use of their intelligence. If the business model of a company is not benevolent than AI has the potential to make that company truly dangerous. Likewise, if an organization aims at improving the human condition than AI might make that organization more efficient in realizing its benevolent potential.”

When something possesses that much power, its makers ought to carry a high level of responsibility. It’s absolutely paramount to ensure that our powerful machines have benevolent “souls”, including intentional kindness, deep understanding of human condition, and cognitive empathy. What if AI could help us solve the biggest Global challenges, help humanity create Global governance, social and corporate systems, and ultimately rebuild trust?

Heartbeat is building a platform for people to share emotions and feelings about “hot” topics, burning political issues, the future, and life in general. We want to use emotion text analytics to help people understand their own emotions, see emotions of others across the world, and with that, have a window into each other’s souls. I hope that a Global “empathy meter” will grow if instead of opinions and agendas, people share personal feelings and see how fear and anger drives us away from each other.

If we are honest with ourselves, we would admit that we get the partner and the government we deserve. At the end of the day, we will get the AI we deserve. 

(This article was published in RE-WORK - see here)

Lana's Talk @Dx3, Toronto: How To Create and Measure Emotional Connection To Brand Narrative

Speaker: Lana Novikova, Founder & CEO - Heartbeat AI

Description: “Human emotions and feeling are part of a complex system that is not yet completely understood.” Emotional connection to brands are a big part of the consumer experience today, and yet one of the most difficult aspects to understand, measure, and improve upon. Lana Novikova, founder & CEO of Heartbeat AI, is an expert in exactly that - understanding and measuring human emotions. In this session, Lana provided tips on how to better foster emotional connection in your marketing efforts.

Read more about Lana's talk  here.

Read more about Lana's talk here.

Where Are They Now? IIeX EU 2016 Competition Winner Heartbeat AI

At IIeX EU in 2016 the winner of the Insight Innovation Competition was Heartbeat AI. Now in honor of the next round of the IIeX EU Competition next week we touch base with Lana on the Heartbeat journey, where they are now and where they are going.

At IIeX EU in 2016 the winner of the Insight Innovation Competition was a unique emotionally-based text analytics solution led by our first female CEO winner, Lana Novikova of Now almost a year later, and in honor of the next round of the IIeX EU Competition next week, I decided it would be interesting to touch base with Lana on the Heartbeat journey, where they are now and where they are going. 

First, a bit about their SaaS platform in their own words:

True deep insights can’t be found among scores and scales. Numbers have a definite role in research, yet only words and actions can point to our true hearts’ desires. Like raw diamonds, words often need skilled work to transform into a genuine sparkle. Survey open-ended comments that contain deep insights into human emotions, needs and motivations, often get discarded. Manual data coding is time consuming and labour intensive, pricey and inconsistent from study to study. HEARTBEAT brought together market research, psychology, neuroscience, data analytics, and technology to enable researchers to find and polish these “insight diamonds” – quickly, affordably, and consistently.

You’ll notice the use of almost poetic language in their description, which is absolutely “on brand” for Lana and her team; they approached the development of their platform not as an exercise in technology advancement but as an effort to combine myriad disciplines to address a need to create scalable “understanding” tools. Market research was one example of use cases, but the needs of healthcare, public policy, education and other categories drove their thinking as much as anything else.

The result is a uniquely powerful, beautiful, and elegantly simple solution that can be used by novices and experienced pros alike, with an API based model that allows their tech to be plugged in to power or augment many different complementary platforms.

Read the full article here and watch the interview:

What is emotion analytics and why is it important? (Interview with Seth Grimes)

by Seth Grimes , Alta Plana Corporation

Link to article

Interactions, facts and feelings shape our relationships. A truism: It's not what you say, but how you say it. Expressions matter, as do the sentiment behind each encounter and the emotions raised. Emotion is entwined with the literal meaning of words used.

This fact/feeling principle applies to both inter-personal and business relationships. "Emotional and factual appeals cannot be easily separated," writes Nigel Hollis of Kantar Millward Brown in an analysis of advertising approaches. "[A] distinction between emotional and rational is one that exists only in the minds of marketers, not consumers," according to Hollis.

The fact/feeling equation is central to corporate customer experience (CX) initiatives. CX practitioners map customer journeys that are defined by both the what and the how-did-it-make-you-feel? of customer-brand interactions. "Emotion drives loyalty," according to CX visionary Bruce Temkin, and loyalty drives profit.

Another truism: You can't improve what you don't measure, not systematically, on a corporate scale.

Sentiment analysis and the varieties of emotion AI

Enter sentiment analysis, software technology that quantifies mood, attitude, opinion, and emotion in digital media, in images, video, audio, and text. One subspecies infers emotion via facial-expression analysis. Providers include AffectivaCrowdEmotionEyerisKairosnVisoNoldus Information TechnologyRealEyes, and Sightcorp. Another variety analyses emotion in speech. Check out audEERINGBeyond VerbalEMOspeechGood VibrationsNICEVerint, and Vokaturi. On the text front, natural language processing (NLP) techniques can identify and extract emotion in online, social, and enterprise sources, delivered by companies that include ClarabridgeCrimson Hexagon, Feedback Ferret, IBM (AlchemyAPI and Watson Tone Analyzer), indicoReceptiviti, and an advisee of mine, Heartbeat AI Technologies

This article aims to get a handle on the state of emotion analytics — specifically, emotion in text — via an interview with Heartbeat founder Lana Novikova. Lana describes herself as a marketer by training and a market researcher by trade, never satisfied with numbers and observations, always pushing to understand the "deep why" behind consumer (and her own) decisions. She'll be presenting at LT-Accelerate, a conference I organise, November 21-22 in Brussels, alongside Odile Jagsch, a consultant at global market research consultancy Kantar TNS, topic "The 'Why' Behind Customer Loyalty."

Seth Grimes. Heartbeat designs "emotionally intelligent technologies." OK, what's an "emotionally intelligent technology"?

Lana Novikova. Let's start with the concept of Emotional Intelligence (EQ), popularised by a psychologist Daniel Goleman in 1990s.

Imagine a newborn human who comes with a basic wiring for recognising and expressing key emotions, and with an enormous capacity to learn. She can cry or stay calm, smell and turn her head towards her mother’s breast. Once she can see faces, her mirror neutrons start learning and mimicking facial expressions; then she develops more and more capacity to read and express emotions — from touch, to tone & voice recognition, to basic language to more complex if-then scenarios. In a perfect world, she grows into a secure and happy person who can recognize and name a wide range of her own emotions, understands what other people feel from multiple expressions, and has a capacity to express and manage her emotions.

SG. So you apply the EQ concept to and via technology.

One day, this technology might surpass humans in understanding human emotions because it will tap into data that humans can not perceive on their own.

LN. Technology today has a super high IQ — it can beat the best human chess, Jeopardy, and Go players — yet it has a very low EQ. At Heartbeat, we want to be a part of an academic and business community that changes this. Emotionally intelligent technology is never going to feel emotions or express them like our baby can, but it will eventually become very good at perceiving and understanding human emotions from data.

One day, this technology might surpass humans in understanding human emotions because it will tap into data that humans can not perceive on their own: biometrics, brain waves, subtle cues from body language and facial expressions, and more.

SG. Relating the tech to Heartbeat...

LN. We are focusing on training the (metaphorical) technology infant to recognize explicit feelings from language, from text, and to guess the range of emotions it communicates. Just as some people can intuitively differentiate between many emotions, our growing algorithm can tell what kind of Joy or Anger is expressed in language. There could be as little as 2-3% and as much as 50% affect words and phrases in any given unstructured text. We find these words and assign them to a cluster of emotions. This process mimics how our brain deciphers emotions from language.

SG. What aspects of emotion does Heartbeat detect and measure? Do you adopt a particular emotion model?

LN. There are a few models and classifications of emotions developed by brilliant psychologists like Paul Ekman and Robert Plutchik, and even by Human-Machine Interaction Network on Emotion (HUMANE). I was inspired by a more intuitive model of W. G. Parrott (2001), originally described by Shaver in 1987. It has a tree structure and includes Primary, Secondary and Tertiary emotions. I also did a lot of reading about effective neuroscience, and tried to combine Parrott's model with what I took from the work of J. LeDoux, R. Davidson, and J. Panksepp. Then my "practical life-long quant researcher” side took over and asked, “How is this segmentation going to be useful to a brand of chocolate, or a bank, or a political party?"

The art of analysing good quality text data lies in understanding (a) how to ask a good question, and (b) how to infer meaning from people’s answers.

We ended with a 2-level clustering of 99 complex emotions and feelings into nine primary emotions: Joy, Love, Trust, Anger, Fear, Disgust, Sadness, Surprise and Void (which is explicit lack of emotion like in “I don’t care"). We also added Body Sense (positive, negative and neutral) as a way to analyse words and phrases that don’t point to a particular emotion, yet are useful for understanding human perception overall, especially for marketing food and body care products.

Many words and phrases are coded into multiple emotion clusters. Would you agree that there is a large overlap between Disgust and Anger, or Love and Joy, or even Anger and Fear? For example, we put the word "terrific" into both Joy and Fear, and let context decide which emotion it is more likely to represent. This is the most challenging part of our journey — understanding how different context colors "terrific" into happy or unhappy expression. It's the domain of Machine Learning that needs lots of training data. We are just scratching the service here, but this is also the most exciting part of my job.

SG. How does the tech work?

LN. Today, our tech is very simple yet very accurate. It’s called "bag of words": 8,000+ words and phrases (including negation, metaphor, and other multigrams) professionally coded into categories and validated by skilled psychologists and psycholinguists. Our software consumes unstructured text from survey responses and social media, and produces a set of visualisations and charts in a simple elegant dashboard.

We do our best work when we analyse data from survey responses that focus on people’s feelings. Data like that produces over 30% affect words, and has the context controlled by researcher. Once the report is ready (which is almost instantly), we curate results by removing some words that do not apply to the report. This is how we deal with another industry challenge, ambiguity. Finally, to prove that we are very good at what we do - we show all words and phrases for each Primary emotion. You can click on any word and see exactly how it appeared in the original text - no “black boxes” here. Since our taxonomy reached 5,000, our match rate — the percentage of affect words that we recognise — is over 95%. Heartbeat is committed to accuracy, depth, and transparency.

SG. How is Heartbeat different or better than the competition?

LN. Heartbeat is different because it was created by a market researcher (myself) who spent hundreds of hours coding open ended survey data. My team built our award winning app for researchers and marketers who appreciate the depth of consumer insight. I love working with good quality data, and would choose quality over quantity any time.

Survey data is under-used and often abused. The art of analysing good quality text data lies in understanding (a) how to ask a good question, and (b) how to infer meaning from people’s answers. I believe Heartbeat is better for distilling emotions from open-ended survey questions than any other company on the market today including IBM/Alchemy and other powerful APIs. They can do a lot of advanced text analytics with huge amounts of data. We made it simple, transparent, and fun. Just check out our dashboards — clients love it! Another big differentiator is that we are 100% focused on emotions — not sentiment or basic emotions, but fine-grained feelings. Our reports can be useful for anyone - from a CEO and CMO to a brand manager to a CX analyst to an agency creative director.

SG. How do Heartbeat emotion text analytics findings complement or compare with insights discovered via neuroscience and biometrics, via facial image recognition and brain-activity measurement?

LN. I strongly believe in cooperation over competition. I think putting our best technologies together will create a better future for our businesses and for this planet. A graphic will communicate how I see the future of emotion AI integration. (See the image below.)

How Heartbeat AI sees the progress of emotion analytics.

How Heartbeat AI sees the progress of emotion analytics.

SG. Could you please sketch out 2 or 3 use cases? What's your target market — clients and applications?

LN. Let’s start with the customer, the consumer or shopper. What do all customers have in common? They're human, they feel emotions all the time, and those emotions drive many (if not most) of their decisions.

Why not measure emotions on every step of your customer journey? It's not easy to put an eye tracking and EEG device on thousands of people, but it's easy to ask one simple question, "Please share a few words that best describe how you feel about X." This natural human question fits nicely into any survey and customer feedback tool, and it’s surprisingly powerful. It not only invites a wide range of un-aided and unbiased feelings on the subject, but it can also aid at predicting what people will do in the future.

We're a start-up, but we have already published two case studies to show the predictive power of emotions measured with HEARTBEAT emotion analytics: banking and political elections. The best application of HEARTBEAT is in on-going customer experience measurement and foresight.

SG. What's on your product roadmap?

LN. We are driving in the fast lane! We launched our first prototype last December and won an international startup competition (Insight Innovation Competition by GreenBook) in March 2016 in Amsterdam. All Spring and Summer, we tested our tool with some of the best research companies in the world. Finally, we launched enterprise SaaS this fall, which will enable anyone — brands, market research suppliers, consultancies, marketing agencies — to access the engine. No more manual coding: leave it to a machine, fast and accurate.

Next, we're venturing into a long complex journey of NLP and machine learning, to crack the challenge of context and meaning.

Here's a quote that resonates with me, especially when it comes to trying to solve one of the biggest puzzles in the Universe, the puzzle of human consciousness: "The most influential thinkers in our own era live at the nexus of the cognitive sciences, evolutionary psychology, and information technology." That's New York Times columnist David Brooks.

The mission of Heartbeat Ai is to design emotionally intelligent technologies and tools to help machines understand peoples' feelings and improve our emotional wellbeing. I don’t know exactly how we'll get there in the end, but I know that we are on the right track today.

SG. Thanks, Lana!

Readers, I've written up a couple of other emotion analytics interviews — an IBM Watson blog contribution, Sentiment, Emotion, Attitude, and Personality, via Natural Language Processing, based on a conversation with IBMer Rama Akkiraju and On Facial Coding, Emotion Analytics, and Emotion Aware Applications with Affectiva principal scientist Daniel McDuff — and of course you can learn more about Lana's Heartbeat work atLT-Accelerate in November. If you can swing a trip to Brussels, see you there!

(This article was originally published at on 12 October, 2016)