Snapchat patent uses facial recognition to detect mood

Snapchat has come up with a way of monitoring people’s emotions at events via their selfies.Reuters / Mario Anzuoni

• Snap has patented a way to monitor precisely how people are feeling at events like gigs or speeches.
• The technology would use facial recognition on Snapchat selfies, as well as take cues from messages, to score how happy, sad, or terrified people are feeling.
• Snap outlined how the technology could help event promoters determine the “aggregate mood” of a crowd at an event.
• Just because Snap has patented the technology doesn’t mean the company will actually use it.

Snapchat has worked out a way of using facial recognition to determine precisely how happy, sad, or terrified people are feeling at events — and how it might make money from the technology.

The United States Patent and Trademark Office granted Snap a new patent on August 28 titled: “Determining the mood for a group.” The patent application was originally filed in April 2015.

It outlines a way of scoring people’s moods based on the selfies and messages they send via Snapchat’s app. And it isn’t just a case of determining whether someone’s generically happy or sad, either. Snapchat floated a ranking system that scores exactly how you feel across a complex range of emotions.

Here’s a diagram from Snap’s patent documents showing an example model of human emotions:

USPTO/Snap

According to the documents, Snap would monitor the emotions of groups of people at an event or location, using location data and signals to determine where lots of people might be gathered. The company provides some helpful examples via diagrams.

Here’s the system detecting emotions in a group selfie, determining the precise level of happiness of the people being photographed:

USPTO/Snap

And how people are feeling during a speech:

USPTO/Snap

The filing suggests that Snap could charge event organisers a fee for monitoring how the crowd is feeling. Here’s what the documents say about commercialisation:

“In one embodiment, the event module charges an event promoter, host, manager, or the like, according to a fee structure. In one example [it] identifies a fee to monitor crowd mood at the event… The event module can allow for various fee structures to charge the third party entity… For instance, the third party entity can be charged per location of use of the mood detection system and/or for exclusive or partially exclusive, use of the aggregate mood level.”

The patent doesn’t mean the technology will make it to the Snapchat app. The company has filed numerous patent applications for facial recognition and emotion recognition tech which have yet to appear in any products. This includes blocking people’s faces out with emoji to protect their privacy.

Business Insider has contacted Snap for comment.

Get the latest Snap stock price here.