How Facebook built language translations in Messenger with M Suggestions

Necip Fazil Ayan and
Laurent Landowski.
Samantha
Lee/Business Insider

• Facebook is on a mission to help people who speak
  different languages understand one another.
• It’s using M Suggestions, its virtual assistant in
  Messenger, to translate real-time conversations, and has just
  added support for French.
• Business Insider spoke to Facebook employees working on
  the project about the tech behind it, and its potential to
  radically affect online communication.

In 2015, virtual assistants were the Next Big Thing.

Every major tech company had one, from Apple’s Siri to Amazon’s
Alexa and Google Now, which would later become the more fully
fledged Google Assistant.
Facebook entered the space in August that year with the
announcement of M — a chatbot that lived inside its Messenger
app, and which users could ask for just about anything, from
hiking recommendations to help buying flowers. 

But while the likes of Alexa and Google Assistant have exploded
over the last few years, finding their way into everything from
fridges to cars, M — in its original incarnation — was never
widely rolled out to Facebook’s users. 

Instead, it morphed into M Suggestions — an AI enhancement that
hovers inside users’ Messenger chats with their friends, and
offers contextual suggestions based on their conversations — from
making payments to initiating video calls. These recommendations
have historically been relatively incremental — send a funny GIF!
Attach a sticker! — but Facebook is now leaning hard into
translations in Messenger, and using M Suggestions as the engine
to do so.

It’s by far the most significant application of the technology
behind M Suggestions to date — one that has the potential to
radically reshape how hundreds of millions of people communicate
in real-time on Facebook. Business Insider spoke to some of the
team at Facebook working on the project, to learn more about the
tech underpinning the ambitious project and their vision for its
future.

“At Facebook we have a lot of different cultures and a lot
of diversity in our team … we are all speaking different
languages, and we know how frustrating it can be not to be able
to communicate … in your own languages,” said Laurent
Landowski, a Facebook product manager. “So being able to also
really provide M Suggestions for translation to the world is
something that we’re super proud of.”

M is an AI assistant that lives inside other conversations

Every time you send a message to a friend via Messenger —
assuming it’s not an encrypted “Secret Message” — it’s being
scanned by Facebook.

That doesn’t mean an actual Facebook employee is reading it, of
course. Instead, Facebook’s automated systems are parsing the
message, trying to understand the intent of the message. This
effort is partly underpinned by the tech Facebook acquired when
it bought natural language startup Wit.AI back in 2015; cofounder
Landowksi is now the product manager of M Suggestions at
Facebook.  

If one of Facebook’s AI neural net models identifies the message
between you and your friend as something it can add context to, M
Suggestions will automatically spring into action. If you mention
a song, it might prompt you to play it on Spotify; if you’re
discussing multiple potential activities in a group, it might
suggest creating a poll.

And it learns over time what different people utilize it for, and
caters its responses accordingly; the version of M that a
GIF-addict sees will be very different to what appears for
someone who is more restrained in their messaging.

When it comes to quick replies — suggested responses M offers
users in conversations to save them time — it even uses users’
conversation histories as a training data to teach the AI how
they speak, making the responses (“yes” versus “yeah” or “yep” or
“yah”) sound authentically like their voice.

Its unique positioning — inside users’ existing conversations
with other people, rather than dedicated human-to-AI chat windows
— means it risks being invasive or jarring. As such, Facebook has
moved slowly adding suggestions to M, Landowski said. “We have
been working on trying to improve and really focus on the delight
and relevance as opposed to the number of suggestions we could be
suggesting.”

He added: “It is super easy to lose user trust.”

Facebook

Translations aren’t easy — but the pay-off is huge

M Suggestions has thus far offered fun enrichments to
conversations, but it’s hardly transformative. Where that
changes is translations.

Facebook has provided language translations on its core social
network for years, first via traditional phrase-based translation
techniques before migrating to a more advanced AI-powered neural
net translation system in August 2017. 

However, users who wanted to be able to talk across language
barriers in real time were out of luck until earlier this year,
when Facebook launched the first Spanish-English translations in
Messenger, underpinned by M Translations. The company followed it
up with the announcement this week that it was adding French.

While Facebook positions Messenger at least in part as a
way to stay in contact with the people users are closest to,
translations opens it up to assisting people who may never have
interacted before — like in Marketplace, Facebook’s peer-to-peer
sales platform. 

“You see more and more ways where translations can be
applied, not only in your personal messaging but also like in the
Marketplace, buyer-and-seller-type of messaging, that can unlock
a lot of further opportunities for use,” said Landowski, a native
French speaker from Paris.

But translation isn’t easy. Languages are always changing
and shifting, evolving as slang becomes common parlance, and the
problem is especially acute on an informal, real-time platform
like Messenger. Facebook’s language team has an “active taskforce
working to adapt its models to the type of data that Messenger
provides, said Necip Fazil Ayan, head of Facebook’s language and
translation technologies team.

“This is one of the hardest problems we have to deal with
while working on translations at Facebook, and it’s not a solved
problem, making our systems more robust to informal language
including slang,” he said.

“It’s a dynamic language right, and people keep inventing
stuff … my best example instead of just saying ‘happy
birthday,’ [they] start introducing lots of P’s or Y’s or I’s all
over the place.”

Facebook’s unprecedented digital archives of billions of users’
public and private conversations means it has a vast dataset with
which to train its AI — but some languages have more material
available than others. “One of the biggest challenges we’re
dealing with, both in terms of language understanding and
translation perspective, is the set of what we call ‘low resource
languages.’ As you can imagine, all the machine learning models
require a lot of data to become accurate … and we don’t have
that luxury for a lot of the languages we are dealing with,” Ayan
said.

And then there’s the issue of bias. AI systems are only as good
as the data they are trained on, and when there are human biases
built into the data, it can creep into the results. “Our data is
biased … we are actively working on this at Facebook … in
machine learning this is a very hot area and it’s a very
difficult area, and it’s going to take a while cleaning up the
data from that type of bias or learning where the bias is.” 

But for all the challenges, automated real-time translations
offer Facebook a way to have a have a fairly profound effect on
the way people around the world communicate. “I really don’t want
language to become a barrier when people are expressing their
opinions or when people are trying to reach other people to get
different perspectives,” said Ayan, a native Turkish speaker who
grew up in the country.

“So that’s the dream, we are going to break down the language
barriers and that’s my personal mission here.”

‘It’s hard from an AI perspective to be able to create a fully
automated assistant that can do everything’

Facebook now describes the original version of M as an
experiment, one that provided valuable insight into the kind of
things users utilise chatbots for but was never intended as a
competitor to Siri et al, and never scaled beyond 2,000-odd users
in the Bay Area. (In contrast, more than 100 million people
interact with M Suggestions a month as of November 2017, a
spokesperson said.)

“It’s hard from an AI perspective to be able to create a fully
automated assistant that can do everything,”Landowksi said, “We
basically realized that people, especially in Messenger, they
really want to focus on the communication [assistance that M
offered] … it was really about trying to be where they are,
which is their actual conversations. Instead of talking to an
assistant directly it was all about focusing on the
communications.”

But there have been rumours circulating for months about Facebook
building a smart speaker in the vein of the Amazon Echo of Google
Home, with a voice-controlled AI assistant built in.

It was
a no-show at Facebook’s annual F8 conference in May 2018, but
speculation was bolstered after a reverse engineer
found references hidden in Facebook’s code for an “Aloha” voice
feature.

“We’re exploring everything,” Landowski said when asked about
whether Facebook was currently looking at speech recognition.

“Speech is also an interesting way to actually interact with an
assistant. We cannot comment more on exactly what we do and what
we test, but definitely we are working and trying to improve
this.”

Contact this reporter via Signal or WhatsApp at +1 (650)
636-6268 using a non-work phone, email at
rprice@businessinsider.com, WeChat at robaeprice,
or 
Twitter DM at @robaeprice. (PR pitches by
email only, please.) 
You can
also contact
Business Insider securely via SecureDrop.