Waymo engineer explains why testing self-driving cars virtually is critical

Neuroscience led James Stout to computer science, which
led to Google and Waymo.
Waymo

• James Stout was convinced that he’d become a
  neuroscientist, but a shift to computer science led him to
  Google and later Waymo,
  where he pioneered on the company’s virtual-realty testing
  simulation, Carcraft.
• Carcraft has tested vehicles in five billion miles of
  simulations, while Waymo has racked up a mere eight million
  miles of autonomous driving in the real world.
• More recently, simulated testing has begun to merge
  with actual testing and Waymo has been able to install sim
  systems in real-world vehicles.

Editor’s note: Business Insider had the chance to speak with
Waymo employees from different parts of the company to learn more
about their work. What we discovered were some of the
coolest jobs at Alphabet, Waymo’s parent company. This is the
second in the series. For a brief history of Waymo,
click here.

Waymo has racked up over eight million self-driving miles since
the former Google Car project kicked off almost a decade ago. But
that impressive real-world number is nothing compared to the
mileage amassed in virtual reality.

“We do eight million miles every day in simulations,” said James
Stout, a senior staff software engineer at Waymo. 

Stout has worked at Google since 2009. He joined the Google Car
project full-time in 2013 and applied his background in
neuroscience and artificial intelligence to the daunting task of
teaching a vehicle to drive itself. Thus far, he and his
teammates have navigated an astonishing five billion miles in a
“Matrix”-like virtual real called “Carcraft.”

“It’s running 24 hours a day on Google’s data centers,” he said.
“And there are 25,000 cars in the virtual fleet.”

The trick, according to Stout, is to isolate what he called the
“truly interesting” miles in that welter of information. And his
group doesn’t make it easy for the virtual vehicles, which in a
system that Neo would appreciate can enter training rooms made
from computer code before they’ve dispatched to a ‘real’ world to
tackle myriad challenges.

Different universes, different worlds — and fuzz

How
a Google Car sees the world.
Waymo

“We have different universes and worlds running,” Stout said.
“And we’re testing small variations over and over while creating
situations that the vehicles have never seen before.”

This is where Carcraft moves beyond anything the Wachowskis
dreamed up.

“Fuzzing comes into play,” Stout explained. 

The concept is esoteric, but without it, Waymo’s actual-world
combinations of laser-radars, sensor, and programming  —
what CEO John Krafcik calls a “driver” — can’t replicate
something the human brain has been managing for over a century:
pilot a powerful machine with four wheels through a wide range of
environments. (Including the Moon, where astronauts drove a lunar
rover in the early 1970s.)

Stout said that he and his team will start with the skeleton of
an idea — say, a four-way stop, something that’s widespread and
routinely dealt with by human drivers, but that can involve
numerous vehicles and entails complex decision-making. 

The many real-world inputs, drawn from what Waymo vehicles have
experienced, can then be fuzzed and varied, multiplied, and then
analyzed as edge cases. This results can then be fed back into
Carcraft and applied to real-world Waymo testing, making the
company’s driver more robust and capable. 

“I look at it like a scientist,” Stout said. “The goal is to
explore those possibilities.”

A
fuzzing sample.
Waymo

Humans are good at driving, but Waymo’s driver have some serious
advantages

There are some obvious advantages here for Waymo’s self-driving
objectives, which recently landed the company, spun-off from
Google’s parent Alphabet in 2016, a $175-billion valuation from
Morgan Stanley. Waymo is moving toward a commercial rollout in
2018 and 2019, with an intense focus on safety and versatility.

Waymo’s driver really is that: a sort of disembodied robot pilot
and navigator that could be installed, according to Krafcik, in
multiple transportation platforms, from passenger cars to semi
trucks. Humans already know how to drive everything that can be
driven. But we have a poor safety record: nearly 40,000 people
die in vehicle-related accidents every year in the US alone.

Very few humans besides possibly professional race-car drivers
are running 24/7 simulations in their brains of what fuzzy edge
cases they might encounter while behind the wheel (and even
race-car drivers have to sleep, when they might or might not
dream of driving). Autonomous vehicles are among the toughest
problems that computer scientists can take on, so Stout and his
Waymo colleagues are using everything they know and amplifying it
with massive amounts of computing power.

How
Waymo maps the world in simulation.
Waymo

Beyond having Carcraft as its collective mega-mind, the Waymo
automated driver never gets tired and is better designed to deal
with the cognitive difficulties of moving a vehicle from point A
to point B.

“The car is always paying attention,” he said. “Human
front-facing eyes evolved for hunting, but the self-driving car
is looking 360 degrees all the time.”

The Waymoers I’ve spoken with over the past year are some of the
most thoughtful people I’ve ever encountered, and Stout was no
exception. His interest in neuroscience began in high school, but
when he discovered that computer science enabled quicker
feedback, he made the shift halfway through his undergraduate
studies at Brown University. (Stout is also involved with
hands-free computing and speech-recognition, which he blogs about
at handsfreecoding.org.)

At Google, he worked on search and Google Earth, leaning how to
do what he called “very big stuff.” 

But after he spotted some early Google Cars — modified Toyota
Priuses — zooming around parking lots at the Googleplex in
Mountain View, the self-driving project became his dream.

“What made me want to join was that the people on the team were
smart and charismatic, but also humble,” he said.

The 20% solution

What
Waymo explores in Carcraft can then be applied to real-world
testing.
Waymo

He wasn’t sure how to get hired, but he figured he had some ways
to make himself useful. He concentrated on simulations, something
that the Google Car team wasn’t then regularly running. At first,
it was an undertaking that consumed 20% of his time.

Google allows employees to work on these side projects with the
understanding that they could yield major dividends, and for
Stout his 20% efforts planted the first seed for Carcraft and led
to him joining the team.

Fast-forward five years and Stout still eats some oatmeal for
breakfast and might review a technical text to get his mental
gears turning before heading into the office. Then he hits the
ground running, as he describes it.

“The team is always working hard, and there are always
interesting discussions going on,” he said. But it never gets
old.

“Overall it’s super exciting.”

And for Stout, the biggest and potentially most satisfying payoff
is anything but virtual.

“I want my parents to be able to hop into a self-driving car,” he
said. “That’s what we’ve all been waiting for.”