This is what it’s like to control an autonomous car from miles away

The exit ramp is a long, curving slope, and you have to make sure the 50-foot big rig you’re driving carefully navigates the bend and doesn’t fly out of control at a high speed.

But the thing is, you’re not actually there. You’re in a room in Silicon Valley, watching the ramp unfold in front of you on several screens. That heavy load you’re carrying is thousands of miles away in Florida. 

Welcome to teleoperated driving, or remote-controlled driving with a human in front of a steering wheel, brake, and gas pedals, and a “windshield” plastered with monitors. It’s a method that allows autonomous vehicles to operate without anyone inside. Instead, there’s a watchful remote driver, or operator, there for trickier moments that the robo-truck or vehicle can’t handle. A few companies offer remote driving systems that work in conjunction with autonomous vehicles, like Phantom Auto and Starsky Robotics. Other robotics companies rely entirely on remote control. Even Waymo, the Google spin-off autonomous vehicle company, has operators at the ready to take over their self-driving cars in a tough situation.

While autonomous vehicles make headway with taxi services in places like Arizona and Las Vegas through Waymo and Lyft, it’s slower going for a truly driverless experience. In California, only Waymo has applied for a full autonomous driving permit — and that’s just for testing. 

Even in places where self-driving cars are operating on public roads, safety drivers are still piled into the front seats, ready to take over the wheel. In contrast, remote-controlled autonomous driving means that cars, truck cabs, delivery vans, and even delivery robots can operate on their own. A human is ready to jump in, much like those safety drivers, but remotely. 

For small bots like Kiwibot, which offer delivery services, the entire operation is remote controlled. Operators wrangle the delivery bots on UC Berkeley’s campus all the way from Colombia, as the SF Chronicle reported. The roving machines rely entirely on the remote operators to move along, which makes Kiwibots different from Amazon’s Scout and Ford’s Digit bots, which have sensors and autonomous capabilities.

Teleoperated autonomous vehicles fall into a gray area on the spectrum of autonomy. Since a person is still in control, they’re not truly autonomous. Remote-controlled driving is mostly seen as a complementary tool, with its nearly imperceptible lag time and high data transmission loads, making it perfect for remote driving and monitoring. One of the teleoperations software companies on the road is Phantom Auto. Its services can also serve as a backup safety system.

Phantom co-founder Elliot Katz said this week in a phone call from an office in Mountain View, California, that his company technically doesn’t provide full autonomy: “Our test vehicles will not move an inch unless someone is driving them.” The difference, of course, is that someone is a driver in a remote location, usually Phantom’s Silicon Valley office.

Eric McCarter, a Phantom Auto test program manager and remote operator, said remote driving “is very much like regular driving,” but you have to rely on different cues. There’s no feel of the vehicle or true pushback on the tires when you turn the wheel. Microphones in and around the vehicle let him hear everything a driver physically in the car would hear; he can see as much and even more thanks to cameras showing what’s going on all around the vehicle, even behind it. So while it’s still the driving basics, “you have to relearn how to drive” in this hybrid digital-real world situation through repetition and in parking lots or on empty, barely used roads.

At the end of April, Bay Area-based Starsky Robotics showed a test drive in Florida with remote driver Jas Bagri teleoperating a truck while safety driver Luis Velez sat in the driver’s seat without actually driving for the 20 minutes of testing. Here’s the 360-degree video version of the drive. 

One of the remote-controlled companies, Starsky, was born from the question, “What if you just made a truck remote-controlled?” to help with the growing long-haul truck driver shortage problem, co-founder and CEO Stefan Seltz-Axmacher told me in a phone call this week. His teleoperations company works with autonomous long-haul trucking companies and brings in remote-controlled driving for the so-called first and last mile. That can involve getting from the distribution center to the highway, pulling up to a fuel station, backing up to a loading dock, or going through toll booths. These tend to be the moments when an autonomous truck struggles. So operators come in and help. “It solves a lot of hard things in autonomy,” he said. 

Driving a truck is really hard, Seltz-Axmacher admits. Doing it through remote control is maybe just as difficult, until you’ve had enough training. After attempting to remotely back up a 50-foot long trailer between two parked trucks, for instance, he said he returned to the professional truckers with his tail between his legs. On the flip side, long-time truck drivers still require practice for remote control driving and go through one-week trainings to get used to operating a truck from in front of a monitor instead of the road.

Phantom operator McCarter finds remote driving similar to traditional driving, since both are high-focus tasks. In the case of teleoperating, McCarter is only monitoring the situation. He says his actual driving sessions are usually shorter, popping in to solve an “edge case” — like if an autonomous delivery robot encounters an unexpected construction crew fixing a pothole. Then McCarter, who has been monitoring the ride, is alerted and he can jump in, get around the construction site, and hand the reins back to the autonomous bot.

Remote-controlled vehicles might also help us get used to autonomy. A Capgemini study on autonomous car perceptions from last month found that about half of consumers trust self-driving cars to run errands for them or pick up and drop off friends and family members. Just under 50 percent of more than 5,500 consumers surveyed said self-driving cars invoke “fear.” It’s comforting knowing there’s a human right there who can take over, even if they’re miles away. 

As Phantom’s Katz summed up, with autonomous cars, we take the human out of the driver’s seat. Now, with remote-controlled cars, we’re “adding back the element people are most used to: the human.”