Facial and voice recognition in cars sounds like a privacy nightmare

I plopped into the front seat, expecting to laugh in the face of the machine attempting to measure my age, gender, emotional state, and comfort level all through infrared cameras and other sensors. But sitting expectantly in the car, equipped with French automotive software company Valeo’s Smart Cocoon 4.0 system, I was flabbergasted when it pinpointed my exact age.

Getting that number right made me trust the car’s biometric system more than I probably should have, even as more tools that measure your heartbeat, track your eyes, head position, voice, and more enter vehicles everywhere. 

At CES this year, driver and passenger monitoring kept popping up. It’s a preview of what will become commonplace in the driver’s seat in the coming years.

All data, all the time

As all this monitoring happens, it’s easy to forget how much data cars are starting to collect — including personal health details like your heart and respiration rate. Different systems manage the influx of data differently, with most keeping any information out of the cloud and away from servers. If the data is kept locally in the car and eventually deleted, or not even stored to begin with, your personal information is less likely to go beyond the car dashboard. At this point you mostly have to trust that your information isn’t vulnerable.

Audi’s Andreas Reich, executive director of infotainment, said at a CES meeting he understands that, “The car is a very private place.” So when your location or health data is loaded, companies like Audi emphasize available security options to opt out of data logging. 

For BMW, using the driver’s voice, eyes, and hand gestures is an idea for future autonomous cars. BMW’s i Interaction EASE is an AI dashboard display that tracks your eyes and the position of your head to better understand how you’re feeling and to control the vehicle. With this conceptual system, cameras and sensors pick up everything you do in the car. Even pushing down on the seat triggers the car to react.

California and your face data   

To power facial or voice recognition systems, cars need to collect information on what your face or voice looks like. Geotab fleet data analytics company executive Colin Sutherland told me in a recent conversation that inward facing cameras open up concerns with personal data storage. With California’s new privacy law, the California Consumer Protection Act, or CCPA, cars need to inform passengers what’s being collected on them just like websites and other online services. This means stored data or information sent back to a server or cloud-based location is much more restricted. 

Alyssa Altman, transportation expert from tech consultancy Publicis Sapient, also said the CCPA and overall concern about data security will force car companies to build systems that “will have to be opt-in for enhanced functionality.”

These AI features can either help with safety or more superficial comfort and convenience, like temperature control and music selection. “[AI features] could give people confidence” in these systems, Altman said, as seen with drowsiness and distraction detectors.

“The car is becoming the hub for AI.”

AI chipmaker Gyrfalcon Technologies, Inc., or GTI’s Marc Naddell, recognizes, “The car is becoming the hub for AI.” It’s an opportunity for convenience with automatic seat adjustments, dashboard settings, and climate control. “Everything can be learned by the vehicle,” he said. On the flip side, the vehicle can learn how to make a ride safer, like slowing down and pulling over when it detects a drunk or sleepy driver.

Voice AI company Cerence developed a public transit voice system to bring real-time information through voice commands. Although based entirely on voice, the set up on a bus could eventually work with a facial recognition program that scans you and gives you personalized information.

Bosch’s Virtual Visor idea is a safety application for AI. It takes monitoring cameras to determine if the sun is bothering a driver’s eyes. The LCD panel darkens only the section of bright light hitting the driver’s face —all without blocking the rest of the windshield. 

But for this to work, the cameras have to track where your eyes, nose, and head are positioned. Cool features don’t just happen.

So emotional 

Companies like the MIT-spinoff company Affectiva focus on reading driver and passenger emotions to determine how the car should respond. 

At CES, software company Xperi demonstrated it’s developing different interfaces that use facial recognition. It’s building one for the car that knows who you are when you get in, then monitors your emotion state and how you’re behaving. Even your body position is tracked, so if you’re sitting in an ‘unsafe’ way the car can let you know.

Valeo’s head health officer said its Smart Cocoon experience wants to eventually incorporate emotional recognition. Are you sad or nervous? Here’s some perfume spritzes that’ll lift your spirits. Angry? The car will turn on the fan to cool you off. 

Watch out for your car knowing how you feel before you do. And watch your data.