Google Assistant is still a lot smarter than Siri, new study shows

In the battle of digital assistants, a winner is emerging: Google Assistant is smartest of them all, followed by Siri, Cortana and Alexa. 

That’s according to a new study by LoupVentures, whose team asked the four digital assistants 800 questions each. All of them have shown big improvements since LoupVentures’ last test, conducted in April — with Siri making the biggest leap forward — but they’re still far from perfect. 

All four assistants were great at understanding queries — Google Assistant had a 100% success rate, Apple’s Siri understood 99% of queries, and Microsoft’s Cortana and Amazon’s Alexa were only slightly behind with a 98% success rate.

Answering queries correctly was another matter: Google Assistant was able to do so in 85.5% of cases, which is… well, acceptable. Siri answered correctly in 78.5% of cases, which isn’t great but it’s a lot better than its April 2017 result of 66.1%. Cortana and Alexa were both pretty bad: Alexa answered 61.4% of queries correctly, while Cortana only managed 52.4% (although even this is an improvement from last year’s 48.8%). 

The study noted that “nearly every misunderstood question involved a proper noun, often the name of a local town or restaurant.” As a personal aside, as someone who uses both Siri and Google Assistant in an area where English is not the primary language, I can say from experience that the results I get are far, far worse than the above, precisely due to the assistants not understanding a local name. The study obviously did not focus on non-English speaking areas but it’s definitely an area that can be improved across the board.

The questions LoupVentures used for its study were broken in 5 categories — Local, Commerce, Navigation, Information, and Command – with Google Assistant winning out in all but one: Command.

“We found Siri to be slightly more helpful and versatile (responding to more flexible language) in controlling your phone, smart home, music, etc,” the study’s authors said. “Our question set also includes a fair amount of music-related queries (the most common action for smart speakers). Apple, true to its roots, has ensured that Siri is capable with music on both mobile devices and smart speakers,” 

While Google Assistant was dominant in most categories, the table above shows that Siri’s only significantly lagging when it comes to finding stuff (Information) and buying stuff (Commerce). Interestingly, in LoupVentures’ testing, Alexa scored abysmally on Commerce, as it forced “Amazon’s Choice” answers instead of actually answering the query. And in both the “Local” and “Navigation” categories, Google Assistant and Siri were clear winners over Alexa and Cortana, thanks to maps integration on their respective platforms.

Overall, the future in which you can just casually chat with your digital assistant and have it answer everything perfectly is still pretty far away, but all tested assistants have shown huge improvements in a little over a year, which is quite promising. The authors conclude: “With scores nearing 80-90%, it begs the question, will these assistants eventually be able to answer everything you ask? The answer is probably not, but continued improvement will come from allowing more and more functions to be controlled by your voice.”