Banuba raises $7M to supercharge any app or device with the ability to really see you

Walking into the office of Viktor Prokopenya – which overlooks a central London park – you would perhaps be forgiven for missing the significance of this unassuming location, just south of Victoria Station in London. While giant firms battle globally to make Augmented Reality a ‘real industry’, this jovial businessman form Belrarus is poised to launch a revolutionary new technology for just this space. This is the kind of technology some of the biggest companies in the world are snapping up right now, and yet, scuttling off to make me a coffee in the kitchen is someone who could be sitting on just such a company.

Regardless of whether it’s immediate future is obvious or not, AR has a future if the amount of investment pouring into the space is anything to go by.

In 2016 AR and VR attracted $2.3 billion worth of investments (a 300% jump from 2015) and is expected to reach $108 billion by 2021 – 25% of which will be aimed at the AR sector. But, according to numerous forecasts, AR will overtake VR in 5-10 years.

Apple is clearly making headway in its AR developments, having recently acquired AR lens company Akonia Holographics and in releasing iOS 12 this month, it enables developers to fully utilize ARKit 2, no doubt prompting the release of a new wave of camera-centric apps. This year Sequoia Capital China, SoftBank invested $50M in AR camera app Snow. Samsung recently introduced its version of the AR cloud and a partnership with Wacom that turns Samsung’s S-Pen into an augmented reality magic wand.

The IBM/Unity partnership allows developers to integrate Watson cloud services such as visual recognition, speech to text, and more into their Unity applications.

So there is no question that AR is becoming increasingly important, given the sheer amount of funding and M&A activity.

Joining the field is Prokopenya’s “Banuba” project. For although you can download a Snapchat-like app called ‘Banuba’ from the App Store right now, underlying this is a suite of tools of which Prokopenya is the founding investor, and who is working closely to realize a very big vision with the founding team of AI/AR experts behind it.

The key to Banuba’s pitch is the idea that its technology could equip not only apps but even hardware devices with “vision”. This is a perfect marriage of both AI and AR. What if, for instance, Amazon’s Alexa couldn’t just hear you? What if it could see you and interpret your facial expressions or perhaps even your mood? That’s the tantalizing strategy at the heart of this growing company.

Better known for its consumer apps, which have been effectively testing their concepts in the consumer field for the last year, Banuba is about to move heavily into the world of developer tools with the release of its new Banuba 3.0 mobile SDK. (Available to download now in the App Store for iOS devices and Google Play Store for Android). It’s also now secured a further $7m in funding from Larnabel Ventures, the fund of Russian entrepreneur Said Gutseriev, and Prokopenya’s VP Capital.

This move will take its total funding to $12m. In the world of AR, this is like a Romulan warbird de-cloaking in a scene from Star Trek.

Banuba hopes that its SDK will enable brands and apps to utilise 3D Face AR inside their own apps, meaning users can benefit from cutting-edge face motion tracking, facial analysis, skin smoothing and tone adjustment. Banuba’s SDK also enables app developers to utilise background subtraction, which is similar to ‘green screen’ technology regularly used in films and TV shows, enabling end-users to create a range of AR scenarios. Thus, like magic, you can remove that unsightly office surrounding and place yourself on a beach in the Bahamas…

Because Banuba’s technology equips devices with ‘vision’, meaning they can ‘see’ human faces in 3D and extract meaningful subject analysis based on neural networks, including age, gender, it can do things that other apps just cannot do. It can even monitor your heart rate via spectral analysis of the time-varying color tones in your face.

It has already been incorporated into an app called Facemetrix, which can track a child’s eyes to ascertain whether they are reading something on a phone or tablet or not. Thanks to this technology, it is possible to not just to “track” a person’s gaze, but also to control a smartphone’s function with a gaze. To that end, the SDF can detect micro-movements of the eye with subpixel accuracy in real-time, and also detects certain points of the eye. The idea behind this is to “Gamify education”, rewarding a child with games and entertainment apps if the Facemetrix app has duly checked that they really did read the e-book they told their parents they’d read.

If that makes you think of a parallel with a certain Black Mirror episode where a young girl is prevented from seeing certain things via a brain implant, then you wouldn’t be a million miles away. At least this is a more benign version…

Banuba’s SDK also includes ‘Avatar AR’, empowering developers to get creative with digital communication by giving users the ability to interact with – and create personalized – avatars using any iOS or Android device.

Prokopenya says: “We are in the midst of a critical transformation between our existing smartphones and future of AR devices, such as advanced glasses and lenses. Camera-centric apps have never been more important because of this.” He says that while developers using ARKit and ARCore are able to build experiences primarily for top-of-the-range smartphones, Banuba’s SDK can work on even low-range smartphones.

The SDK will also feature Avatar AR, which allows users to interact with fun avatars or create personalised ones for all iOS and Android devices. Why should users of Apple’s iPhone X be the only people to enjoy Animoji?

Banbua is also likely to take advantage of the news that Facebook recently announced it was testing AR ads in its newsfeed, following trials for businesses to show off products within Messenger.

Banuna’s technology won’t simply be for fun apps however. Inside 2 years, the company has filed 25 patent applications with the the US patent office and of six of those were processed in record time compared with the average. Its R&D center, staffed by 50 people and based in Minsk, is focused on developing a portfolio of technologies.

Interestingly, Belarus has become famous for AI and facial recognition technologies.

For instance, cast your mind back to early 2016, when Facebook bought Masquerade, a Minsk-based developer of a video filter app, MSQRD, which at one point was one of the most popular apps in the App Store. And in 2017, another Belarusian company, AIMatter, was acquired by Google, only months after raising $2M. It too took an SDK approach, releasing a platform for real-time photo and video editing on mobile, dubbed Fabby. This was built upon a neural network-based AI platform. But Prokopenya has much bolder plans for Banuba.

In early 2017, he and Banuba launched a “technology-for-equity” program to enroll app developers and publishers across the world. This signed up Inventain, another startup from Belarus, to develop AR-based mobile games.

Prokopenya says the technologies associated with AR will be “leveraged by virtually every kind of app. Any app can recognize its user through the camera: male or female, age, ethnicity, level of stress, etc.” He says the app could then respond to the user in any number of ways. Literally, your apps could be watching you.

So for instance, a fitness app could see how much weight you’d lost just by using the Banuba SDF to look at your face. Games apps could personalize the game based on what it knows about your face, such as reading your facial cues.

Back in his London office, overlooking a small park, Prokopenya waxes lyrical about the “incredible concentration of diversity, energy and opportunity” of London. “Living in London is fantastic,” he says. “The only thing I am upset about, however, is the uncertainty surrounding Brexit and what it might mean for business in the UK in the future.”

London may be great (and will always be), but sitting on his desk though is a laptop with direct links back to Minsk, a place where the facial recognition technologies of the future are only now just emerging.