Arthur announces $3.3M seed to monitor machine learning model performance

Machine learning is a complex process. You build a model, test it in laboratory conditions, then put it out in the world. After that, how do you monitor how well it’s tracking what you designed it do? Arthur wants to help, and today it emerged from stealth with a new platform to help you monitor machine learning models in production.

The company also announced it had closed a $3.3 million seed round, which closed in August.

Arthur CEO and co-founder Adam Wenchel says that Arthur is analogous to a performance monitoring platform like New Relic or DataDog, but instead of monitoring your systems, it’s tracking the performance of your machine learning models.

“We are an AI monitoring and explainability company, which means when you put your models in production, we let you monitor them to know that they’re not going off the rails, that you can explain what they’re doing, that they’re not performing badly and are not being totally biassed — all of the ways models can go wrong,” Wenchel explained.

Data scientists build machine learning models and test them in the lab, but as Wenchel says, when that model leaves the controlled environment of the lab, lots can go wrong, and it’s hard to keep track of that. “Models always perform well in the lab, but then you put them out in the real world and there is often a drop-off in performance — in fact, almost always. So being able to measure and monitor that is a capability people really need,” he said.

Interestingly enough, AWS announced a new model monitoring tool last week as part of SageMaker Studio. IBM also announced a similar tool for models built on the Watson platform earlier this year, but Wenchel says the involvement of the big guys could work to his company’s advantage since his product is platform-agnostic. “Having a neutral third party for your monitoring that works equally well across stacks is going to be pretty valuable,” he said.

As for the funding, it was co-led by Work-Bench and Index Ventures with participation from Hunter Walk at Homebrew, Jerry Yang at AME Ventures and others.

Jonathan Lehr, a general partner at Work-Bench sees a company with a lot of potential. “We regularly speak with ML executives from Fortune 1000 companies and one of their biggest concerns as they become more data-driven is model behavior in production. The Arthur platform is by far the best solution we’ve seen for AI monitoring and transparency…” he said.

The company, which is based in New York City, currently has 10 people. It launched 2018, and has been heads down working on the product since. Today, marks the release of the product publicly.