Another Startup Cashes in Millions of Dollars To Make More Sure AI Models and ChatGPT

Thanks to the popularity of ChatGPT, companies everywhere are injecting generative AI into their products. But Vikram Chatterji thinks there’s one big thing stopping businesses from fully embracing AI, and deploying tools in full-scale production: Companies don’t really know how well the tools are going to work once they’re released.

“If you don't solve for this, you’re stuck in almost-production land”, said Chatterji, CEO of Galileo, a start-up focused on evaluating AI models.

Chatterji wants his company to be the one to solve it. Galileo works with customers including Hewlett Packard, Comcast and Twilio to test their AI tools, making sure they are not hallucinating – AI-speak for just making up responses – or giving out private data. On Tuesday, the company told Forbes it has raised a $ 45 million Series B funding round, bringing its total investment to $ 68 million since it was founded three years ago.

The start-up has four products: Fine Tune, which helps corporate customers to train existing models to fit their needs, like incorporating proprietary data; Evaluate, which lets companies quickly decide if a model is ready to ship or still has kinks to work out; Observe, for maintenance and real-time testing once the model is launched; and Protect, to mitigate threats like prompt injection, were users try to manipulate the model to go off the rails or disclose sensitive information. In July, Galileo released Luna, its own language model for evaluating the performance of other models, including answer completeness, data privacy and bias detection.

The announcement comes as companies worldwide race to beef up their AI offerings. But that comes with the pitfalls of AI models making silly mistakes – like Google’s Gemini telling users to make a glue pizza – or sensitive leaks – like Samsung’s internal source code being shared on ChatGPT last year, causing the electronics giant to ban employees from using the chatbot. Galileo isn’t the only AI evaluations start-up in the market. Last week, rival Braintrust, whose customers include Stripe and Notion, said it raised a $ 36 million Series A led by Andreessen Horowitz.

Chatterji and fellow co-founder Yash Sheth are former Google colleagues who joined the tech giant a week apart from each other in 2013. Chatterji was an AI product manager who most recently worked on Google’s BERT, an AI model deployed to help understand a user’s intentions while using its search engine. Sheth, now Galileo’s COO, was an engineer working on Google’s speech recognition tools. Atindriyo Sanyal, Galileo’s CTO and other cofounder, previously held senior AI engineering roles at Uber and Apple.

Galileo’s round was led by Scale Venture Partners, with participation by Premji Invest, an Indian venture capital firm. Other backers include the venture arms of Citibank and the data storage company Databricks, as well as Clem Delangue, founder of AI startup Hugging Face.

Andy Vitus, a partner at Scale Venture Partners, said evaluation tools are so important for corporations because he thinks AI models will soon be seen as brand ambassadors — especially as businesses start to deploy AI agents that perform tasks for customers, like booking travel or placing orders. Companies also need to protect their AI models from saying anything negative about their brands, or referencing past scandals.

“In the long run, there will be an expectation that, say, Siri, doesn't say bad things about Apple”, said Vitus, who is taking a board seat at Galileo as part of the investment. “In the same way employees have managers, these agents need oversight.”

Jim Nottingham, senior vice president of advanced compute solutions at HP, said the company uses Galileo’s services internally for its own AI tools, but also bundles it for customers who use HP’s AI Studio, a subscription suite of tools that helps developers create AI products.

The $ 45 million funding round is modest compared to some of the eye-popping rounds other AI companies have recently announced. Earlier this month, OpenAI said it raised $ 6.6 billion, the largest venture capital round of all time. Elon Musk’s xAI is not far behind with a $ 6 billion raise earlier this year. Even lesser known companies, like next-gen chip startup Groq, raised $ 640 million this summer.

But Galileo isn’t training massive frontier models that need armies of GPUs (the company didn’t train or fine tune Luna from scratch), or building out new chip designs with expensive manufacturing costs. As a result, Chatterji said the company doesn’t require nearly as much cash to operate. “You should just take the amount of money that you actually need”, he said. “And build a business which is sustainable for the long term.”

(Based on a publication of Forbes.)