Liquid AI says it has built upon this work and invented new models that it is keeping secret for now. This September, the company revealed some large language models based on its network design. A version of its language model with 40 billion parameters outperformed the 70-billion-parameter version of Meta’s Llama 3.1 on a common set of problems known as MMLU-Pro, the startup says.
“The benchmark results for their SLMs look very promising,” says Sébastien Bubeck, a researcher at OpenAI who explores how AI models’ architecture and training affect their capabilities.
“Finding a new type of foundation model doesn’t happen every day,” says Tom Preston-Werner, a cofounder of GitHub and an early investor in Liquid AI, who notes that the transformer models that underpin large language models and other AI systems are starting to show their limitations. Preston-Werner adds that making AI more efficient should be a big priority for everyone. “We should do everything we can to make sure we aren’t running coal plants for longer,” he says.
One drawback with Liquid AI’s approach is that its networks are especially suited to certain tasks, particularly ones that involve temporal data. Making the technology work with other types of data requires custom code. And of course, another challenge will be persuading big companies to base important projects on a completely new AI design.
Hasani says the goal now is to demonstrate that the benefits—including efficiency, transparency, and energy costs—outweigh the challenges. “We are getting into stages where these models can alleviate a lot of the socio-technical challenges of AI systems,” he says.
