Be a part of our each day and weekly newsletters for the newest updates and unique content material on industry-leading AI protection. Study Extra
Liquid AI, a startup co-founded by former researchers from the Massachusetts Institute of Expertise (MIT)’s Laptop Science and Synthetic Intelligence Laboratory (CSAIL), has introduced the debut of its first multimodal AI fashions.
In contrast to most others of the present generative AI wave, these fashions usually are not based mostly across the transformer structure outlined within the seminal 2017 paper “Attention Is All You Need.”
As a substitute, Liquid states that its aim “is to explore ways to build foundation models beyond Generative Pre-trained Transformers (GPTs)” and with the brand new LFMs, particularly constructing from “first principles…the same way engineers built engines, cars, and airplanes.”
It appears they’ve executed simply that — as the brand new LFM fashions already boast superior efficiency to different transformer-based ones of comparable dimension resembling Meta’s Llama 3.1-8B and Microsoft’s Phi-3.5 3.8B.
Generally known as the “Liquid Foundation Models (LFMs),” these fashions at the moment are available three totally different sizes and variants:
- LFM 1.3B (smallest)
- LFM 3B
- LFM 40B MoE (largest, a “Mixture-of-Experts” mannequin just like Mistral’s Mixtral)
The “B” of their identify stands for billion and refers the variety of parameters — or settings — that govern the mannequin’s data processing, evaluation, and output era. Usually, fashions with the next variety of parameters are extra succesful throughout a wider vary of duties.
Already, Liquid AI says the LFM 1.3B model outperforms Meta’s new Llama 3.2-1.2B and Microsoft’s Phi-1.5 on many main third-party benchmarks together with the favored Large Multitask Language Understanding (MMLU) consisting of 57 issues throughout science, tech, engineering and math (STEM) fields, “the first time a non-GPT architecture significantly outperforms transformer-based models.”
All three are designed to supply state-of-the-art efficiency whereas optimizing for reminiscence effectivity, with Liquid’s LFM-3B requiring solely 16 GB of reminiscence in comparison with the greater than 48 GB required by Meta’s Llama-3.2-3B mannequin (proven within the chart above).
Maxime Labonne, Head of Submit-Coaching at Liquid AI, took to his account on X to say the LFMs had been “the proudest release of my career :)” and to make clear that the core benefit of LFMs: their potential to outperform transformer-based fashions whereas utilizing considerably much less reminiscence.
The fashions are engineered to be aggressive not solely on uncooked efficiency benchmarks but in addition when it comes to operational effectivity, making them very best for quite a lot of use instances, from enterprise-level purposes particularly within the fields of monetary providers, biotechnology, and client electronics, to deployment on edge gadgets.
Nevertheless, importantly for potential customers and prospects, the fashions usually are not open supply. As a substitute, customers might want to entry them by way of Liquid’s inference playground, Lambda Chat, or Perplexity AI.
How Liquid goes ‘beyond’ the generative pre-trained transformer (GPT)
On this case, Liquid says it used a mix of “computational units deeply rooted in the theory of dynamical systems, signal processing, and numerical linear algebra,” and that the result’s “general-purpose AI models that can be used to model any kind of sequential data, including video, audio, text, time series, and signals” to coach its new LFMs.
Final 12 months, VentureBeat lined extra about Liquid’s method to coaching post-transformer AI fashions, noting on the time that it was utilizing Liquid Neural Networks (LNNs), an structure developer at CSAIL that seeks to make the unreal “neurons” or nodes for reworking knowledge, extra environment friendly and adaptable.
In contrast to conventional deep studying fashions, which require 1000’s of neurons to carry out advanced duties, LNNs demonstrated that fewer neurons—mixed with progressive mathematical formulations—might obtain the identical outcomes.
Liquid AI’s new fashions retain the core advantages of this adaptability, permitting for real-time changes throughout inference with out the computational overhead related to conventional fashions, dealing with as much as 1 million tokens effectively, whereas preserving reminiscence utilization to a minimal.
A chart from the Liquid weblog reveals that the LFM-3B mannequin, for example, outperforms well-liked fashions like Google’s Gemma-2, Microsoft’s Phi-3, and Meta’s Llama-3.2 when it comes to inference reminiscence footprint, particularly as token size scales.
Whereas different fashions expertise a pointy enhance in reminiscence utilization for long-context processing, LFM-3B maintains a considerably smaller footprint, making it extremely appropriate for purposes requiring massive volumes of sequential knowledge processing, resembling doc evaluation or chatbots.
Liquid AI has constructed its basis fashions to be versatile throughout a number of knowledge modalities, together with audio, video, and textual content.
With this multimodal functionality, Liquid goals to handle a variety of industry-specific challenges, from monetary providers to biotechnology and client electronics.
Accepting invites for launch occasion and eyeing future enhancements
Liquid AI says it’s is optimizing its fashions for deployment on {hardware} from NVIDIA, AMD, Apple, Qualcomm, and Cerebras.
Whereas the fashions are nonetheless within the preview section, Liquid AI invitations early adopters and builders to check the fashions and supply suggestions.
Labonne famous that whereas issues are “not perfect,” the suggestions acquired throughout this section will assist the group refine their choices in preparation for a full launch occasion on October 23, 2024, at MIT’s Kresge Auditorium in Cambridge, MA. The corporate is accepting RSVPs for attendees of that occasion in-person right here.
As a part of its dedication to transparency and scientific progress, Liquid says it’s going to launch a collection of technical weblog posts main as much as the product launch occasion.
The corporate additionally plans to interact in red-teaming efforts, encouraging customers to check the bounds of their fashions to enhance future iterations.
With the introduction of Liquid Basis Fashions, Liquid AI is positioning itself as a key participant within the basis mannequin house. By combining state-of-the-art efficiency with unprecedented reminiscence effectivity, LFMs supply a compelling different to conventional transformer-based fashions.
