rysana.com
Inversion is a family of structured language models designed to solve speed, reliability, and reasoning issues in traditional AI systems, achieving up to 100× faster speeds and significantly lower latency.
Main Points
Inversion models are highly efficient
Inversion models achieve high speed and reliability in structured tasks with less overhead and latency.
Dynamic acceleration of inference
Inverted inference process leverages compiled structures to dynamically adjust compute needs, leading to acceleration.
Continuous improvement in model performance
New model generations aim for further improvements in latency, reliability, and quality.
Prioritizing developer experience
Developer experience focuses on ensuring outputs always match expected data types.
Advances in processing and input handling
Experiments promise significant advancements in attention processing and input handling.
Insights
Structured inference is fundamentally accelerative and can massively improve both the speed and quality of outputs.
The key insight is that structured inference is fundamentally accelerative - and that if we build models that can always reliably output structured data with constraints, we can massively improve both the speed and quality of the outputs.
Inversion achieves structured output with significantly reduced latency and increased speed.
We set ourselves to the task of taking the quality of outputs from the best available LLMs for workloads like function calling - or actions/workflows and dynamic UI generation - down from around one minute to under 200 milliseconds, which is roughly the time it takes for a human to perceive a response as instant.
The Inversion compiler processes never-before-seen JSON schemas incredibly fast.
The Inversion compiler processes a typical never-before-seen JSON schema in around 400 μs (microseconds) and samples model constraints at runtime in around 20 μs, supporting up to 50,000 tokens per second inference with perfectly structured output.
Inversion models often match or beat all other models tested, even those with many more parameters.
Always-valid outputs are a game changer for structured workloads, dramatically improving the reliability and reasoning level of LLMs across most tasks. Inversion models often match or beat all other models we’ve tested against, even compared to models with around 10× or 100× as many parameters.
Links
URL
https://rysana.com/inversion