Decision models cut AI latency and cost by restricting outputs
New decision models like Jev offer millisecond responses at a fraction of LLM costs by choosing from predefined options, reducing hallucinations but limiting complex reasoning.
In September 2026, startup TypeSafe AI launched Jev, a new class of artificial intelligence model designed to make rapid decisions without the high computational cost or hallucination risks associated with traditional large language models. This release triggered an immediate industry shift, with major tech companies including Cloudflare, Amazon, OpenAI, and Microsoft quickly releasing their own versions of these "decision models" to handle high-volume, low-latency tasks.
What happened
TypeSafe AI positioned Jev as a "System One model," referencing psychologist Daniel Kahneman’s theory of fast, instinctive thinking versus slow, deliberate reasoning. The company claimed Jev could deliver end-to-end response latencies between 70 and 500 milliseconds, significantly faster than the several seconds typically required by standard LLMs. Independent testing supported these claims, with a study co-authored by Carnegie Mellon University professor Ramayya Krishnan finding that Jev achieved a median latency of 150 milliseconds across various benchmarks. In contrast, OpenAI’s GPT-6 Astra model recorded a median latency of 1.89 seconds during the same period.
The financial implications are equally stark. TypeSafe AI charges $0.042 per million input tokens for Jev, with output tokens provided free of charge. Comparatively, GPT-6 Astra costs $10 per million input tokens and $50 per million output tokens. Even the cheaper, speed-optimized GPT-6 Luna variant costs $0.10 per million input tokens and $0.50 per million output tokens. This drastic reduction in cost and time has attracted significant attention, leading TypeSafe AI to raise $870 million at a $7.5 billion valuation. The model’s success has also spurred a wave of imitation, with independent developers and established firms releasing dozens of similar models within weeks of Jev’s debut.
How it works
Decision models differ fundamentally from traditional LLMs in their output generation process. While an LLM predicts the next token in a sequence repeatedly until a complete response is formed, a decision model performs a single forward pass. It reads all input tokens and generates a single output choice from a predefined set of options, such as A, B, or C. This architectural constraint eliminates the sequential token generation that causes high latency and opens the door for factual errors, or "hallucinations," in LLMs.
These models retain the transformer-based architecture and general intelligence of LLMs, allowing them to generalize across tasks without the domain-specific fine-tuning required by older classifier models. However, they are restricted to structured answer formats. By calculating probabilities only among pre-determined possibilities, decision models prevent the generation of unexpected or fabricated text. This makes them highly effective for tasks like content moderation, spam detection, and routing, where the goal is to categorize data rather than generate creative text.
Key details
- Jev achieves median latency of 150 milliseconds, compared to 1.89 seconds for GPT-6 Astra in independent tests.
- Input token pricing for Jev is $0.042 per million, with free output tokens, vastly undercutting competitors.
- Decision models use a single forward pass to select from predefined options, avoiding sequential token generation.
- Major providers including Cloudflare, Amazon, and Microsoft have released their own decision models following Jev’s launch.
- Performance drops in domains requiring complex reasoning, such as medicine, finance, and law, compared to LLMs.
- Hybrid architectures combining decision models for quick judgments and LLMs for detailed explanations show promise.
Why it matters
For software engineers and technical leads, decision models offer a practical solution for high-volume processing tasks that do not require open-ended generation. Tasks such as moderating social media posts, filtering spam emails, or detecting prompt-injection attacks can now be handled with greater speed and lower cost. Michelle Chen, a group product manager at Cloudflare, suggests that up to 80 percent of typical LLM tasks could be offloaded to these faster, cheaper System One models. This shift allows teams to deploy AI capabilities without maintaining large machine learning teams for custom classifier training.
However, the technology is not a universal replacement for LLMs. Decision models struggle with tasks requiring deep topic expertise or complex logical reasoning. Researchers note that while these models eliminate output hallucinations, they do not eliminate decision-making errors, particularly in math or logic puzzles. Consequently, the most effective implementation may involve hybrid systems where decision models handle initial triage or simple classifications, passing only complex or low-confidence cases to more capable, albeit slower and more expensive, LLMs.
What you can do
- Identify high-volume, low-complexity tasks in your pipeline, such as document classification or spam filtering, for potential migration to decision models.
- Evaluate open-weight decision models like Cloudflare’s Clef or SII-Decision to test performance on your specific data sets without vendor lock-in.
- Implement hybrid architectures where decision models perform initial triage, routing only ambiguous or complex queries to full LLMs.
- Benchmark latency and cost savings against your current LLM usage to quantify the potential operational efficiency gains.
- Review predefined option sets for your use cases to ensure they cover all necessary outcomes without being overly restrictive.
- Monitor accuracy in domain-specific tasks, recognizing that decision models may underperform in areas requiring specialized reasoning.



