Guide

Decision models: fast, typed answers for routine agent choices

By Atlas AI Resources · Published October 6, 2026 · How we make these guides

A lot of what an agent does all day is small sorting: which team gets this ticket, is this source relevant, how urgent is this email. A new kind of model answers questions like these with a pick and a probability instead of a paragraph. This guide explains what these decision models are, where one can sit next to your bots, what the open-source Laya models offer, and how to try one without handing it control too early.

1. What a decision model is

In September 2026 TypeSafe AI released Jev, which it calls a “System One” model. Simon Willison’s explainer suggests “decision models” as the clearer name, and that’s the term this guide uses. You give the model a piece of text or a small record (the “state”) plus one or more questions, and it returns numbers instead of writing. There are three kinds of question:

• Yes or no. You state something, such as “the customer threatens to cancel,” and get back how likely it is to be true.
• Pick one. You list options, such as billing, technical, or other, and get the chosen option plus a probability for each.
• Score. You describe a scale, such as not urgent, soon, or blocking, and get a position on it.

Because nothing is written out, these models are fast and cheap enough to ask many questions at once. Open alternatives followed within days. Laya, from Convai Innovations, is one of them, released on Hugging Face under the Apache 2.0 license, so you can run it yourself.

2. Where one fits next to your bots

A useful way to split the work: a language model does the work that needs writing, research, or planning; a decision model answers the small, repeated questions about what happens next; and plain rules handle anything with exact limits, such as spending, sending, or deleting.

• Good fits. The same few questions asked many times: sorting a support queue, flagging likely spam, labeling incoming mail, or scoring a long list of search results for relevance before a bot reads the best ones.
• Poor fits. Anything that needs an explanation, a draft, or a judgment call you’d want to read the reasoning for. A decision model gives you a number and nothing else.
• Often unnecessary. If a bot makes a handful of these calls a day, just let it decide. A decision model earns its keep when the volume is high enough that speed and cost per finished job actually matter.

For Grok Bot users, this is something you’d use inside your own scripts or tools, for example a small sorting step your bot runs on its computer before it reads the items that matter. It doesn’t change how Grok Bot works. Grok Bot’s own guardrails are rules you write in plain language, checked by a separate review agent, and SpaceXSI’s guides recommend a single front-door coordinator bot once handoffs between bots are real. A decision model can be one more tool that setup calls on for repetitive sorting.

3. What the Laya models offer

These details come from the Laya project’s own model cards and docs.

• Three checkpoints. An English model and a multilingual one for 100+ languages, both general, plus a version fine-tuned on four workflows: agent-trace monitoring, customer service, invoice processing, and security incidents. A built-in router picks the English or multilingual model by the language of the text.
• Easy to call. It installs as a Python package. You can describe the answer you want as a JSON schema or a Pydantic model and get typed values back. There are optional extras for an HTTP server and an MCP server.
• Fine-tuning matters. On the project’s typed-decisions benchmark, the fine-tuned model scored 0.766 accuracy, while the base English model scored 0.362, below the 0.461 you’d get by always guessing the most common answer. The docs say this supports fine-tuning for a similar task, not that the base models will do well on yours. The project includes a notebook that fine-tunes on free Kaggle GPUs in about four to five hours.
• Comparisons are indicative. The model card lists Jev’s published numbers alongside its own, and says plainly that they weren’t measured in the same run, with different prompts and sample sizes.

4. Limits the authors list themselves

Laya’s docs are unusually candid about where it breaks. Check each of these on your own data before relying on it.

• It’s over-confident as shipped. Its probabilities don’t yet match how often it’s right, so the authors say to refit them on held-out examples from your own work before using a confidence cutoff.
• The fine-tuned model is a specialist. Outside its four workflows, expect it to do no better than the base model, or worse.
• Keep option lists short. Accuracy falls off sharply past about 20 options in a single pick-one question.
• Wording and order matter. Changing the order of options can change the answer, and negated or yes/no-style labels have failed in documented examples.
• Long text gets shakier. Answers become less reliable beyond about 4,000 tokens of text.
• Language routing can fool it. The English model can be confident about non-English text it handles badly, which is why the router exists.

Simon Willison adds a broader caution: a decision model is even more of a black box than a chatbot. You can’t ask it why it flagged something, so hidden bias is hard to spot, and he warns against using one for anything like ranking job applicants. An answer that fits your schema perfectly can still be wrong, so testing on real examples matters more here, not less. The upside is that these models are cheap enough to test thoroughly.

5. Rolling one out safely

Laya’s staged-adoption guide opens with a line worth keeping: it “returns a typed decision, not permission to execute it.” Its rollout steps work for any decision model:

• Shadow first. Run it on real work alongside whatever decides today, but let the current process stay in charge. Log both answers and what the right answer turned out to be.
• Compare carefully. A disagreement isn’t automatically the model’s mistake. Keep an “unclear” bucket for cases that need a person.
• Pick a cutoff from evidence. A confidence threshold is your policy, not something the model supplies. Choose it from measured accuracy on held-out examples and the cost of being wrong. High confidence is never permission by itself.
• Promote one small slice. Start with reversible, low-stakes decisions and a cap on volume. Send unclear, failed, and high-stakes cases to a fallback or a human.
• Keep watching and be ready to roll back. Keep sampling promoted decisions and switch back when they fall below the bar you agreed on in advance.

Prompt version for any chat model: Yes, no, or not sure: prompts for strict answers that route to you.

Useful public resources

Related: Decision forks · Agent memory for AI fleets · Small AI fleets: handoffs · AI News: Cloudflare’s Clef decision models