EnvLoop LabsLog inEnvLoop / Use cases / Decision models
A decision model answers a typed question about your users or your business, such as whether an action needs approval, with a calibrated probability over a fixed set of options, read from a single forward pass. TypeSafe’s Jev is one example of this class of model. With EnvLoop, any company can post-train its own decision model on its own data, starting from the recipe behind Pev-27B, the decision model that EnvLoop Research trained.
01 / Definition
Personal agents make many small judgements for every request: which option the user would pick, whether an action needs approval and what may be shared. A decision model treats each judgement as a typed question family with fixed options. It reads the context once and returns a probability for every option, so the agent can act on the answer or ask the user.
Each question belongs to a family with a fixed set of options, such as notify levels or approval decisions, so every answer can be checked and compared.
Pev-27B is supervised on a single option-label token, so each decision is read from one forward pass instead of generated text.
Temperature calibration turns the option scores into probabilities that track how often the model is right, measured by the Brier score and expected calibration error (ECE).
02 / Calibration
An agent that knows how reliable each answer is can act on confident decisions and ask the user about uncertain ones. In the Pev study’s pre-registered one-shot hidden gate, family-macro accuracy rose from 0.754 to 0.905, safety false negatives fell, and the share of questions the model can answer alone within a 5% error budget rose from 0.54 to 0.93.
03 / Results
Pev-Bench asks seven kinds of personal-agent decisions about users who are not in the training data. On its public TEST set of 720 new users, the released Pev-27B has the highest family-macro accuracy and the lowest Brier score (0.125) of the evaluated models, which include the third-party decision models Jev from TypeSafe and Kev-27B. Lower Brier and ECE values are better.
Swipe horizontally to compare all scores.
| Rank | Model | Family-macro accuracy ↑ | Brier ↓ | ECE ↓ |
|---|---|---|---|---|
| 1 | Pev-27BFine-tuned (LoRA on Qwen3.8-27B) | 0.915 | 0.125 | 0.065 |
| 2 | gpt-6-astraAPI model | 0.873 | 0.180 | 0.074 |
| 3 | Kev-27BThird-party decision model | 0.823 | 0.218 | 0.074 |
| 4 | JevThird-party decision model (TypeSafe) | 0.788 | 0.266 | 0.123 |
| 5 | Qwen3.8-27BOpen-weight base | 0.762 | 0.264 | 0.067 |
| 6 | Qwen3.5-4BOpen-weight base | 0.652 | 0.389 | 0.074 |
04 / On EnvLoop
Training your own decision model on EnvLoop uses the experimental Pev-27B label-token recipe on Qwen/Qwen3.8-27B, arranged with our team. Its presets include Pev’s released run and a financial-decision preset on the public financial-decision-bench dataset. Results depend on your data: the scores above belong to the released Pev-27B, and a model you train is measured on your own held-out data.
Contact our team to enable the experimental Pev-27B label-token recipe for your project and plan a trial run.
An administrator imports your prepared train and validation files into the project as a training-data snapshot.
Submit a request with the pev27 recipe on Qwen/Qwen3.8-27B through el job submit --type train.
Training returns a checkpoint whose calibration is not yet fitted. Fit calibration, evaluate decision quality on held-out data and agree export and serving with our team before you deploy it.
05 / Questions
Yes. EnvLoop post-trains decision models on a company’s own data with the experimental Pev-27B label-token recipe on Qwen/Qwen3.8-27B, which our team enables for your project. EnvLoop’s released Pev-27B shows the approach: 0.915 family-macro accuracy on the Pev-Bench public test set, against 0.788 for TypeSafe’s Jev. A model trained on your data is measured on your own held-out set.
No. Jev is TypeSafe’s hosted decision model, and Kev-27B is a third-party model as well; both appear on Pev-Bench as third-party baselines. EnvLoop’s own decision model is Pev-27B.
A chat model generates free text, and the answer has to be parsed from it. A decision model returns a probability for each option of a typed question from one forward pass, so an agent can compare the answer with a threshold and decide whether to act or to ask the user.
Pev-Bench scores family-macro accuracy, the unweighted mean accuracy over seven decision families, on users who are not in the training data. Its TEST set has two halves rendered by different language models, and a network-free Harbor verifier scores every answer.