211 lines
10 KiB
Markdown
211 lines
10 KiB
Markdown
# Jev: what it does and how it works
|
|
|
|
Research summary of TypeSafe AI's Jev, launched 15 September 2026. Written so an
|
|
undergraduate with one AI course behind them can follow it. Sources at the
|
|
bottom; claims are tagged confirmed, self-reported, or speculative.
|
|
|
|
## TL;DR
|
|
|
|
Jev is not a chatbot and not a smaller LLM. It is a **decision model**: you send
|
|
it a piece of state (text or JSON) plus a set of pre-defined questions, and it
|
|
returns typed answers — a category, a number on a scale, or a probability — each
|
|
with calibrated probabilities. It never generates text. That single design
|
|
choice makes it ~100x faster and ~100-400x cheaper than a frontier LLM on these
|
|
narrow questions, and makes malformed answers structurally impossible.
|
|
|
|
## The problem it solves
|
|
|
|
A normal LLM returns **strings**. To use a string inside software you must
|
|
parse it, validate it, and hope the model did not go off the rails. Three costs
|
|
follow:
|
|
|
|
1. **Latency.** LLMs generate one token at a time, each conditioned on the
|
|
last. A 500-token answer is ~500 sequential forward passes. Frontier
|
|
responses take 3s to 329s end to end.
|
|
2. **Cost.** Output tokens cost ~5x input tokens because each one is another
|
|
decode pass.
|
|
3. **Type errors and hallucination.** No matter how smart the model, the output
|
|
space is "any string", so it can emit an unexpected value, malformed JSON,
|
|
or a made-up fact. When the consumer is software — not a human reading it —
|
|
that is a deal-breaker, especially deep in a dependency chain.
|
|
|
|
TypeSafe's thesis: most *machine-consumed* intelligence is not "write me an
|
|
essay". It is small judgments — which bucket, how severe, is this true — that
|
|
software could consume directly if the model were shaped for it. Kahneman's
|
|
System 1 / System 2 distinction is the branding: Jev is fast intuition, LLMs
|
|
are slow deliberation.
|
|
|
|
## The interface
|
|
|
|
```
|
|
state + typed questions → typed probabilistic answers
|
|
```
|
|
|
|
One endpoint (`POST https://api.typesafe.ai/v1/systemone`), SDKs in Python
|
|
(`typesafe-sdk`) and TypeScript (`@typesafe-ai/sdk`). `state` is a string, JSON
|
|
object, or array. Limits: 64k tokens for state + all questions, text only (no
|
|
images).
|
|
|
|
### Example
|
|
|
|
```python
|
|
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient
|
|
|
|
client = TypeSafeClient()
|
|
|
|
response = client.system_one(
|
|
state={
|
|
"ticket": {"subject": "Duplicate charge",
|
|
"messages": [{"from": "customer",
|
|
"text": "I was charged twice for order A-104."}]},
|
|
"refund_policy": "Duplicate charges are eligible for a refund.",
|
|
},
|
|
questions={
|
|
"department": Choice(instructions="Which team should handle this",
|
|
criteria={"billing": "Payment or subscription issues",
|
|
"technical": "Bugs or integration problems",
|
|
"sales": "Pricing or account questions"}),
|
|
"frustration": Score(instructions="How frustrated the customer appears",
|
|
criteria=["Calm, just stating facts",
|
|
"Frustrated but civil",
|
|
"Very angry, strong language"]),
|
|
"refund_wanted": Noul(instructions="The customer explicitly asks for a refund"),
|
|
"policy_supports": Noul(instructions="The stated refund policy covers this"),
|
|
},
|
|
)
|
|
print(response.answers["department"].choice, response.answers["department"].confidence)
|
|
```
|
|
|
|
One call returns every answer in 70-500 ms.
|
|
|
|
## The three primitives
|
|
|
|
| Primitive | Question it answers | Returns |
|
|
| --- | --- | --- |
|
|
| **Noul** | "Is this true?" | a 0-1 probability. No separate confidence field — the number *is* the belief. |
|
|
| **Choice** | "Which of these?" | winning option + probability for every option (up to 255) + confidence |
|
|
| **Score** | "Where on this scale?" | a value that can land *between* levels (e.g. 1.35) + per-level probabilities + confidence |
|
|
|
|
Design rules from the docs, worth internalizing:
|
|
|
|
- Define Choice criteria as concrete situations, and always include an `other`
|
|
option so the model can say "nothing fits" instead of picking the closest
|
|
wrong thing.
|
|
- Score levels should be concrete situations ("service down, no workaround"),
|
|
not vague adjectives ("high").
|
|
- Questions in one call are evaluated **independently** against the same state.
|
|
A tenth question costs a few tokens and almost no latency, so the winning
|
|
pattern is *speculative fan-out*: ask everything, let code ignore what does
|
|
not apply. If question B truly depends on question A's answer, make a second
|
|
call with updated state.
|
|
|
|
## How it works
|
|
|
|
### Why one forward pass is possible
|
|
|
|
An LLM must decode sequentially because its answer space is effectively
|
|
unbounded (any string). Jev's answer space is **fixed and enumerated in
|
|
advance** — 255 options, a scale, or a boolean. A bounded answer space can be
|
|
*scored* rather than generated: one forward pass computes a probability for
|
|
every possible answer, no decode loop afterward.
|
|
|
|
That one fact explains three marketing claims at once:
|
|
|
|
- **Speed** (70-500 ms): one pass instead of hundreds. This is different from
|
|
Transformer training parallelism — LLM *inference* was always sequential,
|
|
and Jev parallelizes inference-time decision computation for bounded
|
|
outputs only.
|
|
- **Free output tokens** ($0.042/MTok input, $0 output): there is nothing
|
|
after the single pass to meter.
|
|
- **"Cannot hallucinate"**: outputs are members of a predefined set by
|
|
construction; a malformed string was never a possible output. Careful: this
|
|
means *no schema errors*, not *always correct*. Jev can confidently pick the
|
|
wrong category. "Calibration" and "correctness" are different properties.
|
|
|
|
What the architecture actually is — encoder-only transformer with
|
|
classification heads, diffusion variant, something purpose-built — is
|
|
**undisclosed**. Treat all such explanations as informed speculation.
|
|
|
|
### RLCD: training for honesty about uncertainty
|
|
|
|
TypeSafe's training method, RLCD (Reinforcement Learning for Calibrated
|
|
Decisions), sits beside its two better-known cousins:
|
|
|
|
| Method | Reward signal | Used for |
|
|
| --- | --- | --- |
|
|
| RLHF | human raters prefer this output | chat models (the lineage Jev's founder, Diogo Almeida, worked on at OpenAI) |
|
|
| RLVR | output passes a programmatic check | math/code reasoning |
|
|
| **RLCD** | stated confidence matches actual accuracy | Jev |
|
|
|
|
Calibration, concretely: a weather forecaster who says "70% chance of rain" on
|
|
100 days should get rain on roughly 70 of them. RLCD rewards the *match*
|
|
between confidence and outcomes, not persuasiveness. The payoff for software:
|
|
you can branch on the number. Act automatically at 0.95, ask a human at 0.55 —
|
|
but only if the number is honest *in aggregate*.
|
|
|
|
Two honest caveats. First, calibration is a distribution property: an
|
|
individual 0.9 answer can still be wrong. Second, TypeSafe's accuracy claims
|
|
come from their own "workflow evals" (agreement with the average of two
|
|
frontier LLMs), not from a public reproducible benchmark. The latency/cost
|
|
mechanism stands on its own logic; the accuracy numbers are so far self-run.
|
|
|
|
## Worked examples
|
|
|
|
**1. Support-ticket triage (the canonical case).** The code example above:
|
|
category, frustration, refund intent, and policy coverage answered in one
|
|
call; Python branches on the answers. On a million tickets, the launch-week
|
|
arithmetic gives ~$6.5k instead of ~$30k with most tickets answered in under
|
|
half a second.
|
|
|
|
**2. Code review risk matrix** (`devagrawal09/jev-review`). A staged reviewer:
|
|
first a Noul risk matrix over a diff, then Choice/Score profiles per file, then
|
|
severity. Cheap filters decide which files a big model looks at.
|
|
|
|
**3. Browser agent** (`browser-use/jev-ultrafast`). Each page becomes a
|
|
numbered element table; one Jev call picks both the operation (CLICK / TYPE /
|
|
SCROLL / DONE / BLOCKED) and its target; a small LLM runs only to type text.
|
|
Booked a flight on real Google Flights in 7.1 s for $0.0039.
|
|
|
|
**4. Playing Doom.** Game state as a structured object in, action Choice out,
|
|
~10 decisions/second. Games are the clean demo of *judgment per tick* — the
|
|
same shape as a trading bot deciding per 300 ms block or a drone's ~2.5 Hz
|
|
tactical layer sitting above a code-owned flight controller.
|
|
|
|
**5. Economics at scale** (1kpapers.com). 1,018 papers summarized by a
|
|
generative model ($3.99), classified into 24 topics by Jev — **$0.08**, median
|
|
256 ms per paper. Generative model writes, decision model classifies.
|
|
|
|
## What Jev is bad at (per TypeSafe's own "jaggedness" page)
|
|
|
|
- **Reads literally.** It answers the question you wrote, not the one you
|
|
meant. Negations and implied conditions land at face value.
|
|
- **Not a calculator.** Counting, date arithmetic, anything orderable-as-text
|
|
is unreliable. Iterate in code, ask one Noul per item.
|
|
- **Context rot.** Accuracy drops when state is padded with material the
|
|
question does not need. Retrieve and filter first.
|
|
- **State is not hostile.** Text engineered to argue for its own classification
|
|
can move the answer. User-controlled content in state is a threat model you
|
|
must handle.
|
|
- **Does not generate.** No strings, no code, no summaries. If you need a
|
|
value extracted from free text, get candidates with code or a generative
|
|
model, then let Jev pick.
|
|
- It also reads *literally*: contradictory criteria confuse it.
|
|
|
|
## Status of the claims
|
|
|
|
| Claim | Status |
|
|
| --- | --- |
|
|
| Three primitives, one parallel pass, RLCD by name | confirmed (TypeSafe's own materials) |
|
|
| RLCD loss/reward construction, architecture | undisclosed |
|
|
| $0.042/MTok, 70-500 ms, 20-200x / 40-400x gains | self-reported, not independently verified |
|
|
| Workflow-eval accuracy vs GPT-6 Astra / Fable 5.1 average | internal methodology, no public ground truth; one chart showed Jev below Sonnet 5 raw accuracy |
|
|
| "Zero hallucinations" | = zero schema/type errors by construction; semantic errors still possible |
|
|
|
|
## Sources
|
|
|
|
- TypeSafe launch post: <https://typesafe.ai/blog/introducing-system-one-models-and-jev>
|
|
- Docs: <https://docs.typesafe.ai/>
|
|
- Mechanism explainer (RLCD, single-pass, confirmed-vs-speculative): <https://www.explainx.ai/blog/how-does-jev-work-rlcd-system-one-model-explained-2026>
|
|
- Practical guide with shipped demos and failure modes: <https://dev.to/valyuai/how-to-use-jev-a-practical-guide-to-typesafes-system-one-model-g5e>
|
|
- The Register's critical writeup: <https://www.theregister.com/ai-and-ml/2026/09/16/typesafe-ai-debuts-model-for-machines-that-plays-doom/5296711>
|
|
- Project reference gist: <https://gist.github.com/pjburnhill/adf8d28efcad9df037bfdece178ef965> |