What is Ratchet

In one sentence

Ratchet is a gateway you point your OpenAI client at. A small model you own answers the requests it is sure about. A large open model answers the rest, and its answers become training data. Every night a new small model is trained and is released only if it passes a held-out exam, a silent trial and a staged rollout.

In plain words

Your app is a busy restaurant. Today every dish is cooked by a famous head chef from an agency that charges per plate, even when the order is toast for the hundredth time.

Ratchet puts an apprentice next to the head chef. The apprentice cooks each dish a few times. If the results agree and the apprentice is sure, the plate goes out. If not, the head chef cooks it, and the dish goes in a recipe book with the chef’s name and a note that copying it is allowed. At night the apprentice studies the book. The next morning’s apprentice cooks only if it passes a sealed exam first.

1,9001,9001,900tests in the suiteThey run offline, with no network and no GPU.

95%held-out precision floorA new adapter must reach it on golden rows it never trained on.

Open teachers onlylicense-clean, in codeClosed models are refused as teachers.

The problem, in numbers you can check

• The bill grows with traffic. Enterprise spend on model APIs was $12.5B in 2025 (Menlo Ventures, 2025 report). Much of it is narrow and repetitive: labelling, extraction, ticket triage, steps inside agents.
• The obvious fix is often not allowed. Training a cheaper model on a closed model’s answers breaks the terms of the largest vendors. OpenAI Business Terms §3.3(e) and Anthropic Commercial Terms §D.4(a) (both pages read Oct 21, 2026) forbid using outputs to train competing models. A tool that distills that traffic puts its users at risk. Not legal advice.
• Doing it by hand is slow. Logging, cleaning, fine-tuning, evaluating, deploying and watching for regressions are separate chores, and most tools cover one.

Before

Every request, repeated or not, is a paid call to the teacher. Distilling by hand is slow, and closed models forbid training on their outputs.

After

A student you own answers what it is sure about, the teacher answers the rest, and a better student is trained every night from open teachers only.

The idea

Learn from the traffic you already have, but only from teachers whose license allows it, and never ship a new model on faith.

Change one line, your base URL, and four parts start working:

1. The student answers first

   Nano 4B + your LoRA

   Sampled 3 times; its confidence decides.

2. The teacher answers the rest

   Ultra on Token Factory

   And checks a random 5% of the student’s answers.

3. Answers become training data

   with provenance

   Each with its teacher and license, after redaction.

4. Released through gates

   every night

   Golden set, shadow, canary 10%, 50%, 100%, rollback.

Every night, again

In more detail:

Part	What it does
The student	Nano · yours NVIDIA Nemotron 3 Nano 4B with a LoRA adapter you own. It answers first and is sampled three times. Its confidence mixes agreement across samples with token probabilities
The teacher	Ultra · Token Factory NVIDIA Nemotron 3 Ultra on Nebius Token Factory. It answers when the student is unsure, and checks a random share of the student’s answers (audits)
The ledger	Every teacher answer is stored, after personal data is redacted, with the teacher that wrote it and the license that permits training on it. Closed models are refused as teachers in code
Night School and the gate	Each night a new adapter is trained from teacher answers only and scored on a held-out golden set. It then goes through shadow mode, a canary at 10%, 50% and 100%, and rolls back automatically if audited precision drops

flowchart TB
  app(["Your app"]) ==> gw["Ratchet gateway"]
  gw ==> st["Student<br/>Nano 4B + your LoRA"]
  st ==>|"confident"| out(["Answer"])
  st -->|"unsure"| te["Teacher<br/>Nemotron 3 Ultra"]
  te --> out
  te --> led[("Ledger<br/>answer, teacher, license")]
  led --> ns["Night School<br/>trains a new adapter"]
  ns --> gate{{"Release gate<br/>golden set, shadow, canary"}}
  gate -.->|"next student"| st
  class st student
  class te teacher
  class led store
  class gate gate
  class out ok

Reading this diagram. Start at the top: every request reaches the student first, and the thick path is the cheap one, where a confident student answers at once. Only unsure requests go to the teacher, and every teacher answer also flows down into the ledger. The dotted line closes the loop: what the ledger collects trains the next student, and the gate decides whether that student replaces the current one. That loop is why the share of rented answers can shrink over time.

• The student you own
• The teacher, rented per token
• Stored data
• A check that can stop a release
• Main path
• Once a night

The same flow for one request, step by step:

sequenceDiagram
  participant A as Your app
  participant G as Gateway
  participant Student as Student
  participant Teacher as Teacher
  participant Ledger as Ledger
  A->>G: chat request
  G->>Student: sample 3 answers
  Student-->>G: answers, probabilities
  alt confident
    G-->>A: student answer
  else unsure
    G->>Teacher: same request
    Teacher-->>G: teacher answer
    G->>Ledger: store with its license
    G-->>A: teacher answer
  end
  opt random audit
    G->>Teacher: check the answer
    Teacher-->>G: agrees or not
    G->>Ledger: record the verdict
  end

Reading this diagram. Time runs downward. The gateway always asks the student first. The alt block is the fork: a confident answer returns at once and costs the teacher nothing, while an unsure one is answered by the teacher and stored with its provenance. The opt block at the bottom is the audit: on a random share of confident answers the teacher quietly checks the student, so wrong-but-sure answers show up in the numbers instead of staying hidden. For the full detail see One request’s journey.

Ratchet also reports the bill honestly: savings per token and by GPU-hours actually paid, and a plain warning when a GPU endpoint costs more than it saves. See Why it matters.

What is real today and what is simulated

Part	Status
Gateway, cascade, ledger, Night School, release gate, Scout, dashboard	Built, with 1,900 tests that need no network and no GPU
Teacher calls	Real Nemotron 3 Ultra on Nebius Token Factory. Real teacher calls built the text-to-SQL data on Oct 22, 2026
Student endpoint	Not served on a Nebius GPU yet. Until it is, student calls on the live demo go to the hosted, untuned Nano 30B Nano 30B · hosted, and pages say so
A text-to-SQL adapter	Trained on a free Colab T4, not on Nebius. Its measured results are on Results
Savings and precision figures from sim.simulate and the story page	Simulation Mock models show how the loop behaves, not real model quality
Live mode, model ids and prices in ratchet/config.py	Unverified until scripts/day1_check.py passes with a real key
Token Factory Terms of Service §5(f) (“competing service”)	Open question. Asked Nebius support in writing on Oct 19, 2026; no final answer. Not legal advice

Simulation. In the reference run (mock models, 2,000 tickets a day, a corrupted deploy of v3): the teacher answers everything on day 1, the student serves 98% of requests by day 3, the corrupted v3 is rolled back automatically on day 4, and on day 10 the cost is $0.147 per 1,000 requests against $1.093 for all-Ultra, 87% saved at unverified list prices.

Where to go next

Why it matters

The value in depth, by audience.

One request's journey

What happens to a single API call.

Proof you can run

Check a claim with one command.

For judges

A ten-minute tour.

Common questions

Is the student as good as the teacher? It is held to agreement with the teacher on held-out rows, not to ground truth (see Escalation and audits). Does it always save money? No, see Cost and savings. What do you own? The adapter and its signed bill of materials; the Nano base stays under NVIDIA’s license. More in the FAQ.