Cactus Compute released Cactus Hybrid, an open-source project that trains Gemma 4 E2B to recognize when it’s wrong. Every response carries a confidence score between 0 and 1, returned as structured data, not extracted from generated text. Above a threshold, the local model responds. Below, the request goes to a larger model in the cloud.

It’s the missing piece in production agentic infrastructure. A 2-billion-parameter local model is fast and private, but you don’t know when it’s failing. With uncertainty calibration built into the weights, routing becomes an automatic decision based on a verifiable score.

Cactus Hybrid matches Gemma 4 E2B with Gemini 3.1 Flash-Lite on common benchmarks, routing only 15-55% of requests to the cloud model. The rest it solves locally. Numbers vary by benchmark and quantization, but the pattern is clear: most requests don’t need the frontier model.

As we reported about Cursor’s swarm economics, the mix of models matters more than single-model power. Cactus Hybrid formalizes the same principle from the ground up: the small model knows when to pass the ball.

For those running open models in production, uncertainty calibration becomes the second selection criterion after raw quality. The repo is on GitHub with models on Hugging Face.

In detail

What came before.

Open-weight local models like Gemma, Llama, or Qwen run on-device or on a private server, but can’t tell you how confident they are in what they say. If you ask “What’s the capital of France?” and “What’s the Bennett ratio formula?”, the model gives you an answer with the same assertive tone for both. No confidence score, no uncertainty signal.

Those building production agentic systems so far had to estimate uncertainty with external tricks. Generate multiple responses and compare them, ask the model to grade itself (something LLMs do poorly), or use a second model as judge. All solutions that add latency and cost with no guarantees.

How it works.

Cactus Hybrid inserts probes directly into the model checkpoint. During post-training, the model learns to give the answer and estimate how reliable it is. The confidence score is a number between 0 and 1 returned as structured data, separate from the response text. It’s not parsed from generated text: it’s a separate output.

The handoff threshold is configurable. The repo shows a readable example: if confidence < 0.85: answer = ask_a_bigger_model(prompt). Above 0.85 the local model responds, below it goes to the cloud. The cloud model in the initial rollout is Gemini 3.1 Flash-Lite.

The numbers.

Cactus benchmarks show the handoff percentage needed to match Flash-Lite. In FP16: ChartQA 15-20%, MMBench 30-35%, LibriSpeech 25-30%, GigaSpeech 30-35%, MMAU 30-35%. At 4-bit the values grow by 10-15 points. MMLU-Pro requires 45-55% handoff in FP16, nearly double the other benchmarks.

Quantization increases the handoff percentage needed, but even compressed models resolve more than half of requests locally. The trade-off is readable: you save memory on-device, you pay with more cloud calls.

Limitations.

The project is young. Three forks and 189 stars on GitHub, a single model in the initial rollout. Benchmarks are provided by the Cactus team itself, not independent evaluation.

Calibrated confidence is a known and difficult problem. A model can be “confident” and wrong, or uncertain and right. True calibration is measured with ECE (Expected Calibration Error) on validation datasets, and the repo doesn’t report this metric.

Those wanting to use it in production should build their own test bench: a set of questions with known answers in their own domain, compare the confidence score against actual correctness, and verify that the chosen threshold really separates right answers from wrong ones. The playbook for comparing two models is a starting point: same task, same test cases, results table.