CLM-8B Turns Agent Judgment into a Scoring Operation

This Is Not Another Chat Model

Contrastive-LM has released CLM-8B as the first open model in its new category, which it calls Contrastive Language Models. The model is not designed to write the next piece of text from context. It evaluates a set of candidate actions against the current state and returns probabilities, choices, or ordered scores. Its main comparison is TypeSafe AI's proprietary System One model, Jev, which also exposes typed outputs with probabilities rather than ordinary text responses.

That distinction changes the boundary of an agent architecture. A conventional agent often asks one generative model to propose an action and then uses another prompt to judge it, forcing the system to repeatedly process text at every step. CLM-8B separates candidate generation from candidate ranking. The generator expands the search space, while the scorer selects among available options. For tool calls, best-of-N selection, and coding-agent verification, this is not merely a new model label. It removes one decision stage from the generation task.

The Speed Comes from Separating States and Actions

CLM-8B is not making Qwen3-8B generate faster. It avoids generation altogether. The model uses a frozen Qwen3-8B backbone with separate projection heads of about 20 million parameters for states and actions. During training, a bidirectional InfoNCE objective pulls the observed action toward its state and pushes other actions away. At inference, the system computes dot products between the state embedding and each action embedding, then converts the scores into a distribution with a softmax.

This design fits an agent loop in which the state changes continuously while the action set remains relatively stable. clm-serve keeps dedicated GPU memory for cached state and action vectors, in a way that resembles vLLM's KV-cache strategy. In the reported example, revisited states on one RTX 4090 with three actions fell from 1.7 milliseconds to 0.6 milliseconds. A model-card test with roughly 1,000 candidates reported up to a 13x speed advantage over Jev. The gain therefore comes not only from parameter count, but also from vector reuse and from avoiding generation for every candidate.

The Training Results Show Why Hard Negatives Need Timing

CLM-8B follows a three-stage training path. The first stage uses roughly 60 million Nemotron DQA question-answer pairs for pretraining. The second adds about 30 million synthetic hard negatives generated by Gemini 2.5 Flash-Lite. The final stage uses about one million agent trajectories from Agent Data Protocol, Endless-Terminals, and LiteCoder-Terminal-SFT. The sequence matters because it first establishes basic state-action alignment, then teaches the model to distinguish similar but wrong choices, and only afterward adapts it to agent workflows.

The reported comparison supports that ordering. On about 100,000 held-out questions, pretraining alone reached 52.1% top-1 accuracy, while the additional mid-training stage raised it to 69.2%. Starting with hard negatives peaked at 62.4% and then overfit. For a technical lead, this pattern is more informative than a single final score. It points to a practical bottleneck in scoring models: if the underlying representation is not yet formed, adding more near-miss alternatives may make the model memorize local boundaries instead of developing a more stable judgment function.

Nine Times Faster Does Not Mean Better Everywhere

CLM-8B's zero-shot advantage is concentrated in tasks with the right structure. In T-Rex, it took 16.5 milliseconds per decision versus 149.8 milliseconds for Jev, with both systems succeeding on all five trials. In Super Mario, CLM-8B took 33.5 milliseconds versus 132.6 milliseconds, again with five successes each. The reported nine-times figure comes from T-Rex, where actions repeat across states and cached action vectors can be reused continuously.

The speed advantage does not produce the same quality everywhere. On BFCL v4 tool calling, CLM-8B took 76.8 milliseconds with 95.2% accuracy, while Jev took 125.5 milliseconds with 99.2% accuracy. On WikiRacing, CLM-8B completed 26 of 30 tasks in 79.8 milliseconds, compared with 30 of 30 for Jev at 225 milliseconds. This draws a clear deployment boundary: when an incorrect action is expensive, trading accuracy for latency may not be acceptable unless the system also has retries, abstention, or review by a stronger model.

The More Practical Role Is Verifying a Generator

The most convincing use of CLM-8B is as a verifier for coding-agent candidates, not as a standalone replacement for every decision component. In the reported tests, Opus 5 generated best-of-four candidates for DeepSWE, while Fable 5 generated best-of-five candidates for Terminal-Bench 2.1. A fine-tuned CLM head then selected among them. On 38 held-out DeepSWE tasks, verification raised the reported result from 73.7% to 81.6%. On 30 held-out Terminal-Bench 2.1 tasks, it raised the result from 84.0% to 87.6%.

Those results need their qualifiers. They use lightweight fine-tuned heads rather than the zero-shot checkpoint, and they cover held-out subsets rather than full leaderboard submissions. Jev scored below pass@1 on both benchmarks, meaning that under these settings its reranking was worse than simply taking the first sample. CLM's verification latency was 79 milliseconds and 32 milliseconds, compared with 449 milliseconds and 131 milliseconds for Jev, a 4.1x to 5.7x advantage. That demonstrates a high-throughput filter more clearly than it demonstrates a generally reliable code reviewer.

In an engineering stack, CLM-8B can sit between a generator and an executor. The generator expands the chance of finding a successful solution, while the scorer reduces the cost of choosing among candidates. Stable action sets can be encoded and cached in advance. But a probability score does not automatically become a safety policy. For tool calls, file changes, and terminal operations, the system still needs explicit rules for abstention, resampling, escalation to a stronger model, an