The Production Problem Is Not Answering, but Acting Automatically
Nokia’s applied research team has open-sourced AnyJev, a Python library that wraps an open large language model as a typed decision model. The goal is not to generate more fluent text. It is to choose one answer from a fixed set and return a probability that a production system can threshold. AnyJev supports three question types: choosing one of several options, answering yes or no, and placing an answer into ordered score bins.
These tasks look simpler than generation, but they expose a problem that ordinary text applications often hide. The same input can produce a different decision when the option order changes. Directly reading the next-token distribution is also affected by label priors, such as a general preference for “Yes,” and by positional preferences for particular slots. In routing, ticket triage, and human-escalation workflows, this invariance is often closer to a deployment requirement than the model’s ability to write an elegant explanation.
AnyJev Repairs the Scoring Interface, Not the Model’s Knowledge
AnyJev borrows its interface from Jev, the System One decision model launched by TypeSafe AI in September 2026, but focuses on making an open model’s next-token distribution usable for decisions. The caller declares a typed question and a set of candidates, then reads the model’s next-token probabilities for those candidates. Nothing is generated or parsed, and no new model is trained. The library is therefore better understood as a decision-calibration layer in front of the model than as a new classifier.
Its default L0 level applies two corrections. First, it uses cyclic shifts. With K options, AnyJev presents the list in K rotations so every option occupies every position, then combines the results in log space as a geometric mean. If positional bias appears as an additive fixed term in logit space, this aggregation removes it exactly. Second, it performs prior correction by maintaining a running mean of predicted distributions on real inputs, then dividing out label preference with a default strength of 0.75. The correction begins after eight items have been observed.
The Benchmark Shows Gains in Stability and Confidence
The evidence targets a specific production failure mode: whether reordering the options changes the answer, and whether the model’s confidence is trustworthy. On Qwen3-8B with BANKING77, the task has 20 categories and 300 test items. With a raw option readout, reversing the options produced a 0.230 flip rate, 0.747 accuracy, and a 0.240 expected calibration error. Only 7.7% of items were automatically decidable under a 5% error threshold.
With L0, the flip rate fell to 0.073, accuracy rose to 0.803, ECE fell to 0.184, and the automatically decidable share rose to 46.3%. L1 then added temperature scaling fitted from 100 to 500 labels. The flip rate was 0.077, accuracy was 0.807, ECE fell to 0.095, and the automatically decidable share reached 52.0%. L1 does not change the answer ranking. Its role is to reshape confidence, which matters when a system must decide which requests to handle automatically and which to send to a human.
The Cost Is One More Prefill for Every Option
Removing positional bias is not free. L0 requires K prefills for a choice question with K options because the model must evaluate every rotation of the list. AnyJev reduces the impact through shared prefixes and batching. The reference reported in the material is about 0.25 seconds per decision on one H100 at batch size 32 with K equal to 20. The library provides Transformers and vLLM backends, and serving can use vLLM with prefix caching.
The architectural trade-off is therefore explicit: AnyJev spends extra compute to obtain an interpretable correction for option order. As the number of options grows, prefill cost still grows linearly. Shared prefixes improve throughput but do not remove the additional work per decision. That may be reasonable for routing among four teams. For high-concurrency classification with dozens of candidates, an engineering team should consider hierarchical routing or question whether per-option calibration remains the right design.
It Fits a Narrow Decision Layer, Not a General Capability Replacement
The most credible use for AnyJev is as a decision layer inside an existing workflow, not as an autonomous replacement for complex judgment. Nokia says its team tried AnyJev on an internal routing problem and saw promising results, but the material does not disclose accuracy, latency, or automation rates. That supports the claim that routing is a plausible target. It does not constitute evidence of a measured production gain.
Calibration and content quality must also be evaluated separately. L0 reduced order flips on all nine model-and-task rows tested, and Qwen3-32B with L1 reached an ECE of 0.036 compared with 0.144 reported for Jev. Yet fine-tuned Laya still led on accuracy. A more defensible deployment path is to use AnyJev first to provide probabilities and escalation thresholds for fixed-label tasks, then test against real traffic before replacing a specialized fine-tuned model. Cleaner confidence is not, by itself, a reason to expand automation.