The mechanism in one line
Compress the visual or contextual input before the main reasoning path.
Route or verify the expensive step instead of repeating the full path.
Translate the mechanism into a bounded deployment or evaluation check.
The Missing Layer in Many Agents Is Not Generation
Fastino Labs has released GLiNER2.5-Decide, a 340-million-parameter open-weight decision model. It takes text and a schema of typed questions, then returns structured answers with probability distributions, confidence scores, and constraint-feasibility metadata. Its target is not open-ended question answering, but the recurring judgments inside agent pipelines: routing, triage, tool selection, and safety guardrails.
That distinction changes the shape of the problem. Many agent systems do not need a polished explanation. They need a stable choice among limited options: who should handle a request, whether a tool should be called, or whether an input should be blocked. When those decisions are still made through generative prompting, downstream code must repair format drift, inconsistent labels, and rule conflicts. GLiNER2.5-Decide takes the opposite approach by making this layer a component with declared inputs, outputs, and constraints.
The Core Mechanism Is Joint Decoding, Not a Longer Prompt
The model is built on a DeBERTa-v3-large encoder and fine-tuned from gliner2-large-v1. It generates no tokens and requires no prompt template. Label sets are supplied at call time. A schema can declare the permitted answers for each question, whether the answer is single-label or multi-label, and whether it represents an ordered value. It can also include instructions, examples, label descriptions, and rules linking answers across questions.
Inference has two stages. The encoder reads the text and schema together and scores every permitted answer. A constrained decoder then searches for the highest-scoring joint assignment that satisfies the rules. The joint step matters. In Fastino’s guardrail example, independent decisions gave prompt-injection detection a score of 0.82 while also labeling the same input safe at 0.52. With a rule that any detected harm requires an unsafe verdict, the output can return safety=unsafe and harm_type=prompt_injection together. The system receives a rule-consistent decision set rather than two conflicting signals that downstream code must interpret.
The Schema Becomes Part of the Agent Workflow
This design makes the schema more than an interface description. It becomes part of the decision logic. Rules can express implications, exclusions, cardinality limits, and ordinal bounds. A single call can also evaluate multiple heads, such as intent, urgency, and route. A single-label head returns one string, while a multi-label head returns every label above the cls_threshold. Labels can carry descriptions, and ordinal scales can be passed as ordinary strings such as “0” through “10”.
For technical leaders, this means some orchestration logic can move into a declarative layer around the model call. A routing model does not need to generate a rationale and then rely on a parser to guess the intended label. Tool selection can first be restricted by permitted values and valid combinations. The wider GLiNER family can also extract entities, relations, and structured records with character-level offsets in one forward pass. However, the material explicitly says that classification answers do not return evidence spans. That makes the system compact, but it also limits how an operator can audit or review a classification.
CPU Deployment Changes Where the Model Can Sit
GLiNER2.5-Decide is released under Apache 2.0, installs with pip install gliner2, and supports CPU, GPU, and air-gapped environments. Fastino also offers hosted inference and fine-tuning through its API. This deployment model differs from a large-model service. If a decision component can run close to the business service, a team does not need to pay a remote generation call, network dependency, and service boundary for every low-complexity routing decision.
Fastino reports end-to-end p50 latency at batch size 1 with two heads and 15 labels. At 64 tokens, the figure is 167.3 ms on a 48-vCPU Intel Xeon Platinum 8581C, 43.6 ms on an NVIDIA T4, 43.4 ms on an L4, 38.3 ms on a V100, and 47.3 ms on an A100. At 1,024 tokens, the A100 reaches 52.6 ms, compared with 75.6 ms on the V100 and 131.4 ms on the L4. Short inputs are dominated by fixed preprocessing and kernel-launch overhead, leaving only about 9 ms between the tested GPUs. Running on a CPU is therefore a deployment option, not proof that CPU is optimal for every workload.
The Evaluation Supports Routing, Not General Reasoning
Fastino evaluated the model on its internally generated, held-out Fast Decisions suite. The suite contains 5,100 test examples across 17 datasets, covering customer operations, routing in banking, clinical, travel, and benefits domains, as well as general content understanding. The metric is exact-match accuracy: a prediction counts only when its label set exactly matches the reference. The model led on 9 of the 17 datasets. Its strongest area was intent routing, with 75.3% on support intent and 64.3% on banking intent, respectively 18.6 and 8.6 points ahead of the next-best models.
Those results make it attractive for specific fixed-label tasks, but they do not establish general decision-making ability. The test suite was generated internally by the publisher, and its task coverage is not the same as a production traffic distribution. More importantly, Fastino is explicit about the model’s limits: it does not reason, explain, or answer open questions. Confidence and probability information can support blocking, routing, or escalation, but they do not automatically provide reliable evidence for why a decision was made.
Where It Fits First—and Where It Does Not
The safer integration pattern is to place GLiNER2.5-Decide in a front-end decision layer rather than make it the primary model. It can handle intent routing, request triage, tool-candidate filtering, and safety decisions with explicit rules. Low-confidence cases, uncovered rules, and requests requiring explanation can then be escalated to a generative model or a human process. This uses the model’s deployment and formatting advantages without asking a non-evidential classifier to perform open-ended judgment.
The trade-off depends on whether a team can express its operational decisions as a stable schema. If label sets, thresholds, and cross-field rules change constantly, the constraint layer becomes a new maintenance burden. If a real request falls outside the predefined labels, joint decoding can still return only the most feasible answer within that limited space. GLiNER2.5-Decide is therefore best understood as a locally deployable, constrained, measurable decision component. Before production use, teams should calibrate confidence on real traffic, test rule conflicts and out-of-distribution inputs, and preserve an escalation path for conclusions that cannot provide evidence spans.