For Vision-Language Models, Speed Depends on More Than a Smaller Model

Why Add 280M Parameters to a 3B Model?

Liquid AI has released LFM2.5-VL-3B-DSpark as an experimental speculative-decoding drafter for its LFM2.5-VL-3B vision-language model. It is not a standalone vision model, nor a smaller replacement for the target model. It is a roughly 279.5M-parameter companion that allows the target, which normally generates one token per step, to verify a group of candidate tokens at once. Liquid AI reports a maximum decode speedup of 3.13x on an Apple M5 Max and 2.66x on an NVIDIA H100, with identical output under greedy decoding.

The release matters to technical leaders because it addresses a tension that is easy to miss in vision-language inference. Images make the input path more complex, but the output path is still autoregressive and token by token. The important optimization may therefore not be a redesign of the vision encoder. It may be a way to carry established text-generation optimizations through the multimodal representation once the information has reached the language model’s hidden layers. DSpark is testing that proposition rather than merely presenting a smaller model.

After the Hidden Layers, the Image Does Not Change the Interface

The logic of standard speculative decoding is straightforward. A small drafter first predicts several tokens ahead. The larger target then checks the entire candidate block in one forward pass and keeps the tokens it would have produced itself. When the drafter is accurate, the target performs far fewer token-by-token passes. When the candidates are frequently rejected, the drafting work and verification work still occur, so the gain falls.

DSpark’s key implementation is to read hidden states from several layers of the target model and predict the next k tokens. The supplied material argues that, at these hidden layers, text tokens and image patches are both represented as tensors. The drafter therefore does not need a separate inference algorithm simply because the original input included an image. It is a simplified attention-only model. Its ablations selected four layers and a draft block of nine tokens. At inference time, Liquid AI recommends a block size of eight or nine depending on the hardware, with eight recommended for Apple silicon.

The design also controls overhead through shared components. The drafter and target share the embedding and language-model head, while the added structure is mainly four attention layers and two small heads. Liquid AI says this increases the deployed parameter count by about 8.9%, rather than duplicating a full language model. That percentage is not the same as the additional runtime cost, because the drafter still has to execute. It does show the central trade-off: add a limited amount of model and memory overhead in exchange for f

3.13x Is a Decode Number, Not a Request-Latency Promise

DSpark’s performance numbers need to be separated into two layers. Liquid AI distinguishes decode speedup from end-to-end speedup. The former measures only generation, while the latter also includes vision encoding and prefill. The supplied demonstration makes the constraint explicit: vision encoding and prefill do not become faster merely because the drafter is added, so they cap the gain for a complete request. This is a classic Amdahl’s law problem. The smaller the accelerable share of the request, the closer the total improvement remains to the limit imposed by fixed costs.

The task-specific figures illustrate the gap. On the M5 Max, the 3.13x decode result comes from COCO image captioning, while the 2.62x end-to-end result comes from MMMU-Pro. Another supplied end-to-end example is a 1.56x result for TextVQA. Therefore, “up to 3.13x” and “how much faster is one user request” are not the same metric, and they do not necessarily come from the same task. A technical leader who cites only the first number can easily turn a local improvement in generation into an unsupported product-level latency promise.

Acceptance rate is the other path that determines the benefit. The material says that acceptance landed in a similar range across the tested Apple stacks. Liquid AI consequently leans toward the view that acceptance depends more on the drafter and workload than on the runtime itself. The report also presents an approximate range of 3.2 to 4.6 accepted draft tokens for its simulation and display. That range cannot be converted directly into a forecast for every applicatio

Runtime Support Lowers the Barrier, Not the Engineering Cost

LFM2.5-VL-3B-DSpark is delivered in a form that is close to an integrable component. The weights are available on Hugging Face in both Safetensors and GGUF formats, with day-one support in SGLang, MLX-VLM, and llama.cpp. For teams using Apple silicon, NVIDIA GPUs, or established open-source inference stacks, this makes evaluation easier than a release limited to a paper or research code. The material also states that all training and ablations were run exclusively on AMD hardware, while data was collected through Liquid AI’s public device-benchmarking infrastructure, Pipette.

Runtime entry points do not mean that the feature can be enabled unconditionally in production. Speculative decoding adds drafting computation, memory pressure, and another scheduling path. The target must verify a candidate block in one forward pass, while the runtime must handle accepted and rejected candidates. If the business produces short answers, or if most request time is spent on image encoding and prefill, the saved target-model passes may not offset the added work. A deployment test should put time to first token, generation speed, end-to-end latency, peak memory, and acceptance rate in the same measurement set.

The evaluation coverage also needs to be interpreted precisely. Liquid AI evaluated six MMSpec categories: General VQA, Text VQA, Image Captioning, Chart VQA, Complex Reasoning, and Multi-turn Conversation. All runs used batch size one and temperature zero, with 16-bit weights for the vision encoder and backbone. Those settings help isolate speculative decoding, but they do not rep

The Deployment Boundary Includes Licensing and Workload Fit

One constraint in this release should not be hidden behind the performance figures. The LFM Open License v1.0 permits free commercial use, but the supplied material specifies that the company must have less than 10 million dollars in annual revenue. A company above that threshold cannot infer commercial permission merely because the weights are public, the formats are open, or the runtimes already support the model. License review is not a post-launch formality. It belongs in the same decision process as model evaluation.

From an engineering perspective, DSpark is best treated as a bounded inference component rather than a universal speed switch. Its value comes from checking multiple candidate tokens in one target-model computation. The mechanism turns the drafter’s extra cost into a visible benefit only when the candidates are sufficiently aligned with the target and decoding represents a large enough share of the complete request. Long outputs, stable task distributions, and available memory strengthen the case for adoption. Short answers, expensive visual preprocessing, or unstable acceptance weaken it.

Before integration, a team should build a task-level measurement table. Each task should record image characteristics, output length, the share of time spent decoding, average accepted tokens per verification pass, decode and end-to-end speed, peak memory, and any output-consistency constraints. The team should then compare the baseline, different block sizes, and different hardware stacks. If the gain appears only in a narrow task-level metric, or if the license condi