Evidence at a glance
The mechanism in one line
Compress the visual or contextual input before the main reasoning path.
Route or verify the expensive step instead of repeating the full path.
Translate the mechanism into a bounded deployment or evaluation check.
The Goal Is Not to Put a Voice Assistant in One Model
NVIDIA released Nemotron 3.5 Bolt E4B 4GB on October 23, 2026, positioning it as a small language model for local voice and device assistants, interactive game characters, and robotics or embedded interfaces. It is post-trained from Google’s Gemma 4 E4B rather than built as a new architecture from scratch. The problem it addresses is specific: enabling an on-device conversational agent to handle multi-turn dialogue and tool use under resource constraints, rather than simply demonstrating that a small model can produce fluent text.
That distinction matters to how the overall system is divided. Bolt’s model card lists both its input and output as text, so “voice assistant” means that it can provide the dialogue and tool-decision layer of a voice application. It does not mean that the model itself performs speech recognition or speech synthesis. The public materials do not specify a single standard configuration for those speech components, so deployers still need to provide them and connect the model to application state, tool interfaces, and the audio pipeline.
“Four Gigabytes” Is a Test Condition, Not a Whole-System Budget
The label “4GB” can sound like a fixed upper limit on memory, but the published figures are better read as a target for a particular deployment path. NVIDIA’s tests used the Q4_K_M quantized build, llama.cpp, and batch size one. On an RTX 5070, peak memory was 3.5GB and 4.3GB across two workloads. On a Jetson Orin Nano 8GB, it was 4.2GB and 4.9GB. In other words, even the same model exceeds four gigabytes under some of the reported conditions, so the label cannot be translated directly into the total memory a device must reserve.
Context and response length also affect the speed users experience. On an RTX 5070, a test with 3K input tokens and 50 generated tokens recorded a 548-millisecond time to first token and 133.4 tokens per second. Another test, with a 32K cache, 2K input tokens, and 256 generated tokens, recorded a 1,238-millisecond time to first token and 106.2 tokens per second. For real-time voice or robotics, generation speed is only one part of the wait. Users also experience the first response, tool execution, and the next model turn after a tool returns. Engineering evaluation should measure end-to-end response time, not quote tokens per second alone.
Training Treats Dialogue Control as a Core Problem
NVIDIA says Bolt was post-trained from Gemma 4 E4B using verifiable-reward and human-feedback reinforcement learning, without supervised fine-tuning. The research account gives a reason for that choice: supervised fine-tuning improved tool calling but harmed Gemma’s conversational quality and made responses more verbose. The trade-off reflects a tension in the target behavior. An agent needs to choose tools reliably and follow formats, but each interaction should not become an overlong, rigid machine response.
Training included both reasoning and non-reasoning modes, with a 2:1 ratio in each batch, and some environments used length penalties to limit answer growth. The model also exposes a conversational mode, a reasoning switch, and a runtime thinking budget. For product teams, these controls make it possible to tune behavior per request: a short spoken reply can use a tighter reasoning budget, while a more complex task can allow additional reasoning. But the existence of controls does not establish reliability on a particular task. That still needs to be tested with the application’s own prompts, state, and tool-return formats.
Benchmark Wins Do Not Replace Workflow Fit
In its NeMo-Gym evaluations, NVIDIA reports that Bolt led the compared models on 15 of 21 benchmarks. One prominent result is an IHEval score of 84.8, compared with 67.0 for Gemma 4 E4B and 60.6 for Qwen 3.5 4B. These figures indicate that Bolt is competitive on particular evaluation tasks. They are BF16 results, however, and should not be conflated with memory measurements for quantized inference. Nor does leading on a majority of tests establish that Bolt is the better fit for every type of agent workflow.
The counterexamples matter just as much for model selection. Bolt scored 55.1 on Tau2-Bench, below Qwen 3.5 4B at 75.1. On the 64K RULER test, Bolt scored 87.0 and Qwen scored 95.1. The materials also report Qwen ahead on LongBench v2 and BFCL-v3, while Gemma is slightly ahead on EQ-Bench v3. Teams working primarily with long contexts, particular tool-calling patterns, or conversational quality should begin with evaluations that match their actual workflow, then test their own tasks. A single count of benchmark wins is not a selection decision.
Speculative Decoding Speeds Up One Part of Generation
Bolt’s deployment path is not limited to low-bit quantization. NVIDIA also provides a DFlash draft model for speculative decoding: the draft model proposes candidate tokens, which the target model then verifies in parallel. The release reports 272.6 tokens per second for Bolt with DFlash on an RTX 5070, compared with 225.9 for Gemma 4 E4B with an MTP drafter. That makes Bolt about 1.21 times faster in this reported comparison. It is a generation-speed result for specified hardware and configurations, not evidence that first-token latency, tool round trips, or the entire application improve by the same amount.
Bolt is therefore best treated as a composable local language layer for agents, not as a complete voice-enabled robot that works out of the box on one machine. Before deployment, technical leads should measure peak memory, time to first token, generation speed, and tool-call success under the target device and realistic concurrency. They should also check whether long contexts compete with other models for resources. If an application depends on speech recognition, speech synthesis, or vision, those components need their own deployment and memory accounting. “Four gigabytes” is a useful starting point only when the workload genuinely needs local multi-turn dialogue and tool use, and the complete interaction path has been validated on the target hardware.