The Change Starts at the Architecture Boundary, Not the Leaderboard
Kyutai has released two 9B open-weight speech-to-speech models under the Voice of Reason name. Both build on GLM-4-Voice-9B and are designed to hear spoken math questions and answer with speech, without an automatic speech recognition step or a separate text language model taking over the reasoning. For real-time voice assistants, this avoids a basic weakness of cascaded systems: every ASR, text-reasoning, and TTS stage can add waiting time and can discard paralinguistic information such as tone.
The release matters not because it has already replaced cascaded systems, but because it reframes the weakness of speech models as a training problem. The GLM-4-Voice base model scores 27.3% on spoken GSM8K, while the earlier STITCH method reached 58.7% by adding reasoning chunks. Voice of Reason extends that direction with supervised fine-tuning and reinforcement learning, suggesting that the gap is not only about model scale. It is also about whether a model has been trained to organize an answer while operating under the constraints of continuous speech generation.
The Model Reasons Under the Constraints of Speaking
GLM-4-Voice does not first generate a complete text answer and then synthesize it. It alternates between 13 text tokens and 26 audio tokens. This allows it to keep producing speech, but it also limits the room available for internal reasoning. Unlike a text-only model, it cannot spend a long interval generating invisible intermediate tokens and then deliver the answer in one batch. It has to keep handing over audio.
The two Voice of Reason checkpoints make this trade-off visible. The direct model speaks its working aloud and reaches 70.3% in the released evaluation, compared with 65.5 ± 1.1% in the corresponding reported result. The STITCH version inserts 100 silent reasoning tokens between spoken blocks and reaches 77.1% in the released evaluation, compared with 74.8 ± 1.1% in the reported result. Kyutai’s design generates later reasoning chunks while earlier speech is playing, so additional thinking does not necessarily add to interactive delay after the first response begins. It does, however, require careful coordination between hidden computation, audio playback, and streaming schedules.
The Training Breakthrough Is How Rewards Enter the Speech Model
The first stage is supervised fine-tuning. Kyutai used 150,616 problems from Orca-Math, had Qwen3-235B rewrite them for spoken delivery, and voiced them with DSM TTS in multiple voices. SFT raised spoken GSM8K accuracy from 27.3% to 61.7%. That jump already shows that spoken problem formats, response conventions, and demonstrations can substantially change model behavior.
The second stage is where speech-native reinforcement learning becomes important. For each spoken question, the model samples four responses at temperature 0.9. Qwen3-235B-A22B-2507 reads the decoded text stream and returns only a binary correct-or-incorrect reward, without seeing the reference answer. Rewards are centered within the group of responses to the same question and used in a group-relative REINFORCE objective. The judge agreed with human labels on 88 of 100 manually checked cases, while training used 16 H100 GPUs for 1,500 updates. The result shows that reward optimization need not use a transcribed text model as the reasoning engine. It can directly shape the behavior of a speech-native model.
Two Details Determine Whether the Reinforcement Learning Works
This is not a matter of applying a generic RL recipe to audio tokens. One crucial correction is temperature handling. Because the model samples at temperature 0.9, the logits in the loss must also be divided by that temperature before the log-softmax is computed. Without this correction, the reported GSM8K score collapses from 65.5% to 12.3%, showing how quickly the reward signal can become distorted when the sampling distribution and training objective disagree.
The other treatment concerns the audio vocabulary. At each audio position, the model does not need to learn which individual audio token will be emitted. The probabilities of the entire audio vocabulary are summed into one abstract “audio occurs” event, and the loss only asks whether audio should come next. The paper argues that, under an assumption that the value is invariant across specific audio tokens, this estimator is unbiased and has lower variance. For engineering leads, this detail matters more than the phrase “uses reinforcement learning” by itself. When the action space contains both text and audio, reward design, sampling distributions, and action granularity jointly determine whether training remains stable.
Deployable, But Not Yet Turnkey Voice Infrastructure
The engineering bar is more specific than the phrase “open weights” suggests. Both BF16 checkpoints can run on a single H100, but deployment also requires the speech tokenizer and decoder from the GLM-4-Voice repository. No Hugging Face inference provider currently hosts the weights, so users must provide the GPU, inference pipeline, and audio decoding components themselves. For a self-hosting research team, this is actionable. For a product team seeking a quick integration, it is still a full systems project.
The performance numbers also need to be kept within their proper bounds. The 77.1% score cannot be compared directly with text models or cascaded systems because the source explicitly says that the comparison systems do not match in scale or architecture. Accuracy on the real spoken transcription evaluation is 72.0 ± 1.9%, while spoken TriviaQA falls from 40.6% to 34.0%, suggesting that math-focused post-training may damage general knowledge performance. The practical judgment is therefore narrower: teams building low-latency math tutoring, constrained question answering, or assistants that must preserve spoken interaction should evaluate silent reasoning and end-to-end training. Teams targeting open-domain knowledge, broad language coverage, or low GPU cost may still prefer cascaded systems because their capabilities are easier to isolate, replace, and govern.