Speech to Text

VoiceRun supports multiple speech-to-text (STT) providers for real-time transcription. Configure STT under Deployment.spec.stt in .voicerun/templates/; available options vary by model.

This page covers the STT models available to voice agents (Deployment.spec.stt.model) and the STT Lab.

Test Models and Configurations#

Test STT models and configurations in the STT Lab.

Supported Models#

Configuration by Model#

Configuration options differ by model. The table below shows which parameters are available for each model.

Reading the Default column. The per-model Configuration tables further down list the model's own default — what the registry advertises and what the provider wrapper falls back to when nothing is sent. That is what the console and the STT Lab apply, because both resolve the default from the model registry when you pick a model.

The manifest path does not work that way. For agents deployed from a release (Deployment.spec.stt, in both coderunner and relay mode), the API fills in a fixed platform default for every omitted field before the session starts, so an unset field never reaches the wrapper unset. Regardless of model:

Three fields are the exception — they are nullable end to end and are emitted only when you set them, so omitting them really does inherit the server-tuned default: stt.eot.thresholdByLanguage, stt.vadStopMs, and stt.allowedLanguages. (On deployed voice agents these three are not consumed yet — see VoiceRun Self-Hosted ASR.) stt.eot.eagerThreshold is emitted as an empty value when unset, which the wrapper also reads as "not set".

The two self-hosted VoiceRun ASR models are a further exception for the EoT pair: the platform's Flux-calibrated eot.threshold/eot.timeoutMs defaults are deliberately not forwarded to them, because 0.8/2500 would override the ASR server's own calibration. Those models read a separate nullable plumb, so leaving the EoT fields unset genuinely inherits the server default — see VoiceRun Self-Hosted ASR.

The practical consequence: on the manifest path, any per-model default that is not en or 300 must be written out explicitly. Models whose own default is auto — qwen3-asr-flash-realtime-2026-02-10, voicerun-asr-v1, voicerun-asr-realtime-v1, scribe_v2_realtime, stt-rt-v4, tencent-16k — transcribe as English until you set language: auto.

voicerun-asr-realtime-v1 additionally supports vadStopMs and allowedLanguages, which no other model accepts. See VoiceRun Self-Hosted ASR.

Turn-Taking Mode#

Turn-taking decides when the user has finished speaking. It is configured under Deployment.spec.turnTaking, a sibling of spec.stt, because the signal can come from the STT provider or from raw audio independent of STT.

Each model declares a default mode, and they differ: nova-3, voicerun-asr-v1, voicerun-asr-realtime-v1, and ink-whisper default to silero; every other model defaults to provider. The Deepgram Flux models are provider-only and cannot switch. smart_turn is offered only by nova-3, qwen3-asr-flash-realtime-2026-02-10, voicerun-asr-v1, voicerun-asr-realtime-v1, and ink-whisper.

Those per-model defaults apply to agents configured through the console and the STT Lab. For agents deployed from a manifest, an unset spec.turnTaking.mode resolves to provider regardless of the model's default — set it explicitly when you want silero or smart_turn:

spec:
  stt:
    model: voicerun-asr-realtime-v1
  turnTaking:
    mode: silero
    externalEndpointing: 300

Models without built-in VAD (nova-3, voicerun-asr-v1, ink-whisper) only produce turns under silero or smart_turn.

Deepgram Nova-3#

General-purpose model with broad language support.

Configuration#

Nova-3 has no server-side VAD, so turn-taking runs externally and defaults to silero.

Prompt Format#

Nova-3 uses keyword-style prompts. Provide comma-separated terms to improve recognition of domain-specific words:

policy, premium, deductible, copay, beneficiary

Deepgram Flux#

Ultra-low latency English model with end-of-turn (EOT) detection.

Configuration#

EOT Detection#

Flux uses confidence-based end-of-turn detection instead of simple silence timing. Higher thresholds wait for more confident turn endings; lower thresholds end turns more quickly.

Flux runs turn-taking on the provider side only — spec.turnTaking.mode cannot be changed for these models. endpointing is not exposed as a Flux setting, but a value supplied at connect time is folded into eot.timeoutMs whenever that field is left at its 2500 default, floored at 2500ms. An endpointing sent at runtime through STTUpdateSettingsEvent always overrides eot.timeoutMs, with the same 2500ms floor.

Deepgram Flux Multilingual#

flux-general-multi is the multilingual Flux variant. It shares Flux's EOT configuration and defaults, and differs only in language coverage.

The default is en, not multi. The wrapper's declared default is en, and the manifest path substitutes en for an omitted language (see Configuration by Model), so a deployment that names flux-general-multi and leaves language unset connects with language_hint=en and behaves as an English model. Auto-detect is opt-in: set language: multi explicitly, which makes the wrapper send no language_hint at all. (auto is accepted as the same sentinel.)

Configuration#

A comma-separated list of codes (e.g. en,es) is also accepted, and each code is sent as its own language_hint.

OpenAI Models#

gpt-4o-transcribe and gpt-4o-mini-transcribe share the same configuration options.

Configuration#

These two models are the only models on the platform that accept vad.mode and vad.eagerness.

VAD Modes#

• server_vad: Uses silence duration (endpointing) to detect turn end
• semantic_vad: Uses context-aware detection with configurable eagerness

When using semantic_vad, the endpointing parameter is ignored and vadEagerness controls sensitivity.

Prompt Format#

OpenAI models use context-style prompts. Provide a natural language description of the conversation context:

This is a customer service call about insurance claims.

Qwen3 ASR#

Low-latency model optimized for Chinese and English. The selectable model is qwen3-asr-flash-realtime-2026-02-10.

Configuration#

VoiceRun Self-Hosted ASR#

VoiceRun's first-party self-hosted Qwen3-ASR models. voicerun-asr-v1 transcribes one buffered turn at a time; voicerun-asr-realtime-v1 streams and can run full server-side turn-taking.

Both models share the same 28-language set as Qwen3 ASR (auto, zh, yue, en, ja, ko, and 22 more) and use context-style prompt biasing.

voicerun-asr-v1#

This model has no server VAD; turns are delimited externally and turn-taking defaults to silero.

voicerun-asr-realtime-v1#

Three of these fields exist for this model alone:

• vadStopMs is the first-consult floor in provider mode. It is advanced tuning; leave it unset unless you are deliberately tuning turn latency.
• eot.thresholdByLanguage lets the server pick the end-of-turn threshold from the turn's detected language. The scalar eot.threshold acts as the map's default. Keys must come from the model's language set.
• allowedLanguages constrains language identification to an allowed set rather than the full 28.

All of these are nullable end to end: only values you set explicitly are forwarded, and anything left unset inherits the server-tuned default.

Not yet live on deployed voice agents. vadStopMs, allowedLanguages, and eot.thresholdByLanguage are emitted by the platform as session parameters, but the voice-session entrypoint does not yet map them onto its Configuration object, so the pipeline reads None for all three regardless of what you set. Today they take effect only on the STT Lab / transcription path. The other fields in this table are unaffected.

The default turn-taking mode is silero. Provider-mode turn-taking (Deepgram Flux event parity: Silero VAD plus a semantic end-of-turn model running next to the ASR) is opt-in per agent — set spec.turnTaking.mode: provider explicitly.

Cartesia Ink-Whisper#

Broad language coverage with simple configuration.

Configuration#

Cartesia does not support prompt/keyword biasing. It has no server VAD, so turn-taking defaults to silero.

ElevenLabs Scribe v2#

Low-latency multilingual model with 99 supported languages plus auto detection. The model id is scribe_v2_realtime — with underscores.

Configuration#

Prompt Format#

Scribe v2 uses context-style prompts. Provide a natural language description of the conversation context:

This is a customer service call about insurance claims.

Soniox#

Real-time STT with provider-side endpoint detection.

Configuration#

Endpointing#

Soniox uses silence-based endpoint detection on the provider side; endpointing maps to its maximum endpoint delay. It does not support vadMode or vadEagerness.

endpointing applies only under spec.turnTaking.mode: provider, which is this model's default. Under silero, VoiceRun disables Soniox endpoint detection and turns are delimited externally.

Inworld#

English-only model with keyword biasing.

Configuration#

The model id is inworld/inworld-stt-1 — the slash is part of the id.

Gradium#

European-language model with built-in end-of-turn detection.

Configuration#

Gradium and voicerun-asr-realtime-v1 are the only non-Flux models that consume eot.timeoutMs; Gradium is the only one of the two that does not also support eot.threshold. It supports neither eot.threshold, eot.eagerThreshold, nor prompt/keyword biasing. endpointing is not exposed as a Gradium setting in the model registry — the model advertises external endpointing only — but a value supplied on the manifest path still reaches the wrapper, where it sizes how much context the decoder buffers before finalizing. Raising it makes transcription less aggressive rather than extending a silence window.

Tencent Cloud#

Chinese and Cantonese model. The model id tencent-16k is a selector: the underlying Tencent engine is chosen from language.

Configuration#

Language Selection#

language picks the engine: zh selects the Mandarin engine, yue the Cantonese engine, and auto — along with any unmapped value — selects the large Mandarin engine. There is no error for an unsupported value, so a value like en silently transcribes as Mandarin.

Provider VAD is always on. The model registry advertises provider endpointing for tencent-16k, so the config surface accepts spec.stt.endpointing and the console renders the field, but the current Tencent wrapper only stores the value — it is never forwarded to Tencent, so the setting has no effect today. Tencent's own endpoint detection governs turn ends under provider mode. To control the silence threshold yourself, switch to spec.turnTaking.mode: silero and set externalEndpointing.

xAI Grok#

Multilingual realtime model over WebSocket.

Configuration#

endpointing is sent only under spec.turnTaking.mode: provider, which is this model's default. Grok STT does not support prompt or keyword biasing — a configured prompt is ignored.

Updating Settings at Runtime#

Use STTUpdateSettingsEvent to change STT settings during a conversation. Available parameters vary by model—see the model-specific sections above for supported options.

async def handler(event: Event, context: Context):
    if isinstance(event, TextEvent):
        user_message = event.data.get("text", "N/A")

        if "habla español" in user_message.lower():
            yield STTUpdateSettingsEvent(language="es")

A runtime model change is applied in place only when the new model belongs to the same provider as the active one. Cross-provider switches are logged and ignored.

Receiving Transcriptions#

Transcribed speech is delivered to your handler as a TextEvent:

from primfunctions.logger import logger

async def handler(event: Event, context: Context):
    if isinstance(event, TextEvent):
        user_text = event.data.get("text", "N/A")
        source = event.data.get("source")  # "speech" for STT
        language = event.data.get("language")  # detected language (if available)

        logger.info(f"User said: {user_text}")

Fallback Configuration#

Any STT model can declare a fallback under Deployment.spec.stt.failover. When the primary model fails to connect, or its wrapper reports an error mid-session, VoiceRun switches to the fallback snapshotted in the release.

There is no fixed list of fallback models. Any model from the Supported Models table is eligible, subject to one rule: the fallback must belong to a different provider than the primary model. A same-provider fallback is silently skipped, and the session continues on the failing primary — so pairing gpt-4o-transcribe with gpt-4o-mini-transcribe gives you no protection.

When spec.stt.failover is omitted, the platform applies a default fallback of gpt-4o-mini-transcribe. Every agent therefore has a fallback unless its primary model is also an OpenAI model, in which case the same-provider rule skips it.

Failover fires at most once per session.

spec:
  stt:
    model: nova-3
    failover:
      model: gpt-4o-mini-transcribe