Speech to Text

VoiceRun supports multiple speech-to-text (STT) providers for real-time transcription. STT is configured in the agent environment, and available configuration options vary by model.

Test Models and Configurations#

Test STT models and configurations in the STT Lab.

Supported Models#

Configuration by Model#

Configuration options differ by model. The table below shows which parameters are available for each model.

Deepgram Nova-3#

General-purpose model with broad language support.

Configuration#

Prompt Format#

Nova-3 uses keyword-style prompts. Provide comma-separated terms to improve recognition of domain-specific words:

policy, premium, deductible, copay, beneficiary

Deepgram Flux#

Ultra-low latency English model with end-of-turn (EOT) detection.

Configuration#

EOT Detection#

Flux uses confidence-based end-of-turn detection instead of simple silence timing. Higher thresholds wait for more confident turn endings; lower thresholds end turns more quickly.

OpenAI Models#

gpt-4o-transcribe, gpt-4o-mini-transcribe, and gpt-realtime share the same configuration options.

Configuration#

VAD Modes#

• server_vad: Uses silence duration (endpointing) to detect turn end
• semantic_vad: Uses context-aware detection with configurable eagerness

When using semantic_vad, the endpointing parameter is ignored and vadEagerness controls sensitivity.

Prompt Format#

OpenAI models use context-style prompts. Provide a natural language description of the conversation context:

This is a customer service call about insurance claims.

Qwen3 ASR#

Low-latency model optimized for Chinese and English.

Configuration#

Cartesia Ink-Whisper#

Broad language coverage with simple configuration.

Configuration#

Cartesia does not support prompt/keyword biasing.

ElevenLabs Scribe v2#

Low-latency multilingual model with keyword biasing.

Configuration#

Prompt Format#

ElevenLabs uses keyword-style prompts, similar to Deepgram. Provide comma-separated terms:

policy, premium, deductible, copay, beneficiary

Soniox#

Real-time STT with semantic VAD for context-aware turn detection.

Configuration#

VAD#

Soniox uses semantic VAD by default and does not support silence-based endpointing. Use vadEagerness to control how aggressively turns are ended.

Updating Settings at Runtime#

Use STTUpdateSettingsEvent to change STT settings during a conversation. Available parameters vary by model—see the model-specific sections above for supported options.

async def handler(event: Event, context: Context):
    if isinstance(event, TextEvent):
        user_message = event.data.get("text", "N/A")

        if "habla español" in user_message.lower():
            yield STTUpdateSettingsEvent(language="es")

Receiving Transcriptions#

Transcribed speech is delivered to your handler as a TextEvent:

from primfunctions.logger import logger

async def handler(event: Event, context: Context):
    if isinstance(event, TextEvent):
        user_text = event.data.get("text", "N/A")
        source = event.data.get("source")  # "speech" for STT
        language = event.data.get("language")  # detected language (if available)

        logger.info(f"User said: {user_text}")

Fallback Configuration#

Any STT model can be configured with a fallback model in the agent environment. When the primary model fails to connect, VoiceRun automatically switches to the fallback.

Available fallback options: nova-3, gpt-4o-transcribe, gpt-4o-mini-transcribe, gpt-realtime, ink-whisper, scribe-v2-realtime