Mistral Voice Agents
AI agent · Voice agent · Customer support by Mistral AI
Build agents that listen, reason, speak, and act.
Mistral Voice Agents enable developers to build interactive AI systems that listen, reason, and speak in 35+ languages. Powered by Voxtral 3, it offers end-to-end audio processing with professional voice cloning and human-in-the-loop controls.
Agent specs
- What it does
- Mistral Voice Agents provide a complete pipeline for voice-to-voice interactions, handling transcription, reasoning, and speech generation through a single API.
- Actions
- Captures voice input, Transcribes speech to text, Generates model-based answers, Synthesizes text to speech
- Integrations
- PyAudio, PortAudio, LiveKit, NumPy, SoX, FFmpeg
- Models
- Voxtral 3
- Availability
- Generally available via the Mistral API for terminal and web applications.
- Launched
- 2026-10-20
- Languages
- English, French, 35+ languages total
Category: Conversational AI
- Channels
- Web, Mobile, Terminal, API
- Models
- Voxtral 3, Voxtral Mini Transcribe Realtime 3, Voxtral TTS 3
- Deployment
- Mistral Cloud, On-premises, Private Cloud, VPC
- Languages
- 35+ languages and 50+ dialects
- Base Model Family
- Voxtral 3
- Supported Languages
- 35+
- Full pipeline: capture, transcribe, reason, and speak
- Supports 35+ languages and 50+ dialects
- Zero-shot and professional voice cloning
- Real-time streaming with low-latency turn detection
Overview
Mistral Voice Agents is an end-to-end agentic solution for building interactive systems that can listen, reason, and speak in real-time. Designed for enterprises and developers, it handles the entire audio pipeline—from capturing and transcribing voice to generating reasoned responses and synthesizing lifelike speech. The system is powered by the Voxtral 3 model family and supports over 35 languages and 50 dialects with low-latency streaming. It distinguishes itself by offering professional voice cloning and deep integration with Mistral's tool-calling and skill-packaging capabilities. Users maintain control through built-in human-in-the-loop approvals for sensitive actions and full observability of agent activity.
Speech Interaction
- End-to-end Audio Pipeline: The Voice Agents API handles the entire pipeline by capturing voice, transcribing it, generating a model-based answer, and speaking that answer.
- Voice Cloning: The system supports zero-shot and professional voice cloning to create customized agent identities.
- Multilingual Support: Voice agents can respond in over 35 languages and 50 dialects with low latency.
- Real-time Streaming: The API supports streaming live audio for real-time transcription and low-latency speech generation.
Agentic Capabilities
- Custom Instructions: Developers can define the agent's personality and behavior through specific natural language instructions.
- Tool Integration: Agents can be equipped with tools to perform actions or retrieve data during a live conversation.
- Configurable Turn Detection: The system provides configurable turn detection and latency settings for agentic use cases.
- Skill Packaging: Developers can package repeatable methods into Skills that the agent applies consistently.
Human Control and Governance
- Approval Controls: The agent shows every tool call and asks for explicit approval before performing sensitive actions.
- Real-time Steering steering: Users can steer the conversation or stop the agent at any point during the interaction.
- Activity Observability: The system provides full observability into model calls, tool calls, and agent activity for Enterprise organizations.
What it does
- Capture voice input\nTranscribe speech to text\nReason over instructions and context\nCall integrated tools\nGenerate speech responses\nClone professional voices
Use cases
- Customer support teams build secure voice assistants grounded in product context to resolve routine requests.
- Broadcasters use real-time audio agents for live fact-checking during broadcasts.
- Enterprises create multilingual digital assistants that can interact with customers in over 35 languages.
- Developers build voice-controlled terminal agents that can execute commands and explain code.
Specifications
- Channels
- Web, Mobile, Terminal, API
- Human control
- Asks for approval before sensitive actions; shows every tool call; allows users to steer or stop at any point
- Models
- Voxtral 3, Voxtral Mini Transcribe Realtime 3, Voxtral TTS 3
- Deployment
- Mistral Cloud, On-premises, Private Cloud, VPC
- Languages
- 35+ languages and 50+ dialects
- Base Model Family
- Voxtral 3
- Supported Languages
- 35+
- Supported Dialects
- 50+
- Primary API Endpoint
- /v1/audio/sessions
- Voice Cloning Types
- Zero-shot, Professional
- Deployment Options
- Cloud, On-prem, VPC
- Interface Support
- Web, Mobile, Terminal, API
How it works
- Install SDK and Tools: Install the Mistral SDK with voice-agents extras and required system tools like PortAudio, SoX, or FFmpeg.
- Initialize Agent Session: Initialize a session using the Voice Agents API, providing natural language instructions to define the agent's behavior.
- Configure Tools and Skills: Define tools and connectors that the agent can call to interact with external systems or proprietary data.
- Deploy and Interact: Run the agent in a terminal, mobile app, or web browser to handle real-time voice conversations.
FAQ
What models power the voice agents?
Mistral Voice Agents are built on the Voxtral model family, specifically Voxtral 3, which handles transcription, reasoning, and speech generation.
Can I customize the agent's voice?
Yes, the system supports both zero-shot and professional voice cloning, allowing you to change the agent's voice using parameters like tts={"voice_id": "fr_marie_neutral"}.
How many languages do the voice agents support?
The agents support over 35 languages and 50 dialects, making them suitable for global enterprise deployments.
Added by AntSpace from public sources (mistral.ai).