Skip to main content

Voice Intelligence

Overview​

The Voice Intelligence module allows you to control how your built-in AI Assistant listens, understands, and speaks to your users. This applies directly to the floating microphone icon found in the bottom corner of your application.

[!IMPORTANT] Voice Intelligence vs. VOX Telephony This page configures the web-based microphone assistant. If you are trying to configure real phone numbers, automated calling campaigns, or phone agents, please use the VOX Telephony module instead.

Before You Start​

Before you begin configuring your Voice Intelligence settings, make sure:

  • You have Organization Administrator permissions.
  • You have a working microphone connected to your computer.
  • Your web browser has permission to access your microphone.

Setting Up Voice Intelligence for the First Time​

Follow this step-by-step guide to configure your assistant.

Step 1: Choose the Assistant's Language​

First, you need to tell the assistant which language to expect your users to speak.

This section controls the primary language and regional dialects your assistant will understand.

What are my options?

  • Global: Choose this for standard international languages (like English, Spanish, or French).
  • Deep Indic: Choose this if your users primarily speak regional languages from the Indian subcontinent (like Hindi, Tamil, or Bengali).

Understanding "Listen Only" You might notice a yellow LISTEN ONLY badge on certain languages like Spanish or Tamil.

  • What it means: The assistant will perfectly understand you if you speak in that language. However, when it replies, it cannot speak the response out loud. It will only provide text.
  • Why does this happen? Generating high-quality spoken audio requires complex phonetic models. By default, Orbit provides free, optimized speech generation only for English and Hindi.

Step 2: Choose the Intelligence Engine​

The "Intelligence Engine" is the underlying technology that powers the assistant's ears and mouth.

  • Speech-to-Text (STT): The technology that converts your spoken words into text the assistant can read.
  • Text-to-Speech (TTS): The technology that converts the assistant's written answer into spoken audio you can hear.
  • Latency: How quickly the assistant responds to you (measured in milliseconds). Lower is faster.

The engine determines how fast the assistant responds and how your voice data is processed.

Decision Guide: Which Engine Should I Choose?

Your GoalRecommended EngineTrade-off
I want the fastest possible response times.Standard (Web Speech)Relies on your browser's internet connection.
I handle sensitive/medical data and need strict privacy.High Privacy (Whisper/Kokoro)Slightly slower response times (200ms - 500ms).
My users speak regional Indian dialects or mix languages.Indian Specialization (Indic-VEXYL)Slower response times (300ms - 600ms).

Step 3: Choose How the Assistant Sounds​

Next, give your assistant a personality by selecting a Voice Artist.

Select an artist and adjust the speaking pace to match your brand's personality.

How to select an artist:

  1. Browse the list of available voices. They are categorized by region (e.g., US, UK) and style (e.g., "Clear, Natural" or "Warm, Kind").
  2. Click the Play icon on any card to hear a sample of their voice.
  3. Click anywhere else on the card to select them. A green checkmark will appear.

Pace Control: You can adjust how fast the assistant speaks using the Pace Control slider.

  • Relaxed (0.5x): Slower, clearer speech. Good for technical or complex information.
  • Optimal (1.0x): Natural conversational speed. Recommended for most users.
  • Energetic (2.0x): Very fast speech.

Step 4: Configure Conversation Behavior​

These settings determine how the assistant reacts during a live conversation.

Fine-tune how the assistant handles interruptions and regional accents.

Active Barge-In (Interruptions) This controls whether the user is allowed to interrupt the assistant while it is speaking.

  • Example: The assistant says, "Your appointment is scheduled for Friday..." but you suddenly remember and say, "Wait, change it to Thursday!"
  • If ON: The assistant immediately stops talking, listens to your correction, and adjusts.
  • If OFF: The assistant ignores you and finishes its entire sentence before listening again.
  • Recommendation: Leave this ON for a natural conversation experience.

Bilingual Response & Dialect Target

  • Bilingual Response: If you ask a question in Spanish, should the assistant strictly reply in English (Strict) or attempt to match your language (Matching)?
  • Dialect Target: Choose Indic (Deep) if you expect users to speak with heavy Indian accents. Otherwise, leave it on Global (Neutral).

Step 5: Configure Privacy and Silence Sensitivity​

Orbit Resolve places a heavy emphasis on data privacy. You control exactly what happens to a user's voice recording after they speak.

Retention Strategy

  • Ephemeral (Zero logs): The audio and text are processed in memory and instantly deleted. Nothing is saved. This is the most secure option.
  • Transcribed (Text only): The audio is deleted, but the text transcript is saved to a conversation history.
  • Archived: Both the audio files and text transcripts are permanently stored. Warning: Choosing this option may increase your invoice due to data storage costs.

Silence Sensitivity This setting tells the assistant how long to wait after you stop speaking before it assumes you have finished your sentence.

  • Example: You say, "I would like to order a..." and then pause for a second to think.
  • If set to 500ms: The assistant will cut you off and say, "Order a what?" before you finish thinking.
  • If set to 1500ms (1.5 seconds): The assistant waits patiently for you to finish your thought.

Step 6: Save Your Changes​

Once you have configured all your settings, scroll to the top of the page and click the global Save Changes button. Your new configuration will take effect immediately!

Verify Your Voice Intelligence Setup​

How do you know if you configured everything correctly? Follow these steps to test it:

  1. Look for the floating microphone icon in the bottom corner of your screen.
  2. Click it and speak a test phrase (e.g., "Hello, how are you today?").
  3. Verify STT: Watch the screen to ensure the assistant correctly transcribed what you said.
  4. Verify TTS: Listen to ensure the assistant speaks back to you using the Voice Artist and Pace you selected.
  5. Test Barge-In: While the assistant is replying, interrupt it by saying "Stop." It should immediately halt its speech and listen to you.

Common Setup Scenarios​

Not sure what to pick? Here are three common setups based on real-world needs:

Scenario A: General English Assistant​

  • Language: English (US) or English (UK).
  • Intelligence Engine: Standard (Web Speech) for the fastest responses.
  • Voice: Sky (Clear, Natural).
  • Active Barge-In: ON.
  • Retention: Ephemeral.

Scenario B: Indian Regional Support​

  • Language Mode: Deep Indic.
  • Language: Hindi or English (India).
  • Intelligence Engine: Indian Specialization (Indic-VEXYL).
  • Hinglish Core Detection: ON (allows users to mix Hindi and English in the same sentence).
  • Dialect Target: Indic (Deep).

Scenario C: High-Privacy Medical/Financial Assistant​

  • Intelligence Engine: High Privacy (Whisper/Kokoro).
  • Processing Sensitivity: Expert (ensures maximum accuracy for complex medical or technical terms).
  • Retention: Ephemeral (Zero logs) to maintain strict compliance.

Common Configuration Mistakes​

  • Selecting a "Listen Only" language and expecting speech. If you select Spanish, the assistant will only output text. Do not mistake this for a broken speaker!
  • Setting Silence Sensitivity too low. If you set it to 300ms, the assistant will constantly cut you off while you are thinking. Keep it around 1000ms.
  • Turning off Active Barge-In. This makes the assistant feel like a robotic answering machine rather than a conversational partner.

Advanced Technical Notes (For Developers)​

The following section is intended for developers integrating or troubleshooting the Voice Intelligence APIs.

  • API Endpoint: The entire configuration is bundled and saved via PUT /org/config/voice-assistant.
  • Database Mapping: The UI controls map directly to the OrganizationConfig table (e.g., voice_stt_engine, voice_persona, voice_speed, stt_quality_mode, barge_in_enabled, silence_timeout_ms, data_retention_policy).
  • Processing Sensitivity (stt_quality_mode): This is only active when the local (Whisper/Kokoro) engine is selected. It alters the size of the loaded STT model. The pro and expert modes require the Ascent Business or Apex Infinity organization tiers, respectively.
  • Voice Preview API: The preview button triggers a GET request to /api/v1/tools/tts/speak?text=...&voice={id}&language=en-US. The language=en-US query parameter is hardcoded during previews to ensure Indian voices do not encounter phonetic parsing errors when reading Latin characters.