Skip to main content

ToothFairyAI — Voice API Integration Guide

This guide shows how to integrate ToothFairyAI Voice Agents into your web and mobile applications using WebRTC and the LiveKit SDK.

How it works (at a glance)​

User speaks → WebRTC → LiveKit → ToothFairyAI Agent → Response → LiveKit → WebRTC → Audio playback:

  1. API Key Setup: Generate API key from workspace Admin settings
  2. Token Generation: Request a voice session token from the Voice API
  3. LiveKit Connection: Connect to LiveKit room using the token
  4. Audio Streaming: Enable microphone and handle incoming audio streams
  5. Agent processes speech and responds in real-time
Plan Requirements

Voice API integration is available exclusively for Business and Enterprise plans.

Prerequisites​

  • A ToothFairyAI workspace with Business or Enterprise plan
  • API key generated from Admin → API Integration
  • A voice-enabled agent configured in your workspace
  • Basic knowledge of JavaScript/TypeScript

Step 1: Get Your API Key​

1.1) Navigate to API Integration​

  1. Log into your ToothFairyAI workspace
  2. Go to Admin → API Integration
  3. Click Generate API Key if you don't have one
  4. Copy and securely store your API key
Security

Your API key is sensitive information. Never expose it in client-side code. Always make token requests from your backend server.

Step 2: Generate a Voice Token​

2.1) Token Request​

Make a POST request to the Voice API to generate a session token:

curl -X POST https://aiv.toothfairyai.com/token \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "workspace_id": "your_workspace_id",
    "agent_id": "your_agent_id",
    "chat_id": "unique_chat_id"
  }'

2.2) Token Response​

The API returns a token and connection details:

{
  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "url": "wss://toothfairyai-15k1aw45.livekit.cloud",
  "room_name": "ws_abc123::chat_xyz789"
}

2.3) Request Parameters​

ParameterRequiredDescription
workspace_idYesYour workspace identifier
agent_idYesThe voice agent to handle the conversation
chat_idYesUnique conversation identifier
user_idNoOptional user identifier (defaults to "voice-user")
Token Lifetime

Tokens are valid for 1 hour (3600 seconds). Generate a new token for extended sessions.

Step 3: Install LiveKit SDK​

3.1) NPM Installation​

npm install livekit-client

3.2) CDN (Alternative)​

<script src="https://cdn.jsdelivr.net/npm/livekit-client/dist/livekit-client.umd.min.js"></script>

Step 4: Connect to Voice Session​

4.1) Initialize Room Connection​

import { Room, RoomEvent } from 'livekit-client';

// Create room instance
const room = new Room();

// Connect using token from Step 2
await room.connect(tokenResponse.url, tokenResponse.token);

console.log('Connected to room:', room.name);

4.2) Enable Microphone​

// Request microphone access and enable
await room.localParticipant.setMicrophoneEnabled(true);

console.log('Microphone enabled');

4.3) Handle Incoming Audio​

// Listen for audio tracks from the agent
room.on(RoomEvent.TrackSubscribed, (track, publication, participant) => {
  if (track.kind === 'audio') {
    // Attach audio track to an audio element
    const audioElement = track.attach();
    document.body.appendChild(audioElement);

    console.log('Agent audio connected');
  }
});

Step 5: Complete Integration Example​

5.1) Full Implementation​

import { Room, RoomEvent } from 'livekit-client';

class VoiceAgent {
  constructor() {
    this.room = new Room();
    this.setupEventHandlers();
  }

  setupEventHandlers() {
    // Handle new audio tracks
    this.room.on(RoomEvent.TrackSubscribed, (track) => {
      if (track.kind === 'audio') {
        const audioElement = track.attach();
        audioElement.id = 'agent-audio';
        document.body.appendChild(audioElement);
      }
    });

    // Handle disconnection
    this.room.on(RoomEvent.Disconnected, () => {
      console.log('Disconnected from voice session');
      this.cleanup();
    });

    // Handle connection errors
    this.room.on(RoomEvent.ConnectionQualityChanged, (quality) => {
      console.log('Connection quality:', quality);
    });
  }

  async connect(token, url) {
    try {
      await this.room.connect(url, token);
      await this.room.localParticipant.setMicrophoneEnabled(true);
      console.log('Voice session started');
    } catch (error) {
      console.error('Connection failed:', error);
      throw error;
    }
  }

  async disconnect() {
    await this.room.disconnect();
    this.cleanup();
  }

  cleanup() {
    const audioElement = document.getElementById('agent-audio');
    if (audioElement) {
      audioElement.remove();
    }
  }
}

// Usage
const agent = new VoiceAgent();

// Get token from your backend
const tokenResponse = await fetch('/api/voice-token', {
  method: 'POST',
  body: JSON.stringify({ agent_id: 'your_agent_id' })
}).then(r => r.json());

// Connect to voice session
await agent.connect(tokenResponse.token, tokenResponse.url);

5.2) React Integration​

import { useEffect, useRef, useState } from 'react';
import { Room, RoomEvent } from 'livekit-client';

function VoiceChat({ agentId }) {
  const [isConnected, setIsConnected] = useState(false);
  const [isMuted, setIsMuted] = useState(false);
  const roomRef = useRef(null);
  const audioRef = useRef(null);

  useEffect(() => {
    return () => {
      roomRef.current?.disconnect();
    };
  }, []);

  const startCall = async () => {
    // Get token from your backend
    const response = await fetch('/api/voice-token', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ agent_id: agentId })
    });
    const { token, url } = await response.json();

    // Create and connect room
    const room = new Room();
    roomRef.current = room;

    room.on(RoomEvent.TrackSubscribed, (track) => {
      if (track.kind === 'audio') {
        track.attach(audioRef.current);
      }
    });

    await room.connect(url, token);
    await room.localParticipant.setMicrophoneEnabled(true);
    setIsConnected(true);
  };

  const endCall = async () => {
    await roomRef.current?.disconnect();
    setIsConnected(false);
  };

  const toggleMute = async () => {
    const enabled = !isMuted;
    await roomRef.current?.localParticipant.setMicrophoneEnabled(!enabled);
    setIsMuted(enabled);
  };

  return (
    <div>
      <audio ref={audioRef} autoPlay />

      {!isConnected ? (
        <button onClick={startCall}>Start Voice Call</button>
      ) : (
        <>
          <button onClick={toggleMute}>
            {isMuted ? 'Unmute' : 'Mute'}
          </button>
          <button onClick={endCall}>End Call</button>
        </>
      )}
    </div>
  );
}

Regional Endpoints​

Select the endpoint closest to your users for optimal latency:

RegionAPI EndpointDescription
Asia-Pacifichttps://aiv.toothfairyai.comDefault, Australia-based
United Stateshttps://aiv.us.toothfairyai.comUS-based servers
Europehttps://aiv.eu.toothfairyai.comEU-based servers
Performance

The LiveKit WebSocket URL (wss://toothfairyai-15k1aw45.livekit.cloud) uses a global edge network and automatically routes to the nearest server.

Best Practices​

Error Handling​

room.on(RoomEvent.Reconnecting, () => {
  console.log('Reconnecting...');
  // Show reconnection UI
});

room.on(RoomEvent.Reconnected, () => {
  console.log('Reconnected');
  // Hide reconnection UI
});

room.on(RoomEvent.ConnectionQualityChanged, (quality, participant) => {
  if (quality === 'poor') {
    console.warn('Poor connection quality');
    // Notify user
  }
});

Microphone Permissions​

async function requestMicrophonePermission() {
  try {
    const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
    stream.getTracks().forEach(track => track.stop());
    return true;
  } catch (error) {
    console.error('Microphone permission denied:', error);
    return false;
  }
}

// Check before connecting
const hasPermission = await requestMicrophonePermission();
if (!hasPermission) {
  alert('Microphone access is required for voice calls');
  return;
}

Session Management​

  • Use the same chat_id to continue previous conversations
  • Generate new chat_id for fresh conversations
  • Store chat_id to allow users to resume sessions

Common Issues and Solutions​

IssueLikely CauseSolution
No audio from agentTrack not attachedEnsure audio element is added to DOM
Microphone not workingPermission deniedRequest permissions before connecting
Connection timeoutNetwork issuesImplement retry with exponential backoff
Token expiredSession > 1 hourGenerate new token and reconnect
Echo/feedbackAudio routingUse headphones or implement echo cancellation

Success Checklist​

  • ✅ API key generated from Admin → API Integration
  • ✅ Backend endpoint created to generate voice tokens
  • ✅ LiveKit SDK installed and configured
  • ✅ Microphone permissions handled
  • ✅ Audio track subscription implemented
  • ✅ Error handling and reconnection logic added
  • ✅ Voice call tested end-to-end
Pro Tips
  • Test on multiple browsers (Chrome, Firefox, Safari)
  • Implement a connection quality indicator for users
  • Add visual feedback when the agent is speaking
  • Consider adding push-to-talk for noisy environments

SIP Telephony Integration​

ToothFairyAI supports SIP (Session Initiation Protocol) integration for connecting voice agents to telephony systems. This enables enterprise-grade phone integration through SIP trunks.

What You Can Do​

FeatureDescription
SIP InboundReceive phone calls from any SIP trunk provider
SIP OutboundMake phone calls from agents to external numbers
Multi-Provider SupportWorks with Twilio, Vonage, Telnyx, and other SIP providers
Enterprise TelephonyIntegrate with existing PBX and call center systems

Integration Options​

MethodUse CaseConnection
WebRTCWeb/Mobile appsDirect LiveKit SDK
SIP InboundReceive phone callsSIP Trunk → LiveKit
SIP OutboundMake phone callsLiveKit → SIP Trunk
Twilio WebhookTwilio phone numbersTwilio → Webhook → Agent

SIP API Endpoints​

The Voice API provides the following SIP endpoints:

EndpointMethodDescription
/sip/trunkPOSTConfigure a SIP trunk connection
/sip/dispatchPOSTManage SIP call dispatch rules
/sip/callPOSTInitiate an outbound SIP call

Setting Up SIP Integration​

Prerequisites​

  • Business or Enterprise subscription
  • A SIP trunk provider (Twilio, Vonage, Telnyx, etc.)
  • A configured voice agent

Configuration Steps​

  1. Configure SIP Trunk:

    curl -X POST https://aiv.toothfairyai.com/sip/trunk \
      -H "x-api-key: YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{
        "workspace_id": "your_workspace_id",
        "trunk_name": "My SIP Trunk",
        "provider": "twilio",
        "outbound_number": "+1234567890"
      }'

  2. Configure Your SIP Provider:

    • Set the SIP trunk to route calls to your ToothFairyAI workspace
    • Configure authentication (IP allowlist or credentials)

  3. Dispatch Rules:

    • Route incoming SIP calls to specific agents
    • Set fallback agents for overflow handling

  4. Outbound Calling:

    curl -X POST https://aiv.toothfairyai.com/sip/call \
      -H "x-api-key: YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{
        "workspace_id": "your_workspace_id",
        "agent_id": "your_agent_id",
        "to_number": "+19876543210",
        "from_number": "+1234567890"
      }'

SIP vs Twilio Webhook Integration​

AspectSIP IntegrationTwilio Webhook
Setup ComplexityHigher (requires SIP trunk config)Lower (webhook URL only)
Provider FlexibilityAny SIP providerTwilio only
Outbound CallsFull API controlLimited
Enterprise FeaturesFull PBX integrationBasic phone support
CostVaries by providerTwilio pricing
SIP Availability

SIP integration is available for Business and Enterprise plans. The feature is stable for production use. Contact support for assistance with your specific SIP provider configuration.

Best Practices for SIP
  • Use dedicated SIP trunks for voice agent traffic
  • Configure appropriate codecs (G.711, G.722, Opus)
  • Set up fallback routing for high availability
  • Monitor call quality metrics from your SIP provider

Twilio Phone Integration​

ToothFairyAI now offers native Twilio integration with mobile phones, enabling businesses to connect their existing phone numbers directly to ToothFairyAI voice agents through a seamless self-service setup.

What You Can Do​

  • Connect Any Phone Number - Use any phone number available in your Twilio account
  • Self-Service Configuration - Set up without technical support intervention
  • Agent Assignment - Map specific phone numbers to specific voice agents for targeted handling
  • 24/7 Phone Support - Transform any mobile or landline number into an intelligent AI voice assistant

Setup Process​

  1. Prerequisites:

    • A Twilio account with a purchased phone number
    • A ToothFairyAI voice agent configured in your workspace
    • Business or Enterprise subscription

  2. Configure Twilio Webhook:

    • In Twilio Console, go to Phone Numbers > Manage > Active numbers
    • Select your phone number
    • Under Voice Configuration, set the webhook URL:
      https://api.toothfairyai.com/webhooks/twilio/voice/{WORKSPACE_ID}
    • Set HTTP Method to POST

  3. Assign Agent to Phone Number:

    • In ToothFairyAI, go to Settings > Agents
    • Select your voice agent
    • Scroll to Agent Channels section
    • Add your Twilio phone number with the assigned agent

  4. Test the Integration:

    • Call your Twilio phone number
    • The assigned voice agent will answer and handle the conversation
    • Monitor the call in ToothFairyAI's chat interface
Multiple Phone Numbers

You can assign different phone numbers to different voice agents. This allows you to route calls based on the number dialed—for example, sales calls to a sales agent and support calls to a support agent.

Best Practices
  • Configure appropriate voice instructions for phone interactions
  • Set up voicemail fallback behavior
  • Monitor call logs in both Twilio and ToothFairyAI
  • Test call quality before going live

API Reference​

For complete API documentation, see the Voice API Reference and select "TF Voice" from the API dropdown.

Resources​