## Documentation Index

Fetch the complete documentation index at: [/llms.txt](https://docs.upliftai.org/llms.txt)

Use this file to discover all available pages before exploring further.

The Realtime Assistants API is currently in **beta**. Features and specifications may change as we continue to improve the platform.

## What are Realtime Assistants?

Realtime Assistants are AI-powered voice agents that can engage in natural, real-time conversations with users. The key benefit is easily creating voice assistants with UpliftAI models, agent hosting, and WebRTC delivery in frontend, mobile, or web apps. They provide:

- **End-to-end latency** of ~1 second for natural conversations (depends on model choices etc.)
- **Natural conversation flow** with interruption handling
- **Multi-modal capabilities** supporting voice, text, and custom tools
- **Dynamic configuration** for real-time behavior updates
  - update tools available for the agent mid session
  - completely update the agent prompt mid session
- **Scalable infrastructure** supporting thousands of concurrent sessions

## Key Features

[**Voice-First Design** 
\Natural speech recognition and synthesis with support for multiple languages and voices](https://docs.upliftai.org/assistants/concepts)

[**Custom Tools** 
\Extend your assistant with custom functions that can access external APIs and services](https://docs.upliftai.org/assistants/tools)

[**Real-time Updates** 
\Update instructions and tools on the fly without restarting sessions](https://docs.upliftai.org/assistants/dynamic-configuration)

[**Easy Integration** 
\Simple SDKs for React, JavaScript, and mobile platforms](https://docs.upliftai.org/assistants/quickstart)

## Use Cases

Realtime Assistants are perfect for:

- **Customer Support** - 24/7 voice-enabled support agents
- **Virtual Receptionists** - Automated call handling and routing
- **Educational Tutors** - Interactive learning experiences
- **Healthcare Assistants** - Patient intake and appointment scheduling
- **Sales Agents** - Product demonstrations and lead qualification
- **Personal Assistants** - Task management and information retrieval

## Architecture Overview

WebRTC

Tool invocation

Custom Tools

MCP Tools

Update Instructions / Tools

User Device

Your UpliftAI Agent

LLM Provider

STT Provider

TTS Provider

External Services

MCP Servers

## Provider Support

### Speech-to-Text (STT)

- **Groq Whisper** (recommended for Pakistani languages, whisper-large-v3)
- **Deepgram** (`nova-3` recommended for English)
- **OpenAI**: `gpt-4o-transcribe` or `gpt-4o-mini-transcribe`
- **UpliftAI** The best Pakistani STT coming soon!

### Text-to-Speech (TTS)

- **UpliftAI Orator**(ultra-fast, natural voices - supports Urdu, Sindhi, Balochi)
  - See [available voices](https://docs.upliftai.org/orator_voices)
- **OpenAI** (standard voices), use model `gpt-4o-mini-tts`

### Language Models (LLM)

- **Groq**(recommended)
  - `openai/gpt-oss-120b` (best quality)
  - `openai/gpt-oss-20b` (faster responses)
- **OpenAI GPT-4** (alternative)

## Getting Started

1. Create an Assistant

Use the API or platform to create your first assistant configuration

```bash
curl -X POST https://api.upliftai.org/v1/realtime-assistants \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Dream Psychologist",
    "description": "Understands the mysterious world of your dreams",
    "config": {
      "agent": {
        "instructions": "You are a dream interpreter who helps people understand their dreams.",
        "initialGreeting": true,
        "greetingInstructions": "Salam! How may I assist you today?"
      },
      "stt": {
        "default": {
          "provider": "groq",
          "model": "whisper-large-v3",
          "language": "en"
        }
      },
      "tts": {
        "default": {
          "provider": "upliftai",
          "voiceId": "v_meklc281",
          "outputFormat": "MP3_22050_32"
        }
      },
      "llm": {
        "default": {
          "provider": "groq",
          "model": "openai/gpt-oss-120b"
        }
      },
      "session": {
        "ttl": 1800
      }
    }
  }'
```

2. Create a Session

Generate a session token for your client application

```bash
curl -X POST https://api.upliftai.org/v1/realtime-assistants/{assistantId}/createSession \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "participantName": "User"
  }'
```

3. Connect Your Client

Use our SDKs to connect to the session

```javascript
import { UpliftAIRoom } from '@upliftai/assistants-react';

<UpliftAIRoom
  token={sessionToken}
  serverUrl={wsUrl}
  connect={true}
  audio={true}
>
  <YourAssistantUI />
</UpliftAIRoom>
```

## Next Steps

[**Read the Concepts** 
\Understand how Realtime Assistants work under the hood](https://docs.upliftai.org/assistants/concepts)

[**Try the Tutorial** 
\Build your first voice assistant in 10 minutes](https://docs.upliftai.org/assistants/tutorial)

[**Explore the API** 
\Deep dive into the API endpoints](https://docs.upliftai.org/assistants/api-reference/create-assistant)

[**View Examples** 
\Check out our example implementations](https://github.com/uplift-initiative/client-js/tree/main/examples/react-assistant-demo)

[Core Concepts](https://docs.upliftai.org/assistants/concepts)