Model Configuration¶

This guide covers how to configure AI models for text generation in Speakr, including support for OpenAI's GPT-5 series and other language models.

Overview¶

Speakr uses AI models for several key features:

• Summary Generation: Creating intelligent summaries of your transcriptions
• Title Generation: Automatically generating descriptive titles for recordings
• Event Extraction: Identifying calendar-worthy events from conversations
• Interactive Chat: Answering questions about your recordings
• Speaker Identification: Detecting speaker names from conversation context

These features are powered by large language models (LLMs) configured through your .env file.

Basic Configuration¶

The text generation model is configured using three environment variables:

TEXT_MODEL_BASE_URL=https://openrouter.ai/api/v1
TEXT_MODEL_API_KEY=your_api_key_here
TEXT_MODEL_NAME=openai/gpt-4o-mini

Choosing a Provider¶

OpenRouter (recommended for most users): Provides access to multiple AI models through a single API, often at competitive prices. Supports GPT-4, Claude, and many other models. Configure using TEXT_MODEL_BASE_URL=https://openrouter.ai/api/v1.

OpenAI Direct: Use OpenAI's API directly for access to their latest models including GPT-5. Configure using TEXT_MODEL_BASE_URL=https://api.openai.com/v1. This option is required for GPT-5 models with their specialized parameters.

Custom Endpoints: Speakr works with any OpenAI-compatible API endpoint, including self-hosted solutions like LocalAI, Ollama with OpenAI compatibility, or enterprise API gateways.

Local servers without a key: TEXT_MODEL_API_KEY is optional. Leave it unset for a server that needs no key, such as Ollama, vLLM or llama.cpp on your own network; set only TEXT_MODEL_BASE_URL and TEXT_MODEL_NAME.

Google Gemini (OpenAI-compatible): Google exposes Gemini models behind an OpenAI-compatible URL. Point Speakr at it like any other base URL:

TEXT_MODEL_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/
TEXT_MODEL_API_KEY=your_google_api_key
TEXT_MODEL_NAME=gemini-2.0-flash

No special connector is needed; Speakr's standard LLM client works with the endpoint directly.

GPT-5 Support¶

Speakr fully supports OpenAI's GPT-5 model family, automatically detecting and adjusting API parameters when you use GPT-5 models with the official OpenAI API.

Requirements¶

• OpenAI Python SDK: Version 2.2.0 or higher (included in requirements.txt)
• OpenAI API: Must use TEXT_MODEL_BASE_URL=https://api.openai.com/v1
• Valid API Key: An OpenAI API key with GPT-5 access

Supported GPT-5 Models¶

• gpt-5: Best for complex reasoning, broad world knowledge, and code-heavy tasks
• gpt-5-mini: Cost-optimized reasoning and chat; balances speed, cost, and capability
• gpt-5-nano: High-throughput tasks, especially simple instruction-following
• gpt-5-chat-latest: Latest GPT-5 chat model

Key Differences from GPT-4¶

GPT-5 models use different parameters than previous models:

Unsupported Parameters (will cause errors if used):

• temperature - Replaced by reasoning_effort and verbosity
• top_p - Not supported
• logprobs - Not supported

New GPT-5 Parameters:

Reasoning Effort: Controls how many reasoning tokens the model generates before producing a response.

• minimal: Fastest responses, minimal reasoning tokens (best for simple tasks)
• low: Fast responses with basic reasoning
• medium: Balanced reasoning and speed (default, recommended)
• high: Maximum reasoning for complex tasks like coding and multi-step planning

Verbosity: Controls how many output tokens are generated.

• low: Concise responses
• medium: Balanced detail (default)
• high: Thorough explanations and detailed code

Token Limits: GPT-5 uses max_completion_tokens instead of max_tokens.

Configuring GPT-5¶

Add these settings to your .env file:

# Use OpenAI API endpoint
TEXT_MODEL_BASE_URL=https://api.openai.com/v1
TEXT_MODEL_API_KEY=your_openai_api_key
TEXT_MODEL_NAME=gpt-5-mini

# GPT-5 specific parameters (optional, defaults shown)
GPT5_REASONING_EFFORT=medium
GPT5_VERBOSITY=medium

GPT-5 Configuration Examples¶

Fast Summarization (Low Cost):

TEXT_MODEL_NAME=gpt-5-nano
GPT5_REASONING_EFFORT=minimal
GPT5_VERBOSITY=low

Standard Usage (Recommended):

TEXT_MODEL_NAME=gpt-5-mini
GPT5_REASONING_EFFORT=medium
GPT5_VERBOSITY=medium

Complex Analysis (High Quality):

TEXT_MODEL_NAME=gpt-5
GPT5_REASONING_EFFORT=high
GPT5_VERBOSITY=high

Automatic Detection¶

Speakr automatically detects when you're using:

1. A GPT-5 model (based on model name)
2. The official OpenAI API (based on base URL containing api.openai.com)

GPT-5 Through Azure AI Foundry and Other Gateways¶

The automatic detection only activates on the official OpenAI API URL. If you serve a GPT-5 model through Azure AI Foundry or another gateway, requests fail with an error such as Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead. In that case, force GPT-5 parameter handling explicitly:

# Force GPT-5 parameters (reasoning_effort, verbosity, max_completion_tokens)
# regardless of the base URL
GPT5_FORCE_MODE=true

# Same, for a separately configured chat model (not inherited from the above)
# CHAT_GPT5_FORCE_MODE=true

# Same, for a separately configured translation model (not inherited from the above)
# TRANSLATION_GPT5_FORCE_MODE=true

When both conditions are met, Speakr automatically:

• Removes temperature parameter from API calls
• Adds reasoning_effort parameter
• Adds verbosity parameter
• Uses max_completion_tokens instead of max_tokens
• Logs that GPT-5 parameters are being used

Check your logs for confirmation:

Using GPT-5 model: gpt-5-mini - applying GPT-5 specific parameters

Using GPT-5 Through OpenRouter¶

If you use GPT-5 models through OpenRouter or other proxy services, the automatic GPT-5 parameter handling will not activate. These services typically handle parameter translation themselves, so Speakr uses standard parameters (temperature, max_tokens, etc.).

Use Cases¶

Summarization:

• Fast summaries: gpt-5-nano with minimal effort and low verbosity
• Standard summaries: gpt-5-mini with medium effort and medium verbosity
• Detailed summaries: gpt-5 with medium effort and high verbosity

Chat:

• Quick Q&A: gpt-5-mini with minimal effort and low verbosity
• Standard conversation: gpt-5-mini with low effort and medium verbosity
• Complex analysis: gpt-5 with high effort and medium verbosity

Troubleshooting GPT-5¶

Error: "Unsupported parameter 'temperature'"

This means GPT-5 detection failed. Check that:

1. TEXT_MODEL_BASE_URL contains api.openai.com
2. TEXT_MODEL_NAME starts with gpt-5 or is one of: gpt-5, gpt-5-mini, gpt-5-nano, gpt-5-chat-latest

Error: "Invalid reasoning_effort value"

Valid values are: minimal, low, medium, high

Error: "Invalid verbosity value"

Valid values are: low, medium, high

Migrating from GPT-4 to GPT-5¶

1. Update dependencies (required for GPT-5):

   pip install -r requirements.txt
   

   This upgrades the OpenAI SDK to version 2.2.0 or higher.

2. Update your .env file:

   TEXT_MODEL_NAME=gpt-5-mini  # or gpt-5, gpt-5-nano
   

3. Add GPT-5 parameters (optional):

   GPT5_REASONING_EFFORT=medium
   GPT5_VERBOSITY=medium
   

4. Restart Speakr:

   docker compose restart
   

5. Check logs for confirmation:

   Using GPT-5 model: gpt-5-mini - applying GPT-5 specific parameters
   

Performance Considerations¶

• Cost: gpt-5-nano < gpt-5-mini < gpt-5
• Speed: minimal < low < medium < high reasoning effort
• Quality: Generally increases with model size and reasoning effort
• Token usage: Higher verbosity = more output tokens

For most use cases, we recommend:

• Model: gpt-5-mini
• Reasoning: medium
• Verbosity: medium

This provides a good balance of cost, speed, and quality.

Separate Chat Model Configuration¶

Speakr allows you to configure a separate model specifically for real-time chat interactions, while using a different model for background tasks like summarization and title generation. This enables you to:

• Use different service tiers: Configure a faster, more expensive model for interactive chat while using a cheaper model for background processing
• Optimize costs: Use a budget-friendly model for summarization while keeping a high-quality model for chat
• Balance speed and quality: Prioritize low latency for chat while accepting slower processing for summaries

Configuration¶

Add these optional environment variables to your .env file:

# Chat Model Configuration (Optional)
# If not set, chat will use TEXT_MODEL_* settings
CHAT_MODEL_API_KEY=your_chat_api_key
CHAT_MODEL_BASE_URL=https://openrouter.ai/api/v1
CHAT_MODEL_NAME=openai/gpt-4o

Fallback Behavior¶

The text model's key is sent only to the text model's server. For a chat model on another server, CHAT_MODEL_API_KEY is used, or no key.

GPT-5 Settings for Chat¶

If you use GPT-5 models for chat, you can configure separate GPT-5 parameters:

# Chat-specific GPT-5 settings (optional)
# Falls back to GPT5_* settings if not specified
CHAT_GPT5_REASONING_EFFORT=medium
CHAT_GPT5_VERBOSITY=medium

Example Configurations¶

Cheap Summarization + Premium Chat:

# Background tasks: Use budget model
TEXT_MODEL_BASE_URL=https://openrouter.ai/api/v1
TEXT_MODEL_API_KEY=your_openrouter_key
TEXT_MODEL_NAME=openai/gpt-4o-mini

# Interactive chat: Use premium model
CHAT_MODEL_API_KEY=your_openai_key
CHAT_MODEL_BASE_URL=https://api.openai.com/v1
CHAT_MODEL_NAME=gpt-5-mini
CHAT_GPT5_REASONING_EFFORT=low
CHAT_GPT5_VERBOSITY=medium

Same Provider, Different Models:

# Background tasks: Smaller model
TEXT_MODEL_BASE_URL=https://openrouter.ai/api/v1
TEXT_MODEL_API_KEY=your_api_key
TEXT_MODEL_NAME=google/gemini-2.5-flash-lite

# Interactive chat: Larger model (same provider)
CHAT_MODEL_NAME=openai/gpt-4o
# Note: CHAT_MODEL_API_KEY not needed if using same key
# Note: CHAT_MODEL_BASE_URL not needed if using same endpoint

Different Service Tiers (OpenAI):

# Background tasks: Standard tier
TEXT_MODEL_BASE_URL=https://api.openai.com/v1
TEXT_MODEL_API_KEY=your_standard_tier_key
TEXT_MODEL_NAME=gpt-4o-mini

# Interactive chat: Priority tier for faster responses
CHAT_MODEL_API_KEY=your_priority_tier_key
CHAT_MODEL_NAME=gpt-4o

When to Use Separate Chat Models¶

Recommended for:

• High-volume deployments where chat responsiveness is critical
• Users who need different service tiers for different operations
• Cost optimization when chat usage is significantly higher than summarization

Not needed for:

• Small deployments with low usage
• When using the same model for all operations is acceptable
• Simple setups where configuration simplicity is preferred

Separate Translation Model Configuration¶

Speakr can translate recordings with a dedicated model instead of the text model used for summaries and titles. This lets you send translation to a cheaper, faster or multilingual model while keeping the text model for the rest of the application. Speaker labels and timestamps are preserved regardless of the model; the model only translates the spoken text.

The dedicated model is used for both manual translations and automatic translation after transcription. Translation token usage is recorded under the translation operation type, so it is separated from summaries and chat in the token statistics.

Configuration¶

Add these optional environment variables to your .env file:

# Translation Model Configuration (Optional)
# If not set, translation uses the TEXT_MODEL_* settings
TRANSLATION_MODEL_API_KEY=your_translation_api_key
TRANSLATION_MODEL_BASE_URL=https://openrouter.ai/api/v1
TRANSLATION_MODEL_NAME=openai/gpt-4o-mini

A dedicated translation model needs TRANSLATION_MODEL_NAME and either its own key or its own base URL. Without both parts, Speakr falls back to the text model.

Fallback Behavior¶

The text model's key is sent only to the text model's server. For a translation model on another server, set TRANSLATION_MODEL_API_KEY, or leave it unset for a local server that needs no authentication.

Example Configuration¶

# Summaries, titles and chat: general-purpose model
TEXT_MODEL_BASE_URL=https://openrouter.ai/api/v1
TEXT_MODEL_API_KEY=your_openrouter_key
TEXT_MODEL_NAME=openai/gpt-4o-mini

# Translation: cheaper multilingual model on the same provider
TRANSLATION_MODEL_NAME=google/gemini-2.5-flash-lite
TRANSLATION_MODEL_API_KEY=your_openrouter_key
# Note: TRANSLATION_MODEL_BASE_URL not needed when the text model uses the same endpoint

When to Use a Separate Translation Model¶

Recommended for:

• Translating long or numerous transcripts where a cheaper model is sufficient
• Deployments that need a model with better coverage of the target languages
• Routing translation to a local server while summaries and chat use a hosted API

Not needed for:

• Small deployments where the text model already translates well
• When using the same model for all operations is acceptable
• When translation is disabled and no user will trigger it manually

Model Selection Guidelines¶

For Summaries¶

The model you choose significantly impacts summary quality:

• GPT-4 or better: Produces nuanced, context-aware summaries with excellent understanding of complex topics
• GPT-5-mini: Excellent balance of quality and cost for most summarization needs
• GPT-3.5/4o-mini: Budget-friendly option, suitable for straightforward content
• Claude models: Strong performance on structured content and technical material

For Chat¶

Chat features benefit from more capable models:

• GPT-5: Best for complex multi-turn conversations and detailed analysis
• GPT-5-mini: Recommended for most chat use cases
• Claude: Excellent for technical discussions and code-related queries

Cost Optimization¶

To reduce costs while maintaining quality:

1. Use smaller models for simple tasks: gpt-5-nano or gpt-4o-mini handle straightforward summaries well
2. Adjust GPT-5 reasoning effort: Use minimal or low for quick tasks
3. Set token limits: Configure SUMMARY_MAX_TOKENS and CHAT_MAX_TOKENS in your .env
4. Use OpenRouter: Often provides better rates than direct API access

Testing Configuration¶

After changing model configuration:

1. Restart the Speakr container
2. Create a test recording
3. Review the generated summary and title
4. Test the chat feature
5. Monitor logs for any errors or warnings

Provider-Specific Request Parameters¶

Reasoning ("thinking") is controlled differently by each provider, and some models think by default, which makes summaries, titles and translations slower and more expensive. TEXT_MODEL_EXTRA_BODY holds a JSON object whose fields are added to every request to the text model. CHAT_MODEL_EXTRA_BODY is the same setting for a separately configured chat model, and TRANSLATION_MODEL_EXTRA_BODY for a separately configured translation model.

# Turn off thinking for a Qwen3 model on vLLM, SGLang or llama.cpp
TEXT_MODEL_EXTRA_BODY={"chat_template_kwargs": {"enable_thinking": false}}

The fields are sent unchanged, so they must be ones your provider and model accept:

The name of the chat_template_kwargs field depends on the model's chat template and is given on the model card.

• The fields are added after Speakr's own parameters and replace any with the same name, including the GPT-5 settings and max_tokens.
• When chat or translation uses the text model, TEXT_MODEL_EXTRA_BODY applies to it as well. A separately configured chat or translation model does not inherit it.
• In .env, the JSON is written without surrounding quotes.
• If the value is not a JSON object, requests to that model fail with an error that names the variable.
• When a provider rejects a request, the log lists the fields added by this setting. Some servers ignore fields they do not recognise, so compare response times or token counts before and after a change to confirm its effect.
• Reasoning tokens are counted as completion tokens in the token statistics. The reasoning text is not stored with summaries, titles or chat answers.

Environment Variables Reference¶

# Required: API endpoint
TEXT_MODEL_BASE_URL=https://api.openai.com/v1

# API key (can be left out for a local server without authentication)
TEXT_MODEL_API_KEY=your_api_key_here

# Required: Model identifier
TEXT_MODEL_NAME=gpt-5-mini

# Optional: Maximum tokens for summaries (default: 8000)
SUMMARY_MAX_TOKENS=8000

# Optional: Maximum tokens for chat responses (default: 2000)
CHAT_MAX_TOKENS=2000

# Optional: Maximum tokens for AI title generation (default: 100)
# Bump for reasoning models (o1, Kimi 2.5, etc.) that consume budget on hidden thinking tokens
TITLE_MAX_TOKENS=200

# Optional: Maximum tokens for event extraction from transcripts (default: 4000)
EVENT_MAX_TOKENS=4000

# Optional: Prefix-cache-friendly title/summary prompts (default: false). See below.
PREFIX_CACHE_OPTIMIZED_PROMPTS=false
# Optional: cache breakpoints for Anthropic models (default: false). See below.
TEXT_MODEL_CACHE_BREAKPOINTS=false

# GPT-5 specific (only used with GPT-5 models and OpenAI API)
GPT5_REASONING_EFFORT=medium  # minimal, low, medium, high
GPT5_VERBOSITY=medium          # low, medium, high

# Chat model configuration (optional - falls back to TEXT_MODEL_* if not set)
CHAT_MODEL_API_KEY=your_chat_api_key
CHAT_MODEL_BASE_URL=https://openrouter.ai/api/v1
CHAT_MODEL_NAME=openai/gpt-4o

# Chat-specific GPT-5 settings (optional - falls back to GPT5_* if not set)
CHAT_GPT5_REASONING_EFFORT=medium  # minimal, low, medium, high
CHAT_GPT5_VERBOSITY=medium          # low, medium, high

# Translation model configuration (optional - falls back to TEXT_MODEL_* if not set)
TRANSLATION_MODEL_API_KEY=your_translation_api_key
TRANSLATION_MODEL_BASE_URL=https://openrouter.ai/api/v1
TRANSLATION_MODEL_NAME=openai/gpt-4o-mini

# Optional: force GPT-5 parameter handling for a separately configured
# translation model (not inherited from GPT5_FORCE_MODE)
# TRANSLATION_GPT5_FORCE_MODE=true

# Provider-specific request parameters, a JSON object (optional)
# See Provider-Specific Request Parameters above
TEXT_MODEL_EXTRA_BODY={"reasoning": {"effort": "low"}}
CHAT_MODEL_EXTRA_BODY={"reasoning": {"effort": "low"}}
TRANSLATION_MODEL_EXTRA_BODY={"reasoning": {"effort": "low"}}

Prefix-Cache-Friendly Prompts (PREFIX_CACHE_OPTIMIZED_PROMPTS)¶

Default: false. Opt-in.

For each recording, Speakr makes two LLM calls over the same transcript back-to-back: title generation, then summary generation. By default these two calls use different system messages and different text before the transcript, so they share no common prefix.

Self-hosted, OpenAI-compatible inference servers that implement Automatic Prefix Caching — vLLM (APC), llama.cpp / Ollama (KV reuse), SGLang (RadixAttention), TGI — can skip recomputing a prompt's KV cache when a new request shares a prefix with a recent one. Because the title and summary prompts don't share a prefix today, the expensive transcript prefill is computed twice.

When PREFIX_CACHE_OPTIMIZED_PROMPTS=true, Speakr rewrites the title and summary prompts so they use a byte-identical system message and a byte-identical transcript-first user block, pushing all per-task instructions into the suffix after the transcript. The backend then prefills the transcript once (on the title call) and reuses it on the summary call, prefilling only the short trailing instructions.

When to enable it:

• You self-host the text model on a backend with automatic prefix caching, or you use a hosted model with prompt caching (OpenAI models cache a shared prefix automatically; Anthropic models also need TEXT_MODEL_CACHE_BREAKPOINTS, described below), and
• Your transcripts are large (long diarized meetings), so the transcript dominates the title and summary cost.

When to leave it off (the default):

• You want the exact upstream prompt wording. With the flag off, prompts are unchanged.

Scope is title + summary only. The flag has no effect on chat, event extraction, or other calls.

Enabling prefix caching on your backend¶

This flag only changes how Speakr shapes its prompts so that a shared prefix exists. The caching itself has to be turned on at the inference server, and it is off by default on most backends. The settings below are for vLLM; other servers (llama.cpp / Ollama, SGLang, TGI) have their own equivalents.

• Turn caching on. Start vLLM with --enable-prefix-caching. Without it, the title and summary calls still send a byte-identical transcript prefix, but the server recomputes that prefix every time and the flag delivers no benefit.

• Make the saving visible (optional). vLLM only reports per-request cache hits in the API response when it is also started with --enable-prompt-tokens-details, which defaults to off. With that flag set, each response includes usage.prompt_tokens_details.cached_tokens, which Speakr records and shows in the admin dashboard under System Statistics → Usage and Cost, in the prompt-cache line below the language-model chart. Without it, caching still works, but the dashboard shows zero cache reads because the server never reports them. You can confirm caching independently from vLLM's /metrics endpoint, which exposes vllm:prefix_cache_hits_total and vllm:prompt_tokens_cached_total regardless of this flag.

Example vLLM launch:

vllm serve <model> \
  --enable-prefix-caching \
  --enable-prompt-tokens-details \
  # ... your other flags (--tensor-parallel-size, --max-model-len, etc.)

A note on what to expect: the very first call over a transcript is always a cache miss and reports zero cached tokens; the saving appears on the second call that reuses the prefix.

Cache breakpoints for Anthropic models (TEXT_MODEL_CACHE_BREAKPOINTS)¶

Default: false. Used only together with PREFIX_CACHE_OPTIMIZED_PROMPTS=true.

Anthropic models, called directly or through OpenRouter, cache a prompt only up to an explicit cache_control breakpoint. Without one, the transcript is billed in full on both the title and the summary call.

PREFIX_CACHE_OPTIMIZED_PROMPTS=true
TEXT_MODEL_CACHE_BREAKPOINTS=true

With both settings on, the user message of the title and summary calls is sent as two text parts: the shared transcript block, marked with "cache_control": {"type": "ephemeral"}, and the task instructions. The text of the prompt is unchanged. The title call writes the transcript to the cache, and the summary call a few seconds later reads it at a tenth of the input price. Anthropic charges 1.25 times the input price for a cache write, so a call with no second call within five minutes, such as a summary regenerated on its own, costs slightly more on its transcript part. Prompts shorter than the model's minimum cacheable length (about 1,000 to 4,000 tokens, depending on the model) are not cached.

The cache reads and writes are recorded and shown under System Statistics → Usage and Cost.

Enable this setting only for a provider that accepts a list of text parts in a message and the cache_control field. Through OpenRouter, other models accept the request and ignore the field. Some local and OpenAI-compatible servers reject it; with such a server, the title and summary calls fail until the setting is turned off.

Per-Upload, Per-Tag, Per-Folder Transcription Models¶

By default Speakr uses the single TRANSCRIPTION_MODEL set in .env for every recording. If your users transcribe different kinds of recordings (calls, meetings, dictations, multi-speaker interviews) you can publish a list of models they're allowed to choose from at upload time.

# Comma-separated list of model identifiers users can pick.
TRANSCRIPTION_MODELS_AVAILABLE=whisper-1,gpt-4o-transcribe,gpt-4o-transcribe-diarize,vibevoice
# Optional parallel list of display names. Falls back to the model id when omitted.
TRANSCRIPTION_MODEL_LABELS=Whisper,GPT-4o,GPT-4o (Diarize),VibeVoice

When the list is non-empty, a "Transcription model" dropdown appears in the upload form's Advanced ASR Options and in the reprocess modal. Tags and folders also gain a "Default transcription model" field in their edit forms — set one and any recording uploaded with that tag or in that folder uses the chosen model unless the user picks a different one at upload time.

Resolution order at upload time:

1. Per-upload selection (Advanced ASR Options dropdown)
2. First tag's default_transcription_model
3. Folder's default_transcription_model
4. The default transcription model chosen by an administrator in the admin settings
5. The connector's own model. For the OpenAI, Mistral and VibeVoice connectors this is TRANSCRIPTION_MODEL. The asr_endpoint connector has none: it sends no model parameter, and the ASR service uses its own default.

If the override isn't in TRANSCRIPTION_MODELS_AVAILABLE, it's dropped with a warning in the log and the recording uses the connector's own model, which guards against stale browser caches sending old model ids. The same resolution applies to reprocessing, bulk reprocessing, API v1, watch folders, incognito mode and the startup voice embedding check.

The override is propagated to the connector via the model field on TranscriptionRequest. Connectors that key on a model name (OpenAI Whisper / Transcribe, Mistral, VibeVoice) honour it directly. The asr_endpoint connector forwards the override as a model= query parameter; the whisperx-asr-service fork uses it to switch the loaded Whisper model on demand, while the upstream ahmetoner/whisper-asr-webservice ignores unknown query parameters, so the override is safe in either case.

Admin-Managed Model List with /v1/models Discovery¶

For connectors that expose an OpenAI-compatible /v1/models endpoint (OpenAI, Azure OpenAI, vLLM, and the WhisperX ASR service when configured to advertise its loaded models), admins can curate the available list directly from the Default Prompts → Transcription Models section of the admin dashboard rather than via env var. Click "Refresh from connector" to query /v1/models, tick the models you want users to see, and save.

The DB-backed list (stored in the system_setting table) overrides TRANSCRIPTION_MODELS_AVAILABLE when set. To revert to env-var control, clear the list in the dashboard. The env var is still useful for installations where the connector does not advertise its models or where you want config-as-code behaviour.

When a connector returns a richer model object (display name, description, supported languages), those fields are used to render the dropdown labels. Otherwise the model id is used as both id and label, the same as with TRANSCRIPTION_MODEL_LABELS.

Configurable Embedding Model¶

Speakr's Inquire mode (semantic search) uses sentence-transformers to embed transcript chunks locally by default. The default model is all-MiniLM-L6-v2 (384-dim vectors), which is fast, small, and sufficient for most use cases.

To use a different local model, set:

EMBEDDING_MODEL=all-mpnet-base-v2          # 768-dim, higher quality
# or
EMBEDDING_MODEL=multi-qa-MiniLM-L6-cos-v1  # tuned for question-answering

Any sentence-transformers compatible model name works.

Remote (API-based) Embeddings¶

If you would rather offload embeddings to an OpenAI-compatible HTTP endpoint (vLLM, OpenRouter, OpenAI, Together, or any other compatible provider), set EMBEDDING_BASE_URL and Speakr switches to API mode. The same EMBEDDING_MODEL env var becomes the model identifier sent in each request.

# OpenAI directly
EMBEDDING_BASE_URL=https://api.openai.com/v1
EMBEDDING_API_KEY=sk-...
EMBEDDING_MODEL=text-embedding-3-small
EMBEDDING_DIMENSIONS=768   # optional, only honoured by providers that support output-dim trimming

# Self-hosted vLLM serving an embedding model
EMBEDDING_BASE_URL=http://vllm-host:8000/v1
EMBEDDING_API_KEY=not-needed
EMBEDDING_MODEL=BAAI/bge-large-en-v1.5

# OpenRouter
EMBEDDING_BASE_URL=https://openrouter.ai/api/v1
EMBEDDING_API_KEY=sk-or-...
EMBEDDING_MODEL=openai/text-embedding-3-large

For Voyage AI's official endpoint, use:

ENABLE_INQUIRE_MODE=true
EMBEDDING_BASE_URL=https://api.voyageai.com/v1
EMBEDDING_API_KEY=your-voyage-api-key
EMBEDDING_MODEL=voyage-4-lite
# EMBEDDING_DIMENSIONS=1024  # optional; must be supported by the model

When EMBEDDING_BASE_URL points at api.voyageai.com, requests are sent in Voyage's format: EMBEDDING_DIMENSIONS is sent as output_dimension, recording chunks are sent with input_type=document and searches with input_type=query. Requests to any other host, including a proxy for Voyage, keep the OpenAI-compatible format. No Voyage SDK or local model is required. Meeting text and search queries are sent to Voyage for embedding. If Voyage refuses the configured dimension, indexing fails and searches use text search until the setting is corrected.

After changing embedding providers, models, or dimensions, use Admin → Vector Store → Re-embed all recordings to rebuild existing vectors. Audio does not need to be transcribed again.

API mode does not require sentence-transformers to be installed, so the lightweight Docker image (learnedmachine/speakr:lite) can now run full semantic search by combining EMBEDDING_BASE_URL with any compatible provider. The lite image already ships openai and scikit-learn, so no further dependencies are needed.

Compatibility Note¶

The active embedding identifier (provider plus model) is recorded in system_setting on first startup. If you later change EMBEDDING_MODEL or EMBEDDING_BASE_URL, Speakr logs a warning at startup and Inquire mode will return wrong results until you reprocess affected recordings, because chunks embedded with the previous configuration occupy a different vector space. To rebuild embeddings after a change, reprocess each recording.

Mistral Voxtral Chunking¶

Voxtral handles up to 3 hours per request natively, but the cloud API can time out near that limit on long meeting recordings. To opt into app-side chunking for the Mistral connector:

TRANSCRIPTION_CONNECTOR=mistral
MISTRAL_ENABLE_CHUNKING=true
MISTRAL_MAX_DURATION_SECONDS=7200   # 2 hours; chunks at 80% of this

Diarization across chunks: Mistral doesn't return voice embeddings, so speakers are remapped per chunk (SPEAKER_00 in chunk 1 ≠ SPEAKER_00 in chunk 2). If you need consistent speaker identity across an entire long recording, use gpt-4o-transcribe-diarize (uses known-speaker references) or whisperx-asr-service with embeddings enabled.

Troubleshooting¶

Model Not Responding¶

Check logs for authentication errors:

docker compose logs -f app | grep "LLM"

Common issues:

• Invalid API key
• Model name not available on your plan
• Rate limits exceeded
• Insufficient credits

Poor Summary Quality¶

Try these adjustments:

• Upgrade to a more capable model
• Increase SUMMARY_MAX_TOKENS
• Review and refine custom prompts
• For GPT-5: increase reasoning effort to medium or high

High Costs¶

Reduce costs with:

• Switch to smaller models (gpt-5-nano, gpt-4o-mini)
• Lower token limits
• For GPT-5: reduce reasoning effort to minimal or low
• Use OpenRouter for better rates

Additional Resources¶

• OpenAI GPT-5 Documentation
• OpenAI Chat Completions API
• OpenRouter Documentation
• Custom Prompts Guide
• System Settings

Next: Default Prompts | Back to Admin Guide