Skip to main content

LLM Model Configuration

LLM (Large Language Model) settings control the AI model used for RAG chat responses in the Playground.

Available Providers​

ProviderModelsNotes
OpenAIGPT-4o, GPT-4o MiniMost popular, excellent quality
AnthropicClaude Sonnet 4, Claude 3.5 Sonnet, Claude 3.5 HaikuLong context, nuanced responses

Configuration Options​

  1. Provider: Select OpenAI or Anthropic
  2. API Key: Enter your provider's API key
  3. Model: Choose the specific LLM model
  4. Temperature: Adjust creativity (0.0 = focused, 2.0 = creative)
  5. Max Tokens: Limit response length

API Key Management​

Each provider requires its own API key:

The settings page shows a status banner indicating whether your API key is configured:

  • Green checkmark: API key is configured and ready
  • Yellow warning: API key required - enter your key to enable chat

Choosing an LLM Model​

ModelContextBest ForQuality
GPT-4o128K tokensComplex reasoning, detailed responsesHighest (OpenAI)
GPT-4o Mini128K tokensFast, cost-effective responsesHigh
Claude Sonnet 4200K tokensLong documents, nuanced analysisHighest (Anthropic)
Claude 3.5 Haiku200K tokensFast responses, simple queriesGood

Temperature Guide​

TemperatureBehaviorBest For
0.0 - 0.3Focused, deterministicFactual Q&A, technical docs
0.4 - 0.7BalancedGeneral use (default: 0.7)
0.8 - 1.2Creative, variedContent generation
1.3 - 2.0Highly creativeBrainstorming, exploration