Skip to content

Getting Started

Prerequisites

  • Rust (stable toolchain)
  • Docker and Docker Compose
  • Python 3.13+ (for the evaluation framework)
  • An OpenAI API key (or any OpenAI-compatible API)

1. Configure Environment Variables

Copy the example env file and fill in your API keys:

cp .env.example .env

Edit .env with your values:

# LLM for the chatbot
OPENAI_API_KEY=sk-proj-...
LLM_BASE_URL=https://api.openai.com/v1
LLM_MODEL=gpt-4o

# LLM for evaluation judges (can be the same or different)
JUDGE_API_KEY=sk-proj-...
JUDGE_BASE_URL=https://api.openai.com/v1
JUDGE_MODEL=gpt-4o

# Langfuse (pre-seeded by docker-compose)
LANGFUSE_PUBLIC_KEY=pk-local-dev-chatbot
LANGFUSE_SECRET_KEY=sk-local-dev-chatbot

Then source it:

source .env

2. Start Infrastructure

Docker Compose brings up Langfuse and the three MCP servers:

docker compose up -d

This starts:

Service Port Purpose
Langfuse 3000 Observability UI and API
Langfuse Worker 3030 Async job processing
PostgreSQL 5432 Langfuse database
ClickHouse 8123 Langfuse analytics
Redis 6379 Langfuse cache
MinIO 9090 S3-compatible storage
mcp-server-fraud 3001 Fraud detection tools
mcp-server-kyc 3002 KYC verification tools
mcp-server-funding 3003 Deposit/withdrawal tools

Langfuse is pre-seeded with a dev organization, project, and API keys so it works immediately.

Note

The MCP servers are built from the Rust workspace via a multi-stage Dockerfile. The first build will take a few minutes.

Run everything in Docker

To run the chatbot and eval suite entirely in Docker (no local Rust or Python needed): 

docker compose --profile integration up -d
See Docker Integration Suite for details.

3. Run the Chatbot

HTTP Server Mode

Start the chatbot as an HTTP server:

cargo run -p chatbot -- serve

The server listens on port 8080 (configurable in chatbot.toml). Send messages via the API:

curl -X POST http://localhost:8080/chat \
  -H "Content-Type: application/json" \
  -d '{
    "session_id": "session-1",
    "user_id": "test_user_3",
    "message": "Hi, I made a deposit but it is not showing up in my account"
  }'

CLI Mode

For interactive use, run in CLI mode:

cargo run -p chatbot -- cli

This starts a REPL where you can type messages directly.

4. Run Evaluations

Set up the Python environment:

cd evals
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Then run the eval pipeline:

# Push golden datasets to Langfuse
python run_eval.py sync

# Run all test cases and score them
python run_eval.py run --run-name "my-first-eval"

# View pass/fail summary
python run_eval.py report

See Evaluation Framework for details.

5. View Results in Langfuse

Open http://localhost:3000 and log in:

  • Email: dev@localhost
  • Password: password

You'll see traces for every conversation turn and scores from the evaluation judges.

Configuration Reference

The chatbot is configured via chatbot.toml:

[llm]
base_url = "https://api.openai.com/v1"
api_key_env = "OPENAI_API_KEY"   # env var name, not the key itself
model = "gpt-4o"
temperature = 0.3

[server]
port = 8080

[langfuse]
base_url = "http://localhost:3000"
public_key_env = "LANGFUSE_PUBLIC_KEY"
secret_key_env = "LANGFUSE_SECRET_KEY"
enabled = true

[[mcp_servers]]
name = "fraud"
url = "http://localhost:3001/mcp"

[[mcp_servers]]
name = "kyc"
url = "http://localhost:3002/mcp"

[[mcp_servers]]
name = "funding"
url = "http://localhost:3003/mcp"

Tip

The api_key_env, public_key_env, and secret_key_env fields reference environment variable names, not literal values. This keeps secrets out of the config file.