Skip to main content

​
Why Ollama?

  • Completely free — no API costs
  • Privacy — no data sent to external services
  • Air-gapped — works without internet access
  • No API key needed

​
Setup

  1. Install Ollama from ollama.com/download
  2. Pull a model:
ollama pull llama3.2        # 3B — fast, good reasoning (~2GB)
ollama pull llama3.1:8b     # 8B — better quality (~8GB RAM)
ollama pull mistral          # 7B — strong analysis (~8GB RAM)
  1. Verify Ollama is running:
ollama list
  1. Configure in .langsight.yaml:
investigate:
  provider: ollama
  model: llama3.2    # default
No OLLAMA_API_KEY needed.

​
Models

ModelRAMQualitySpeed
llama3.22 GBGoodFast
llama3.1:8b8 GBBetterMedium
mistral8 GBGoodMedium
qwen2.5:14b16 GBExcellentSlow

​
Remote Ollama

If Ollama runs on another machine: 
investigate:
  provider: ollama
  model: llama3.2
  base_url: http://my-gpu-server:11434/v1

​
Troubleshooting

ConfigError: Ollama request failed: Connection refused
→ Start Ollama: ollama serve 
ConfigError: model not found
→ Pull the model: ollama pull llama3.2