LLM Models: Cloud vs Local

Choosing an LLM is a trade-off between cost, speed, privacy, and capability. This guide covers all major options.

Cloud APIs (Hosted Models)

Anthropic Claude

Strengths:

• Best reasoning ability (Opus)
• Excellent code understanding
• Handles ambiguous prompts well
• Good at following complex instructions

Weaknesses:

• Most expensive for volume usage
• Opus slower than GPT-4
• No fine-tuning available

Use when: You need accuracy over cost (analysis, code review, strategy)

OpenAI GPT

Strengths:

• Fastest API response time
• Cheapest for raw throughput (3.5)
• Largest ecosystem (integrations everywhere)
• Fine-tuning available

Weaknesses:

• GPT-4 expensive for high volume
• 3.5 lower quality than recent alternatives
• API sometimes rate-limited

Use when: You need speed and integrations (customer chat, high-volume tasks)

Google Gemini

Strengths:

• Longest context (1M tokens = 100+ documents)
• Cheapest option (Flash often free)
• Good multimodal (vision + text)
• Fastest inference

Weaknesses:

• Reasoning weaker than Claude/GPT-4
• Smaller ecosystem
• Free tier limited to 15 requests/min

Use when: You have large documents or tiny budget (summarization, extraction)

Other Cloud APIs

Meta Llama via API (AWS Bedrock, Together AI)

• 70B: Best free option quality-wise
• Cost: $0.70/$0.95/1M tokens (Together AI)
• Good for cost-sensitive use

Mistral (via Mistral API)

• 8B: Ultra-fast, mobile friendly
• Cost: $0.14/$0.42/1M tokens
• Low latency (<100ms)

Self-Hosted Models (Local)

Llama 2/3

Strengths:

• Truly open source
• Can fine-tune locally
• Low cost after initial GPU investment
• Good reasoning for size

Weaknesses:

• Slower than cloud (tokens/sec)
• Requires 16GB+ GPU
• Lower quality than Claude/GPT-4
• No external knowledge (static training data)

Use when: High-volume inference, privacy critical, fine-tuning needed

Mistral

Strengths:

• Very fast (2x Llama 7B)
• Good code understanding
• 8B: best small model
• Extended context (32K)

Weaknesses:

• Smaller context than Gemini
• Quality gap vs closed-source models
• MoE version needs more VRAM

Use when: Speed matters, 7B sufficient (customer support, content generation)

Qwen (Alibaba)

Strengths:

• Best for Chinese language
• Very small models available (1.8B)
• Long context versions
• Competitive with Mistral at same size

Weaknesses:

• License restrictions for commercial
• Less community adoption
• Harder to find good prompts

Use when: Chinese language needed, tiny models for edge devices

Gemma (Google)

Strengths:

• Tiny model (2B) for edge devices
• Clean architecture, easy to run
• Good for its size
• Google-backed quality

Weaknesses:

• Smaller context than Mistral/Llama
• Less adoption/community
• 7B: similar cost to Mistral, slower

Use when: Mobile/edge device, tiny VRAM budget

Comparison Table

Cost Analysis

Scenario 1: Customer Support (1M prompts/month)

Cloud (GPT-3.5):  1M × $0.0005 = $500/month
Cloud (Claude):   1M × $0.003  = $3,000/month
Self-hosted 7B:   1 × $3000 GPU amortized over 12 months = ~$250/month

Winner: Self-hosted at scale

Scenario 2: Ad-hoc Analysis (10K prompts/month)

Cloud (Gemini):   10K × $0.05/1M = $0.50/month
Cloud (Claude):   10K × $3/1M    = $30/month
Self-hosted:      Fixed $3000/12 = $250/month

Winner: Cloud (Gemini)

Decision Tree

Start here: Do you have a GPU?

No GPU → Use Cloud

1. Budget < $100/month?

   • Use: Gemini Flash (mostly free)
   • Task: Summarization, extraction

2. Need reasoning?

   • Use: Claude Opus (best reasoning)
   • Task: Analysis, strategy, code review

3. Need speed?

   • Use: GPT-4 Turbo
   • Task: Customer chat, real-time

4. Balanced?

   • Use: GPT-3.5 (cost) or Mistral API (speed)

Yes, GPU (8GB) → Consider Self-Hosted

1. High volume (>100K prompts/month)?

   • Use: Mistral 7B or Llama 7B
   • ROI breaks even after ~5,000 prompts

2. Privacy critical?

   • Use: Llama 70B (best local quality)
   • Only if 48GB VRAM available

3. Speed important?

   • Use: Mistral 8B (fastest 8B model)
   • 40 tokens/sec on 3090

4. Tiny device (mobile)?

   • Use: Qwen 1.8B or Gemma 2B
   • Quantized to 4-bit

Running Models Locally

Ollama (Easiest)

# Install: ollama.ai
# Run Mistral
ollama run mistral

# Or in docker-compose
docker-compose up ollama

Models run at: http://localhost:11434

vLLM (Fast Serving)

pip install vllm

# Serve Llama 70B with 4-bit quantization
vllm serve meta-llama/Llama-2-70b-chat-hf \
  --quantization awq \
  --tensor-parallel-size 2

LM Studio (GUI)

• Download: lmstudio.ai
• Browse models, download, chat
• No terminal required

Hybrid Approach (Recommended)

Use both cloud and local:

User Request
  ├─ Small/fast → Local Mistral 7B (instant)
  ├─ Complex/slow → Cloud Claude (wait)
  ├─ Long document → Gemini Flash (free)
  └─ Time-sensitive → GPT-3.5 (quick)

In n8n:

Trigger
  ├─ If prompt < 100 tokens: Use local Ollama
  ├─ Else if document > 10K tokens: Use Gemini
  └─ Else: Use Claude

Checklist

• Identify your constraints (budget, latency, privacy)
• Estimate monthly prompt volume
• Check available GPU VRAM
• Test 2-3 models locally (Ollama)
• Benchmark cloud APIs (cost vs quality)
• Design hybrid strategy
• Document model selection in project README
• Set up fallback (if primary model fails)

Sources and Links

• Hugging Face Models