Train your own model — the weekend training loop

By the end of this tutorial you will have captured production traffic from a running agent, fine-tuned a 3B open-source model on it, and deployed the result on your own hardware. The fine-tune itself is a short GPU job — free on Colab, or a few dollars of rental time elsewhere; what decides whether the model is any good is the data you capture and how you evaluate it. A model served locally has no per-token API bill.

Before you start

• A Sagewai project with at least one running agent — the first agent guide gets you there in ten minutes.
• A Hugging Face account (free) — Unsloth pulls the base model from HF.
• Access to one GPU tier:
  • A Google account for free Colab T4, or
  • A Vast.ai, RunPod, or Modal account (~$0.20–0.70/hr), or
  • An Apple Silicon Mac with 16 GB+ RAM for the mlx-lm path.
• Python 3.10+ and pip install "sagewai[training]".

How the loop works

Your agent calls a cloud LLM in production. Sagewai records each call as a structured JSONL record. Once you have around 1,000 records, run an Unsloth fine-tune on that JSONL. Unsloth finishes in 30–60 minutes on any of the GPU tiers listed above and saves a LoRA adapter (~50 MB) to models/. Load that adapter into Ollama or mlx_lm.server and point your agent at the local endpoint. The cloud spend stops growing.

Step 1 — Capture production traffic

Run Example 25 — training data pipeline. It fires a few agent calls and writes each one to ~/.sagewai/training/<agent-name>.jsonl. Confirm the output looks right:

head -1 ~/.sagewai/training/your-agent.jsonl | jq .

Each record contains the instruction, input, output, model name, and timestamp. Let the agent run in production for a week or two until you have at least 1,000 records — that is the minimum for a fine-tune worth deploying.

Step 2 — Close the loop (optional)

To trigger fine-tuning automatically once the dataset reaches a threshold, run Example 36 — autopilot training loop. It watches the dataset and kicks off a fine-tune job when the threshold is crossed. Skip this step if you prefer to trigger fine-tunes manually.

Step 3 — Pick a GPU tier

Each example provisions the GPU, syncs the dataset, and tears down on completion or failure.

Step 4 — Run the fine-tune

Run Example 38 — Unsloth fine-tune. It fine-tunes Qwen2.5-3B-Instruct on your captured dataset, prints the loss curve, and saves a LoRA adapter to models/.

Expected wall-clock on the GPU tiers above: 30–60 minutes for ~1,000 examples, 3 epochs.

Step 5 — Deploy locally

Pick your deploy target:

• Anywhere — Ollama. Convert the LoRA to GGUF, write a Modelfile, and run ollama create. Sagewai's harness auto-discovers Ollama on startup.
• Apple Silicon — mlx-lm. Run Example 38a. mlx_lm.server uses Metal directly; no Docker layer needed.
• Custom endpoint. Run Example 46 to wrap any HTTP completion endpoint as a Sagewai-callable tool — on-prem, customer-hosted, or air-gapped.

Once the local model is running, point your agent at it:

from sagewai import UniversalAgent, providers

agent = UniversalAgent(
    name="legal-reviewer-v2",
    **providers.ollama("legal-llm"),
)

response = await agent.chat("Review this non-compete clause...")

Cloud handles novel queries; the local model handles the patterns it has seen before.

Where you'd use this

The capture-accumulate-fine-tune-deploy loop suits four common situations.

SaaS support triage

You shipped a triage agent on Claude Haiku. By Q2 you're triaging 12,000 emails a month at $0.0007 per call.

E-commerce product description generation

Your catalogue has 50K SKUs. You're generating descriptions on GPT-4o at $0.0024 per SKU — $120/month and climbing.

Healthcare-compliant note summarisation

Your scribe app summarises clinical notes. HIPAA forbids sending PHI to OpenAI without a BAA.

Internal knowledge-base Q&A on engineering wikis

Your platform team answers "why is X failing?" questions across 200 services and a 5K-page Confluence. Cloud LLM via RAG costs $300/month.

How Sagewai protects your training data

• Capture writes to ~/.sagewai/training/ on the host you control. Conversations never leave your infrastructure.
• The fine-tune runs on a GPU you rent or own. The dataset is uploaded to that GPU and torn down with it.
• Per-project isolation: each project's captured dataset is scoped to its project ID. Agents in project A cannot read training data from project B.
• The deployed model is served from your own Ollama, mlx-lm, or custom endpoint. No data crosses to a model provider after deployment.

What you're responsible for

• Redact PII before fine-tuning if your dataset contains regulated data. The export step has a strip_pii=True flag — verify it removed what you expect.
• Watch the GPU rental meter. Vast.ai and RunPod bill until you tear the pod down. The example scripts include cleanup-on-failure, but a hung process will still accumulate charges.
• Check the base model licence. Llama, Mistral, and Qwen have different commercial-use terms.
• Back up the LoRA file (~50 MB). It is the only artifact you need to redeploy.

Companion examples

See also

• Training & fine-tuning guide — the full SDK reference for capture, export, fine-tune, and serve.
• Inference deployment — the deployment paths in detail, with a decision table for each one.
• Observability and cost — measure the $/call delta after the local model is live.
• Local LLM routing and multi-model — foundation primitives for routing between cloud and local.
• Inference overview — when each GPU path wins, with explicit decision guidance.