Capstok — learn by doing

Why this matters

There is a clean dividing line that most people learn the hard way: fine-tuning is excellent at shaping behavior — style, tone, output format, following a niche instruction pattern, being concise — and bad at adding knowledge the base model never saw. If you fine-tune a model on 500 facts about your product, it will learn to sound like it knows your product while confidently making facts up, because supervised fine-tuning teaches the shape of answers, not a reliable fact store. Internalizing 'style and format: yes; new facts: no' saves you from the classic disappointment of a fine-tune that talks like an expert and hallucinates like an amateur.

Demo

The demo contrasts two training examples — one teaching a format/style the model can generalize, one trying to teach a specific fact — and explains why the first sticks and the second leaks into hallucination.

Try it yourself

List three things in your task that are behavior (style/format) and three that are knowledge (facts) — only the first list is a fine-tuning target.
Reason about why a fact taught via SFT goes stale, while the same fact in a RAG document can be updated instantly.
Find a borderline case (e.g. a fixed taxonomy of categories) and argue whether it's behavior or knowledge.
Predict what happens if you fine-tune on 50 product facts and then ask about a 51st product it never saw.

Prompt your AI

Use these three in order. Each builds on the one before.

1. Basics & terminology

In plain terms, what kinds of problems does fine-tuning fix well, and what does it fail at?

2. Why it works (the mechanism)

Walk me through why supervised fine-tuning teaches output style and format reliably but is an unreliable way to add new factual knowledge.

3. Advanced — application & what's next

Given a model that needs both a strict JSON output format and access to frequently-updated pricing, explain exactly which part to fine-tune and which to leave to retrieval, and why.

References

# GOOD fine-tuning target: a STYLE/FORMAT the model can generalize.
good = {"messages": [
    {"role": "user", "content": "Summarize this ticket."},
    {"role": "assistant", "content": "TL;DR: <one line>\nNext step: <one action>"}]}
# The model learns the SHAPE 'TL;DR + Next step' and applies it to any ticket.

# RISKY fine-tuning target: a specific FACT.
risky = {"messages": [
    {"role": "user", "content": "What is our refund window?"},
    {"role": "assistant", "content": "Our refund window is 30 days."}]}
# The model learns to ANSWER refund questions confidently -- but if the policy
# changes to 14 days, it keeps saying 30. Facts belong in RAG, not the weights.

for name, ex in [("style/format -> learns to generalize", good),
                 ("specific fact -> will go stale / hallucinate", risky)]:
    print(name, "::", ex["messages"][-1]["content"][:40])

Run: python3 main.py

What fine-tuning can and can't fix