Cut the per-token bill and the vendor lock-in: teach an open model your task on one free GPU, then serve it from hardware you control.
You're paying per token for a model that still doesn't match your tone, and every request leaves your infrastructure. This course takes you the other way — from renting a large language model (LLM) to running one you shaped yourself. You start by deciding whether training is even the right lever, or whether a sharper prompt or a retrieval step (RAG: looking facts up at query time) would do the job in an afternoon. Then you pick an open-weights base model, verify its license permits your use, and curate a few hundred clean training examples. You run a real fine-tune using LoRA (low-rank adaptation — training a few million extra weights instead of all several billion), which fits on a single free Colab graphics processor (GPU). You measure the result honestly against the base model on a held-out set, shrink it with quantization so a 14GB model runs in about 4GB, and serve it behind your own application programming interface (API) speaking the OpenAI format, using Ollama, vLLM, or TGI. You finish with Docker packaging, a load test, a cost-per-token figure, and the retraining loop that keeps it getting better.
Built by Lakshya Kumar
Paste this into any AI chat. Fill in the bracketed parts with your context — you'll get back a straight answer on whether this belongs on your plate.
We grant free access case-by-case — students, career-switchers, builders on a tight budget. Sign in to send us a note.
Sign in to applyFinished the tasks? Take the prompt to your AI and get tested on it. We copy the prompt and open the app — just paste it in.
The wrong base model shows up three days in — out of memory, or licensed so you can't ship the thing you just built on it.
The model learns whatever pattern dominates, including your sloppiness — so build a deduped, leakage-free set you can finish this week.
Train a few million extra weights instead of billions, and a 7B model fits in 16GB while the result ships as a few-megabyte file.
Finish a real fine-tune on a free Colab GPU, including the batch-size, learning-rate, and out-of-memory fixes that stall first attempts.
Without a held-out set and a before/after against the base model, you can't tell a real improvement from a hopeful one.
Squeeze a 14GB model into 4GB so it runs on cheap hardware — then measure exactly what quality you gave up before you ship it.
Serve your model behind the exact API shape your app already calls, so leaving a hosted provider is a one-line config change.
Turn a laptop demo into a service that survives real traffic — containerized, monitored, load-tested, with a cost per token you can quote.
Your production traffic is the next training set — version adapters, compare them live, and know when retraining beats a prompt tweak.
Complete all modules, then submit the required number of capstone projects. Each must earn a passing rating from an admin reviewer.
Take an open base model and a custom dataset you built, fine-tune it with LoRA/QLoRA on a single GPU, evaluate it against the best-prompted base on a frozen held-out set, quantize it for deployment, and self-host it behind an OpenAI-compatible API. Submit the adapter, a before/after eval report, the quantized artifact, and the running endpoint with a swap into a small app.
I'm taking a hands-on, single-GPU "Fine-Tuning & Self-Hosting LLMs" course. It covers: deciding when to fine-tune vs. prompt vs. RAG, picking an open base model and checking its license, building a clean dataset, LoRA/QLoRA training on a free Colab/Kaggle GPU with Hugging Face TRL/PEFT, evaluating against the base, quantizing (GGUF/AWQ/GPTQ), and self-hosting behind an OpenAI-compatible API with Ollama/vLLM/TGI, plus production and iteration practices. Here is my situation: - My task / what I want the model to do better: [describe the behavior or format] - My base model (or "help me pick"): [e.g. Qwen2.5-7B-Instruct] - My GPU / where I'll train: [e.g. free Colab T4, 16GB] - My dataset (or what I have to build one from): [describe source + rough size] - Where I'll serve it: [e.g. Ollama on CPU / vLLM on a GPU VM] Act as my fine-tuning coach. First sanity-check whether fine-tuning is even the right tool for my task (vs. a better prompt or RAG). If it is, walk me through the smallest viable LoRA/QLoRA setup that fits my GPU, what dataset to build and how to verify it, how to evaluate against the best-prompted base, which quantization/format to use for my serving target, and how to self-host it. Flag the most likely failure (OOM, overfitting, wrong format, license issue) for my specific situation and how to avoid it.
Build a clean, leakage-free, versioned fine-tuning dataset for a real task from a raw source: clean, dedupe, format as chat messages, split train/val, and optionally augment with filtered synthetic data. Submit the JSONL dataset, metadata/provenance, and a quality report (clean fraction, dedup stats, leakage check, data-scaling note).
Run a complete QLoRA fine-tune on a single (free) GPU and document it: hyperparameter choices (rank/alpha/targets, batch/accumulation/LR), memory budget vs. your GPU, loss curves, any OOM fixes applied, and the saved adapter. Submit the training log, curves, config, and a writeup justifying each decision.
Merge an adapter and quantize the model across at least three levels/formats (e.g. fp16, GGUF q4/q5, AWQ), then measure the size-vs-quality trade-off on a held-out eval and pick a deployable variant. Submit the size-vs-quality table, the chosen format with justification for a stated runtime, and a verification report.
Self-host a model behind an OpenAI-compatible server, containerize it, and swap it into an app that previously used a hosted API — behind a config flag with rollback. Submit the Dockerized server, endpoint tests, a documented capacity + cost-per-token number, and evidence the app's behavior held after the swap.
SFTTrainer is the workhorse of this course's training runs. Read the SFT section before Module 5.