Pick a depth. Each prompt opens in your AI pre-loaded with the lesson. Click a row to preview the prompt.
Hybrid retrieval gives you top-50 candidates. The LLM only sees top-5. The ordering matters enormously. A reranker (Cohere Rerank-3, Voyage Rerank-2, BGE Reranker self-hosted) is a cross-encoder model that scores (query, doc) pairs precisely. Costs ~$1/1K queries managed; usually lifts answer accuracy 10-25%.
Workflow: hybrid → top-50 → fetch chunk text → rerank → top-5 → feed to LLM. Latency: ~80-150ms for reranking 50 docs. Cost: ~$0.001/query for Cohere. Self-hosting BGE Reranker on a small GPU box ($50-100/mo) is free at scale. For the project, Cohere/Voyage is the fastest path.
import cohere
co = cohere.Client(os.environ["COHERE_API_KEY"])
async def rerank(query, candidate_ids, k=5):
# fetch text
rows = db.query("SELECT id, text, metadata FROM chunks WHERE id = ANY(%s)", (candidate_ids,))
chunks_by_id = {r["id"]: r for r in rows}
docs = [chunks_by_id[i]["text"] for i in candidate_ids if i in chunks_by_id]
r = co.rerank(
model="rerank-english-v3.0",
query=query,
documents=docs,
top_n=k,
)
# map back to chunk records preserving rerank order
ordered_ids = [candidate_ids[item.index] for item in r.results]
return [chunks_by_id[i] for i in ordered_ids]
async def search(query, k=5):
candidates = await hybrid_search(query, k=50)
return await rerank(query, candidates, k=k)python3 main.py