Small model vs frontier model, same task
Judge honestly where a small local model is good enough and where a frontier model earns its cost and its data-sharing trade-off — the right-sizing call at the heart of Day 4.
Start from a workspace with documents already loaded, so the corpus and the questions stay identical. Only the model changes.
Run five questions against your local model and keep the answers: one factual lookup, one multi-step reasoning question, one summarisation, one where the corpus does not have the answer, and one asking for structured output such as JSON or a table.
OPTIONAL — only if you have your own API key and are willing to spend a few cents: switch the workspace LLM provider to Anthropic or OpenAI and paste your key.
Re-run the identical five questions. Change nothing else.
Score each pair yourself: which answers were actually better, and by how much?
When you are done, remove the key from the app and revoke it at the provider if the workshop is over.
What to notice
- On lookup and summarisation over your own documents, a good small model is often indistinguishable — retrieval is doing the work, not the model.
- The gap opens on multi-step reasoning and on strict output formats. That is where paying for a frontier model buys something real.
- On the unanswerable question, watch which model abstains. Confident fabrication is a capability failure that size does not reliably fix.
- The moment you switch providers, every question AND every retrieved chunk leaves your machine. That is the whole local-vs-hosted trade-off, and it is a compliance decision before it is a technical one.