Sampling parameters and a model A/B
Turn "the model felt random / slow / dumb" into numbers you can defend: determinism versus temperature, and size versus speed on your own hardware.
0 of 6 done
Load a small model (roughly 1–3B parameters) in LM Studio.
Set temperature to 0, then send the same prompt five times, starting a fresh chat each time so history does not change the input.
Raise temperature to about 1.0 and repeat the same five runs.
Try the same at a low top_p (e.g. 0.1) versus the default, with temperature held constant.
Note the tokens-per-second LM Studio reports for each response.
Download a larger model in the same family, ask it the identical set of questions, and record tokens/sec and answer quality side by side.
What to notice
- At temperature 0 the outputs are effectively identical run to run. That reproducibility is what makes a model testable — and why security tooling pins it.
- At higher temperature the same prompt diverges. Useful for brainstorming, unacceptable for a control that must behave the same every time.
- top_p and temperature both narrow or widen the candidate pool, in different ways.
- The bigger model is slower per token but often needs fewer attempts. Right-sizing is a measured trade-off, not a reflex to pick the biggest.