Generation controls, model choice and cost
Comes up in: AI engineering, AI system design
This concept is being written. It gets the full five steps — explanation, predict, interview angle, practice with tests, and review cards. Here is exactly what it will cover, so you can study it elsewhere in the meantime.
What it covers
- sampling concepts (temperature, top-p) and that some current APIs no longer accept them (facts: Claude API rejects them from 2026-10-21
- use effort controls instead)
- max tokens
- choosing model tiers (small/fast vs large)
- price per million tokens
- latency (TTFT, tokens/s)
- a model-selection checklist
Words you will know
What you will practise
- cost calculator for a feature at 3 traffic levels
Interview questions this answers
- How do you choose between a small and a large model for a feature?