The AI system design interview
Comes up in: AI system design
This concept is being written. It gets the full five steps — explanation, predict, interview angle, practice with tests, and review cards. Here is exactly what it will cover, so you can study it elsewhere in the meantime.
What it covers
- what changes from classic system design: model choice, context construction, retrieval, evals, cost, latency, safety, human oversight
- requirements questions specific to AI (accuracy target, risk of wrong answers, data sensitivity, languages)
Words you will know
What you will practise
- requirements checklist drill
Interview questions this answers
- Design a customer-support assistant that answers from the company's documentation.
- How would you evaluate a RAG system before and after launch?
- Design an internal LLM gateway for many teams.
- Design an agent that can issue refunds. What safeguards do you add?
- How do you control the cost and latency of an LLM feature at scale?