Your goal: hold one level across an AI-deployment question and a practical build, the way an AI-lab loop is decided.
How this works (about 43 minutes)
- Round 1 (about 23 minutes): Ryan plays the customer, an insurer that wants an AI agent for claim emails. Ask what you need; he gives facts one at a time. The details are made up for the exercise.
- Round 2 (about 18 minutes): a practical build. Talk it through first; the notepad is there if you want to write code or pseudo-code. Ryan reads it when you point at it and never writes the solution.
- Ryan tells you when he switches. This is a practice room in the style of an AI-lab loop, built from what candidates report, not any company's real process.
- No hints and no scores during the call. Your report comes after.
What scores well
- The people, the outcome and the workflow before the model
- An eval set and a ship bar before launch, and a staged rollout with a human in the loop
- Naming the failure modes: made-up policy details, prompt injection in an email, rare claim types
- Cost and latency, and the tradeoff when data cannot leave
- Clear, tested code, and edge cases said aloud
Tips
- Take the LLM and RAG, evals and AI-assisted coding coaches first.