TOUGH TONGUE AI
  • Home
    LogoTOUGH TONGUE AI

    RESOURCES

    Loading...

    HomeLibraryGen AI Mock · Serving a Chat Assistant at Scale

    Gen AI Mock · Serving a Chat Assistant at Scale

    Gen AI Mock · Serving a Chat Assistant at Scale

    Overview

    A serving round with Felix: design a ChatGPT-style assistant and defend the trade-offs between speed, cost and quality. Size it, route it, cache it, roll it out, then handle a launch spike and a router that saves money and loses quality. Written report after.

    Your Instructions

    Your goal: design a chat service you can size, speed up and ship in stages, and make a call when saving money costs quality.

    How this works (about 30 minutes)

    • Felix asks which role you are interviewing for (product, engineering or other) and pitches his follow-ups to it. The question stays the same.
    • One design question, taken deep. Ask him about users, traffic, models and goals. He hands over what he has, one line at a time, and later a latency and cost sheet. All numbers are made up and illustrative; none is a real price or speed.
    • Many follow-ups, one curveball, then a trade-off turn from your own routing choice.
    • If you want to sketch, say "can I use the notepad?" and Felix puts it up.
    • No hints and no scores during the call. Your report comes after.

    What scores well

    • Questions about users, traffic, latency targets and quality before you draw
    • A request path with streaming, state, and capacity sized in tokens, not only requests
    • A model choice by request type, with a router and an escape for its mistakes
    • Caching and batching with their costs named
    • A staged rollout with slice gates, a stop rule and a way back
    • A call you can defend, and what would change it
    • Answers that say where they are going and land in a minute or two

    Tips

    • Take Gen AI Coach · Shipping LLMs first.
    • Say your numbers aloud and label them as assumptions.

    More in Gen AI Interview Prep: AI Coach Edition

    • Gen AI Mock · RAG Over Private Data
    • Gen AI Coach · Responsible AI
    • Gen AI Mock · Agentic AI in High-Stakes Settings
    • Gen AI Coach · Agents and Tools
    • Gen AI Mock · An Agent That Adapts to New Tasks
    • Gen AI Coach · Shipping LLMs
    • Gen AI Coach · AI Product Sense
    • Gen AI Mock · File Upload for an AI Chat App
    • Gen AI Coach · Vibe Coding
    • Gen AI Mock · A Live Prototype with an AI Tool
    • Gen AI Mock · Google Room
    • Gen AI Mock · Meta Room
    View the full course (40 sessions) →