TOUGH TONGUE AI
  • Home
    LogoTOUGH TONGUE AI

    RESOURCES

    Loading...

    HomeLibraryDS Coach · Experimentation

    DS Coach · Experimentation

    DS Coach · Experimentation

    Overview

    Read the Experimentation notes? Now practise the framework out loud. Coach Maya walks one example through the eight steps of an A/B test answer on her live board, where your answers land as you give them: clarifying questions, north star, primary and guardrail metrics, the randomization unit, sample size, peeking, novelty, sample ratio mismatch, multiple testing, and what to run when an A/B test won't work. Six quick checks, then you design a test yourself with Maya playing the product manager.

    Your Instructions

    Your goal: answer "how would you test this?" with a structure you can reuse on any product.

    How this works (about 20 minutes)

    • Maya's live board walks the eight steps; your answers land on it in your own words, and an experiment ticket fills as you go. Tap to predict when she asks for a guess.
    • Six quick multiple-choice checks along the way.
    • Final drill (about four minutes): you design a test for a new feature on the notepad while Maya plays the product manager. Then you compare with a model design.

    What scores well

    • Clarifying the goal before naming metrics
    • Metrics in four layers, with at least one guardrail that could really get worse
    • Choosing the randomization unit by asking whether users can affect each other
    • Naming peeking, novelty, SRM and multiple testing without being asked

    Tips

    • Answer in your own words; there is no script to match.
    • In the drill, ask Maya clarifying questions, just as you would in the interview.
    • Interrupt any time: "repeat that", "go back", "why?"

    More in Data Science Interview Prep: AI Coach Edition

    • DS Mock · Regression
    • DS Coach · Hypothesis Tests & Confidence Intervals
    • DS Mock · Hypothesis Tests & Confidence Intervals
    • DS Coach · Power Analysis & Impact Sizing
    • DS Mock · Power Analysis & Impact Sizing
    • Data Science: A/B Test Sample Size
    • DS Mock · Experimentation
    • Data Science Interview: A/B test question
    • Data Science: A/B Testing Interview Question (1)
    • Data Science: A/B Testing Question (2)
    • A/B Testing: Google Homepage
    • PM Interview: A/B Test - Facebook Ads Revenue
    View the full course (49 sessions) →