TOUGH TONGUE AI
  • Home
    LogoTOUGH TONGUE AI

    RESOURCES

    Loading...

    HomeLibrarySD Mock · Design a File Uploader for an AI Chat App

    SD Mock · Design a File Uploader for an AI Chat App

    Overview

    Design the document-upload pipeline behind an AI chat app on a live architecture board: drag components, wire them up, and defend your design to an expert interviewer who watches every move.

    Your Instructions

    Design a file uploader for an AI chat app

    A 20-minute mock system design interview. You drive the pace. Jump to the board when you are ready; skip capacity math if you want. The interviewer grades your thinking, not a template match.

    The prompt: a user drops a 200-page PDF into a chat and expects to ask questions about it soon after. Before that can happen, the file has to be stored, scanned, parsed, split into chunks, embedded, indexed, and reported as ready. Design that.

    Two shared surfaces (only one is on screen at a time; use the panel arrow to switch):

    • Notepad. Shared notes. Requirements, capacity math, schema, and end-of-session feedback live here. You can type in it too.
    • Architecture board. Your design canvas. Drag typed components, wire them, label edges, mark scaled tiers as distributed. If it is not on the board, it is not in your design.

    Build incrementally while you talk. Trace one upload end to end through your diagram, from the user's laptop to the first cited answer. Argue the tradeoff, not "industry standard."

    Design choices interviewers probe

    Read this before you go in. These are the decisions, and why they matter.

    Where the bytes go

    Through your API. The obvious first draft: the client streams the file to an upload service. It works for one user. At a bulk-import peak it means your API fleet proxies gigabits of file data for no benefit, and a dropped connection at 90% starts over.

    Straight to object storage. The API validates type and size, then hands back pre-signed URLs, one per part of a multipart upload. The client sends parts directly to the Blob Store and retries only a failed part. The API never touches file bytes. This is the default.

    Xu's Google Drive design sends bytes through block servers that chunk, compress and encrypt. That is right for a sync client doing delta sync. A chat upload has no delta sync, so direct upload is simpler. Knowing why the two differ is a good answer.

    When you say "got it"

    The contract is: never lose an acknowledged upload, and never make the request wait for processing.

    So acknowledge only after the multipart upload is complete in storage and the file row is committed as uploaded. Then return 202 and let processing run on its own. Storage events are delivered at least once and can arrive late or twice, so do not bet the promise on one event: write an outbox row in the same transaction, and run a sweep that finds files stuck in uploaded.

    One job, or one job per stage

    One job per file. Parse, OCR, chunk, embed and index in one worker. If embedding fails at chunk 580 of 600, the retry re-parses and re-OCRs everything. The user sees the bar jump back from 97%. Interviewers call this out by name.

    A stage per step. Each stage (scan, parse/OCR, chunk, embed, index) gets its own queue, worker pool, retry policy and checkpoint. Each writes a durable output for the next: parsed text to the Blob Store, chunks to a table, vectors to the index. A crash repeats one stage.

    Then classify failures. They are not all "retry 3 times":

    • Corrupt or encrypted PDF: not retryable. Fail and tell the user exactly why.
    • OCR timeout on a big scan: retry differently, with a bigger budget on a bigger worker.
    • Embedding 429: backoff with jitter, and rate-limit that stage.
    • A file that crashes the parser every time: small retry budget, then a dead-letter queue with diagnostics.
    • Run parsers sandboxed. PDF and Office parsers are attack surface.

    Replays and duplicates

    Queues and storage events deliver at least once. Every stage runs at least once. So every stage must be safe to repeat.

    • Chunk IDs are deterministic: (file_id, chunk_seq). Writes are upserts.
    • Embedding checks for an existing vector before calling the model.
    • Indexing upserts by chunk ID.

    Replay any stage and you converge on the same 600 chunks, not 1,200.

    Dedup. Hash the file during upload. Ten teammates uploading the same handbook get one processed copy and ten references. When someone uploads a revised version, re-embed only the chunks whose text changed (key reuse by the chunk's text hash, since an insertion shifts every later sequence number).

    Keep dedup inside a tenant. If a different company's upload comes back "ready" instantly, you have told them someone else already has that exact file.

    Progress and partial availability

    "Processing..." is not progress. Job metadata already knows pages parsed and chunks embedded; expose that per stage, by polling or push.

    Can a user ask about page 12 while page 400 is still in OCR? Chunks are searchable the moment they are indexed, so you can allow it. If you do, retrieval returns only indexed chunks, the answer must say coverage is partial, and the ready event clears that. The danger is the confident absence: "the contract does not mention arbitration," said while half the contract is unread. Either answer (allow it, or wait for ready) is fine if you state the retrieval contract.

    Tenants: fairness and isolation

    Fairness. One company imports 50,000 files. Another user uploads one memo. With one FIFO queue, the memo waits hours. Use per-tenant queues with weighted round-robin, per-tenant concurrency caps, and ideally an interactive lane separate from bulk imports.

    Isolation. Every chunk carries tenant_id, file_id and ACL tags. The retriever takes the tenant from the authenticated identity, never from the request or the model's tool call, and checks the user's current permissions at query time (sharing changes after upload). Filter inside the search, or partition the index by tenant: filtering after an approximate nearest-neighbour search can return almost nothing for a small tenant.

    Deletion

    A file becomes an original, parsed text, hundreds of chunks, hundreds of vectors, keyword index entries, and maybe cached answers. Deleting the metadata row leaves all of that behind.

    Hide first: mark it deleted and have the retriever skip it within seconds. Then purge each artifact type with its own idempotent, retried task. With dedup, only purge shared content when the last reference goes. A periodic sweep looks for orphans and alerts on any.

    Embedding-model upgrades

    Vectors from two models live in different spaces. Mixing them quietly ruins retrieval. Keep a versioned index, re-embed into v2 from the stored chunk text (never re-parse a year of files), dual-write new uploads, and switch each tenant once its backfill is done. Queries must embed with the model that matches the tenant's version.

    Reference solution

    One complete design you can study, then rebuild in your own words. Do not recite it. In the interview, draw your graph and defend the choices.

    Requirements (lock these or take the defaults)

    • Upload PDFs, Office docs, text, and scanned images needing OCR, up to 512 MB, resumably
    • Pipeline: scan, parse/OCR, chunk, embed, index; per-stage progress; ready or failed with a reason
    • Retrieval for chat, scoped to what the user can see, with document and page for citations
    • Multi-tenant: bulk imports must not starve others; no cross-tenant retrieval, ever
    • Delete a file and everything derived from it
    • 1M uploads/day, 5 MB average, bulk-import bursts of about 20x
    • Never lose an acknowledged upload; the request path never waits for processing
    • Out of scope: answer generation beyond the retriever interface; desktop sync

    Capacity

    • 1M uploads/day ÷ 86,400 ≈ 12 uploads/sec average; ×20 bursts ≈ 230/sec
    • 1M × 5 MB ≈ 5 TB/day of originals (about 58 MB/s, or ~9 Gbit/s at 20x: the reason bytes skip your API)
    • A 200-page PDF ≈ 600 chunks; assume 100 chunks per file on average → 100M embeddings/day ≈ 1,200/sec sustained
    • At 2,000 chunks/sec per GPU instance: about 1 instance on average, ~12 at a 20x burst. Embedding is the throughput stage, so it sits behind a queue and scales on its own
    • Optional: 1,536-dim float vectors are ~6 KB each → ~600 GB/day of index growth. Partition the index and keep chunk text for re-embedding

    Rough numbers are fine; the method is what gets graded.

    Components to put on the board

    How the graph works: the API is a thin control plane. Bytes go from the client to the Blob Store. A storage event starts a chain of queued stages, each writing a durable output and a checkpoint. The chat's retriever only ever reads the index, always filtered by tenant.

    Web Client, Mobile Client
        → Load Balancer → Upload Service (pre-signed URLs, status)  ── Postgres (files, stage jobs, chunks)
        → Blob Store (direct multipart PUT; originals + parsed text)
            → Pub/Sub Topic (upload-complete event)
            → SQS stage queues (one per stage, per-tenant weighted)
            → Worker pools (mark distributed): Scan → Parse/OCR → Chunk → Embed (ML Model on GPUs) → Index
                → Elasticsearch (vectors + BM25, tagged tenant_id / file_id / acl)
                → Queue (dead-letter)
    Chat Service (agent) → Retrieval Service → Elasticsearch (filter by tenant + current ACLs)
    Cron: reconciliation sweep (stuck uploads) + orphan sweep (deletions)
    

    Components:

    • Web / Mobile Client. Requests an upload, gets pre-signed part URLs, PUTs parts straight to storage, resumes failed parts, calls complete, then watches status.
    • Load Balancer + Upload Service. Validates type and 512 MB limit, checks the content hash for dedup, issues URLs, commits the file row, acknowledges, serves per-stage status. Stateless and small because it never carries bytes.
    • Blob Store. Durable source of truth for originals and parsed text. Its upload-complete event starts the pipeline.
    • Pub/Sub Topic + SQS stage queues. One queue per stage so each scales and retries independently. Per-tenant sub-queues or weights live here. Without them, one tenant's import starves everyone.
    • Workers (distributed). Separate pools because the hardware differs: sandboxed CPU for parsing, bigger boxes for OCR, GPUs for embedding. Each records a checkpoint after its durable write.
    • ML Model. The embedding model on GPU instances, called in batches. Its version is recorded on every vector.
    • Postgres. File rows, stage jobs (status, attempt, checkpoint, error code), chunk text, file references, deletion tasks. Partitioned by tenant.
    • Elasticsearch. Vector plus keyword index for hybrid search; every entry carries tenant, file and ACL tags; partitioned by tenant. (pgvector, OpenSearch or a managed vector DB are equally fine.)
    • Queue (dead-letter). Where poison files land after a small retry budget, with diagnostics for a human.
    • Retrieval Service. The agent's search tool: resolves tenant and permissions from identity, embeds the question with the tenant's model version, searches, returns chunks with page and a coverage flag.
    • Cron. Reconciliation sweep for files stuck in uploaded; orphan sweep that verifies deletions.

    Walk one upload: Priya's 200-page handbook, from laptop to cited answer

    1. The client hashes the file and calls the Upload Service with name, size and hash. No match in Priya's tenant, so it records the file as uploading and returns pre-signed part URLs.
    2. The client PUTs 8 MB parts straight to the Blob Store in parallel. Part 2 fails on bad Wi-Fi; only part 2 is retried.
    3. The client calls complete. The service finalizes the multipart upload, verifies size and hash, commits uploaded plus an outbox row, and returns 202. That is the acknowledgement.
    4. The upload-complete event (or the outbox relay, whichever lands first) enqueues the scan stage. A duplicate is a no-op: jobs are keyed by file and stage.
    5. Scan passes. A sandboxed parse worker extracts text with page numbers, writes it to the Blob Store, updates pages done, and enqueues chunk.
    6. The chunk worker writes 600 chunks keyed (file_id, 0..599) to Postgres. Embed workers batch them to the GPU model, skipping any chunk that already has a vector.
    7. Index workers upsert vectors and keywords into Elasticsearch with Priya's tenant and ACL tags. The last chunk flips the file to ready and pushes a status update.
    8. Priya asks a question. The Retrieval Service resolves her tenant and current permissions, embeds the question, runs a filtered hybrid search, and returns chunks with pages. The answer cites "handbook.pdf, p. 42".

    Sam uploads the same file: the hash matches a ready file in the same tenant. Sam gets a reference and ready immediately; nothing is uploaded or embedded.

    Embedding hits a 429 at chunk 580: only the embed stage retries, with backoff. Parsed text and chunks are already durable; the 580 existing vectors are skipped.

    Priya deletes it: hidden from retrieval within seconds, then per-artifact purges. Sam still references the content, so shared chunks and vectors stay until his reference goes too.

    Schema

    File(file_id, tenant_id, owner_id, name, size, content_hash, blob_key, status, deleted_at)
    StageJob(file_id, stage, status, attempt, checkpoint, error_code, updated_at)
    Chunk(file_id, seq, tenant_id, page, text, text_hash)
    FileRef(tenant_id, user_id, file_id)
    DeletionTask(file_id, artifact, status, verified_at)
    VectorEntry(chunk_id, index_version, vector, tenant_id, file_id, acl_tags)   -- in the index
    
    • Chunk primary key (file_id, seq) makes replays upserts; text_hash enables reuse on revisions
    • Partition metadata and chunks by tenant_id; partition the index by tenant (large tenants get their own shard)
    • Relational for files, jobs and chunks: small rows, transactional status changes, easy to query for the progress UI
    • Replicate storage and metadata; the Blob Store is the durability anchor

    Scaling and wrap-up

    • Upload Service scales on request count, not bytes; the Blob Store absorbs the bytes
    • Each worker pool scales on its own queue depth; GPUs are the pool to watch
    • Index grows by hundreds of GB a day: partition and tier cold tenants
    • First metrics: time to ready (p50/p99 by page count), queue age per stage and tenant, dead-letter rate, orphan count (must be 0)
    • If time: embedding-model backfill, prompt injection in documents, cross-tenant dedup leak, contextual chunk headers for retrieval quality

    Equally defensible variants

    Kafka or RabbitMQ instead of SQS. A workflow engine driving stages instead of each worker enqueueing the next. pgvector, OpenSearch or a managed vector DB instead of Elasticsearch. Polling instead of push for status. All-or-nothing ready instead of partial availability. What matters is that you can argue the tradeoff.

    More in System Design Interview Prep: AI Coach Edition

    • SD Coach · Media Streaming
    • SD Mock · Design YouTube / Netflix
    • SD Mock · Design Instagram
    • SD Coach · Agentic AI
    • SD Coach · Agentic AI Architectures (Slides)
    • SD Mock · Design ChatGPT
    • SD Mock · Design Claude Code
    • SD Mock · Design an AI Customer Support System
    • SD Mock · Design a Real-Time Voice AI
    • SD Mock · Design an LLM Query Batching System
    • SD Coach · Reliability and Security
    • SD Mock · Design It Live
    View the full course (43 sessions) →