Week 8 · Mon Sep 14, 2026 · Sep 14-20, 2026

Day 1: Web Crawler CORE HLD

A frontier-owned crawler design with explicit visited state, raw/object storage, parse/index stages, and durable restart points.

Where does crawl progress live after every crash?Frontier · leases · retries · dedupe · recovery3 Heap/Priority Queue slots

Exact Time Blocks

Follow the Week 4 delivery pattern: blank-page first, source reading second, DDIA decision third, spoken close, contiguous algorithm practice, evening recall.

Time	Block	Required output
08:30	Wake + setup	Open this page, start the board, write the day question: Where does crawl progress live after every crash?
08:50—09:00	Board set	List entities, queues, leases, retry states, dedupe keys, metrics before reading.
09:00—10:05	Blank-page attempt + Hello reading	Draw the crawler before reading: URL submission, normalization, frontier partition, fetcher, robots/politeness guard, raw page store, parser, URL extractor, dedupe, index sink. Then read the Hello setup through HLD and mark exactly where each piece lives.
10:05—10:45	Deep dive + DDIA decision	The ownership move is the frontier: workers borrow URL leases; canonical URL and content hash dedupe decide whether a URL can progress; the raw page pointer is the stable handoff between fetch and parse. If the queue is the source of truth, replay must be bounded by a durable visited/frontier table.
10:45—10:55	Spoken close	Speak the contract: at-least-once fetch is acceptable only when URL/content dedupe and idempotent index writes make retries harmless.
14:30—16:15	NeetCode Heap/Priority Queue	Complete contiguous slots W8-H01—W8-H03; capture invariant and bug risk for each.
20:30—21:15	Recall / Q&A / mock	Use Staff Q&A, lecture assets, or mock script for the day's system.

Canonical Heading Anchors

Read only the listed Hello Interview headings for today's assignment. Every link is a direct day-page anchor.

Understanding the ProblemDay 1

CORE scope and requirements gate

Functional RequirementsDay 1

CORE functional contract

Non-Functional RequirementsDay 1

CORE scale, availability, politeness constraints

The Set UpDay 1

CORE setup and interviewer framing

Planning the ApproachDay 1

CORE planning spine

API or System InterfaceDay 1

CORE crawl submission and status surface

Data FlowDay 1

CORE fetch/extract/index/event flow

High-Level DesignDay 1

CORE architecture

Decision Notes

CORE means you must answer requirement, HLD, deep dive, failure, recovery, metric, and Staff-level tradeoff.
TRANSFER means blind attempt first, then delta against source, invalid assumption, new bottleneck, and follow-up pressure question.
Do not add open-ended AI radar unless it changes a concrete Week 8 design decision; this week it does not.

DDIA Decision Map

Use exact sections only from Ch5, Ch9, and Ch12.

Chapter	Exact DDIA decision section	Concrete Week 8 use
Ch5	持久化执行与工作流	Every fetch, parse, and index stage records status, input pointer, output pointer, attempt, and next action.
Ch5	持久化执行	Do not trust in-memory worker progress; write progress before acknowledging frontier work.
Ch5	事件驱动的架构	Use topic boundaries for fetched page, extracted URL, and indexable document events.

Heap/Priority Queue Slots

Three contiguous slots today. Keep one invariant and one failure mode per problem.

Slot	Problem	Pattern	Invariant / bug risk	Time · Space
W8-H01	Kth Largest Element in a Stream	min-heap of size k	Heap owns the retained frontier; never sort the stream on every insert.	O(log k) · O(k)
W8-H02	Last Stone Weight	max-heap simulation	The next state is defined only by the two largest stones.	O(n log n) · O(n)
W8-H03	Kth Largest Element in an Array	bounded min-heap	Keep only candidates that can still affect the answer.	O(n log k) · O(k)

Evidence To Save

One compact artifact is enough if it proves mechanism depth.

Invariant

State safety

Name the state that survives restart and the write that makes retry safe.

Failure

Concrete break

Use a worker crash, zombie lease, redelivery, poison message, stale robots decision, or duplicate price event.

Metric

Operational proof

Attach lag, retry rate, duplicate suppression, stale lease reject count, host politeness violation, or alert freshness.

Local study materials
Detailed lecture notes, audio, recall scripts, PDFs, Staff Q&A, and mock packs are archived locally and are intentionally not published on this site.