hard google meta · part of Practice Questions · Senior SWE Roadmap
Requirements to clarify
- Functional: count ad clicks aggregated by dimensions (ad ID, time window, geography) for billing and reporting.
- Non-functional: extremely high event volume, aggregation must be accurate (billing depends on it — undercounting or overcounting both matter), must handle late-arriving and out-of-order events.
Core components
- Streaming ingestion: click events flow through a message queue/stream (e.g., partitioned by ad ID) to a stream-processing layer rather than hitting a database directly (see Message Queues & Event-Driven Architecture).
- Stream aggregation: a stream processor (windowed aggregation) counts clicks per ad per time bucket (e.g., per minute), maintaining running counts in a fast store.
- Watermarks for late events: define how long to wait for late/out-of-order events before finalizing a window’s count — a core streaming-systems concept trading completeness against latency.
- Exactly-once counting: idempotency keys or dedup windows to avoid double-counting on retries/duplicate delivery, since this directly affects advertiser billing.
- Storage: finalized aggregates written to an OLAP-style store for reporting/dashboards.
Key tradeoffs
- Waiting longer for late events improves accuracy but delays when a count is “final” — this exact tension (completeness vs latency) is the crux of the question.