Streaming Operations Playbook

Structured streaming work should be observable before it becomes urgent. Lag, schema drift, input rate, and failed expectations are the first signals to put near the operator.

Investigation order

1. Check input rate and processing rate.
2. Inspect recent schema or source format changes.
3. Review failed expectations or quarantined rows.
4. Look at downstream freshness alerts.

Backlog math

If a stream receives (r_i) rows per minute and processes (r_p), then backlog grows when:

[ r_i > r_p ]

That simple inequality is often enough to distinguish a data spike from a broken processor.

Operator handoff

Every streaming table should name a responder, an expected latency window, and the first query to run during incident review.