WikiWatch streams every Wikipedia edit and alerts a (fictional) communications team when a watched brand page gets a risky edit. How it works
The alerts are only as good as the pipeline behind them. Updated every 5 minutes while a session runs.
How recent the newest data in each layer is.
Every event Redpanda accepted lands in Bronze or the dead-letter queue; Silver keeps one row per event ID.
Duration and input rate per Spark streaming query.
| Streaming query | Batches | Input rate | Batch duration p50 | Batch duration max |
|---|---|---|---|---|
bronze_edits | 60 | 27.5 rows/s | 2.6 s | 3.1 s |
gold_alerts | 61 | 108.1 rows/s | 5.1 s | 6.4 s |
gold_edits_per_min | 61 | 27 rows/s | 3.9 s | 9.5 s |
silver_edits | 60 | 27.5 rows/s | 5.5 s | 8.3 s |
From the edit on Wikipedia to the alert row in Gold.
No on-time alerts in the last 7 days to measure. Alerts on events that arrived late (for example after a resume) are left out, because they measure the outage, not the pipeline.
Total lag 1,669 events across 6 partitions of wiki_edits.
| Partition | Latest offset | Processed by Silver | Lag |
|---|---|---|---|
| 0 | 76,743 | 76,452 | 291 |
| 1 | 71,092 | 70,821 | 271 |
| 2 | 66,202 | 65,933 | 269 |
| 3 | 70,136 | 69,866 | 270 |
| 4 | 69,460 | 69,194 | 266 |
| 5 | 78,402 | 78,100 | 302 |
Gold models and every dbt test, built every 30 minutes, then exported to the site.
Health checked 17 Oct 2026, 23:45 UTC.