Parked sketch — throughput + spike speed on status cards (2026-06-05)
DARE.CO.UK · PARKED SKETCH · 2026-06-06
Mirrored from ~/.claude/.../memory/parked_sketch_status_card_throughput_metrics_2026-06-05.md. This is a design sketch parked for future build — read for context, not as a current deliverable.
Extend the self-updating status-card pattern to surface live sustained MiB/s + peak-burst MiB/s read back from transfers-history.jsonl; data source already exists, just needs reader logic + card render
Dan, after watching the live Dropbox card update with only bytes_archived
and an over-counted mib_s_rolling:
“through-put speed, spike speed, would be the next greatest thing to add to future cards”
Why this is the right next move
The schedcard on /transfer-windows/ exists to answer “WHEN should I
schedule X?” The status card answers “IS it moving bytes RIGHT NOW?”
Without throughput, the status card only proves the transfer is alive —
not that it’s healthy. A near-zero MiB/s for 30+ min is a stall; a
spike to 80 MiB/s confirms the VM-route is delivering on its promise.
Throughput data also closes the loop with the APAC-window policy: a sustained-rate chart over the run lets you see the in-window vs out-of-window throughput delta, which validates (or invalidates) the “NYC 10:00–18:00 = Asia overnight = least-contested” hypothesis with empirical bytes-per-second instead of inference.
What the metrics mean
| Metric | Definition | Read from |
|---|---|---|
mib_s_sustained_5min |
Rolling avg of last 5 min of samples — “what is it doing right now” | transfers-history.jsonl rows filtered to current run, last 5 min |
mib_s_spike_peak |
Highest MiB/s observed in any 30-sec window since fire | Same source, per-row mib_s field max |
mib_s_p50_since_fire |
Median MiB/s across all samples in this run — robust “typical” rate | Median of mib_s for current run |
mib_s_p95_since_fire |
95th percentile — characterises bursty behaviour | 95th percentile of mib_s |
Drop the existing mib_s_rolling (it includes pre-fire baseline noise
when the dest already had bytes from a prior run; misleading).
Architecture
The pattern lives in gvr-dropbox-dan-dare-status-update.sh (see
toolkit commit 5e7aa0c). Extend it:
- Read
~/Code/status.gf.cx/data/transfers-history.jsonl, filter rows wherets >= started_atANDvia=$(hostname)(orvia=gvr-ingest-singaporefor VM-route). - Compute the four metrics above.
- Write them into the status JSON as siblings of
bytes_archived. - Render on the card — small inline sparkline OR three pill chips showing sustained / spike / p50. Reuse the schedcard’s traffic-light colour grammar (green / amber / red) keyed to whether sustained is above or below the expected ceiling for the route (Mac ~7 MB/s, VM ~14 MB/s sustained / 67 MB/s burst per Azure-cap memory).
The update script + render template both need a small change; the data
source is already populated by ~/bin/rclone_logspeed.py which wraps
every rclone call in the existing transfer scripts.
Touch points when picked up
| File | Change |
|---|---|
~/bin/gvr-dropbox-dan-dare-status-update.sh |
Add 4 new fields; drop mib_s_rolling |
~/bin/gfcx_status_hub_render.py |
Render throughput chips/sparkline in the active-card body for kind: transfer cards |
| The general pattern | Generalise to ~/bin/status_card_throughput.py helper so future cards can from status_card_throughput import compute(job_id, started_at, via) |
Stop rules
- Don’t build this until at least one transfer has accumulated >=30 min
of
transfers-history.jsonlrows under the new pattern — otherwise there’s no empirical signal to drive the threshold colours. - Don’t compute fancy windowed averages if the rolling samples are sparse (rclone_logspeed appends per-completed-op, not per-second); for low-frequency samples, just show the last value + max.
- Keep the rendered card readable — three numbers max, or one sparkline. This card is on the operational dashboard, not a detail page.
Cross-references
project_transfer_speed_logging_shipped_2026-06-04— the data source (rclone_logspeed.py+transfers-history.jsonl)parked_sketch_asia_vm_insights_json_2026-06-04— sibling parked sketch that derives hour-of-day / off-peak insight from the same dataproject_apac_transfer_window_policy_2026-06-04— the empirical signal these metrics validateproject_dropbox_dan_dare_parked_for_window_2026-06-05— first card that would benefit (and a real data source for tuning thresholds)