Evernote cull tooling on evernote.gf.cx — batch-LLM Layer-2 value_signal pipeline (parked sketch 2026-05-29)
DARE.CO.UK · PARKED SKETCH · 2026-05-31
Mirrored from ~/.claude/.../memory/parked_sketch_evernote_cull_tooling_on_surface_2026-05-29.md. This is a design sketch parked for future build — read for context, not as a current deliverable.
Build plan for the 24K → ~2.5K diamond+gold cull pipeline that lives on evernote.gf.cx once the corpus is ingested. Per-note Haiku call → value_signal scoring → Diamond/Gold/Reference/Archive/Noise tier assignment → /diamonds/ + /timeline/ + curation UI surfaces. Architecture already designed in the 2026-05-22 parked sketch; this entry is the trigger conditions + build order specific to the evernote.gf.cx locus.
Sketch: A batch-LLM cull pipeline that processes the ingested 24K-note corpus and surfaces the ~10% signal tier (Diamond + Gold ≈ 2.5K notes) as the default view on evernote.gf.cx. Nothing deleted — every note remains searchable; the default view just shows signal. Per-note value_signal score drives tier classification; manual Dan-validated overrides win over the LLM.
Why parked: Blocked on (a) full-corpus ingest landing in ~/Code/evernote.gf.cx/_substrate/ (estimated 6–12 hours wallclock for the 24K with heavy video) and (b) a Dan-validated check on the dry-run substrate output shape before betting $25-35 of Haiku spend on the full pass. Build the cull tooling AFTER the substrate is locked, not in parallel.
Resume conditions:
- ENEX exports complete from Evernote desktop (~30 manual clicks)
pa-evernote-substrateR2 bucket created + 1Password creds wired- Full 24K ingest run completed;
_index.jsonshows expected note count - Dan eyeballs ~20 sample notes in the substrate and confirms ENML→markdown fidelity is acceptable
- Then unblock this sketch
Architecture (lifted verbatim from 2026-05-22 parked sketch):
Per-note Haiku call returns standard schema PLUS a value_signal block:
STASH7
Tier classification:
| Tier | Score band | Treatment |
|---|---|---|
| Diamond (top 5%, ~1.2K) | 0.85+ | Hero surfacing on landing + yearly rollups + retrospectives |
| Gold (next 15%, ~3.6K) | 0.65–0.85 | Default browse view; full search indexing |
| Reference (next 30%, ~7.2K) | 0.40–0.65 | Searchable but not surfaced |
| Archive (next 40%, ~9.6K) | 0.20–0.40 | Hidden from defaults; available via “show all” |
| Noise (bottom 10%, ~2.4K) | <0.20 | Preserved but excluded from rendered surfaces |
Build order (when unblocked):
~/bin/evernote_layer2_score.py— read substrate notes, batch-LLM call per note, appendvalue_signalto the<date>_<slug>.jsonsidecar. Idempotent (skip notes already scored). Cost gate: confirm ~$25-35 estimate before running.- Tier classifier: derive tier from score, write
_tiers.jsonlindex for fast surface queries. - Renderer surfaces (in priority order):
-
/diamonds/— top 5% landing, sorted by score desc -/diamonds/<year>/— yearly rollup (“best of 2014”) -/timeline/— 17-year chronological scroll, Diamond+Gold only by default -/diamonds/by-category/<cat>/— category-filtered diamond views -/_resurface/— weekly cron picks 10 from Archive tier, re-scores against current context -/_diagnostics/value-distribution.html— histogram + per-tier counts for calibration - Dan-validated curation UI: per-diamond confirmation buttons (✓ confirm / ⬇ demote to gold / ⬇ demote to archive); appends to
_curation.jsonl; renderer respects manual over LLM. - Cross-substrate boost computation: lift score for notes mentioning Amazon order IDs, pa vehicles, claim IDs, GPS-matches-property-coords, photo-sha1-matches.
- Near-dupe detection: title-similarity + body MinHash/SimHash + same-attachment-sha1. Within cluster, keep highest-scoring as canonical; mark rest
dup_of: <canonical_guid>. Collapse in default view with “+3 similar” pill.
Cost: Already absorbed in the per-note Haiku Layer-2 call (value_signal block adds ~200 tokens to response). No extra calls. Total: $25-35 one-shot for the 24K — well within Audrey commercial-ROI filter.
1% canon (~240 notes): Within the top 5% Diamonds, the irreducible cultural artifact of 17 years of Dan’s thinking. Reserve a hand-curated experience downstream: maybe a printed book, maybe /evernote/canon/ with annotated commentary, maybe an Opus-summarized “what you’ve learned” retrospective. Substrate should capture enough metadata up front to support it (writing style, recurring themes, evolution over time).
Public/private split: Default private. /diamonds/ surfaces gated via CDN, security layer, and DNS provider sitting in front of dare.co.uk.">CF Access for the substrate views. A curated public subset (hand-reviewed) ships to io.gf.cx/notes/ or similar — io = public per the principle.
Cross-references
project_evernote_gf_cx_migration_2026-05-29.md— the project this falls underparked_sketch_evernote_one_shot_ingester_2026-05-22.md— full original architecture (this entry is the build-order-and-trigger-conditions slice)feedback_one_batch_llm_beats_hand_iteration.md— why batch-LLM beats note-by-note manual cullfeedback_two_layer_decision_architecture.md— separating per-note score (Layer-2) from tier classification (derived) keeps the LLM call cacheable across re-tiering experiments