Skip to main content

Process Library

The Process Library is how the Hub captures what the shop actually does and reconciles it against what the software does for it. It is maintainer/chassis tooling — a way to inventory every workflow in the business, mark which ones the code already covers, and surface the gaps — not a customer-facing feature. It does not appear on the feature list.

Built v0.6.242 → v0.6.266 · migrations 070–091

The earlier "Process Builder" (a floating per-step editor, v0.6.238–239) was reverted in v0.6.240; process_steps and the category column were dropped. The three-layer model (Code / Vertical / SME) shipped in v0.6.262 (migration 085) and is the load-bearing concept from this batch forward. The v0.6.265 ticket-carryover demo was reverted in v0.6.266 as redundant. What's documented here is the shipped state at HEAD.

The three-layer model (v0.6.262, migration 085)​

Every workflow is now described from three independent perspectives, all stored as rows in the same processes table, distinguished by a layer column:

LayerWhat it answersSource of truthPopulated by
L1 — codeWhat does the Hub actually do?src/index.js + public/index.htmlscripts/extract_processes.py (idempotent UPSERT by code_anchor; migration 087 ships the first run's seed)
L2 — verticalWhat is every bike shop expected to do?The bike-vertical playbookscripts/generate_l2_canon.py (migration 088 = 83 canon entries) + scripts/generate_l2_links.py (migration 089 = 109 typed relationships)
L3 — smeWhat does THIS shop actually do?The shop's subject-matter expert (James, for Swicked)Hand-authored in the Library editor

The three layers JOIN on concept_key — a stable slug naming the workflow independent of who described it (e.g. 'sales:charge', 'customers:create', 'service:ring-up-with-ticket'). A given workflow may exist in any subset of the three layers; the gaps between layers are the insight:

  • L2 \ L1 = vertical wishlist (every bike shop should do this; we don't yet)
  • L3 \ L1 = this-shop wishlist (James wants this; we don't yet)
  • L3 \ L2 = this shop's idiosyncrasy (James does it; not part of the canon)
  • L1 \ L2 = code that doesn't map to a recognised vertical workflow (cruft? or capability nobody documented?)

code_anchor is L1-only: a stable pointer back into the source — e.g. 'src/index.js:apiSalesCreate' or 'dom:#user-menu-signout' — and carries a UNIQUE WHERE NOT NULL index so the extractor can UPSERT idempotently on re-runs.

The backfill in migration 085 walked the existing 56 hand-seeded rows: those with created_by_staff_id IS NULL (the seed authors) flipped to layer='vertical', becoming the L2 bike canon baseline. SME hand-authored rows kept the default layer='sme'. Migration 086 then populated concept_key for every existing row (one UPDATE per row, generated by scripts/backfill_concept_keys.py).

The core idea: two narratives per process​

Every process is one row in the processes table holding two parallel descriptions:

  • Store action (description) — the human workflow, in plain shop language. "When a customer drops off a bike, take their details, describe the problem, and agree a promised-by date."
  • Software process (software_process) — what the Hub does to support it. "New Drop Off opens the customer picker, creates a service ticket, and prompts for a deposit."

Splitting the two is the point: it makes the difference between "how we work" and "what the system automates" explicit and reviewable, one workflow at a time.

Each process is tagged to a page (region_screen — a real tab name like Sales or Service) and an area (region_label — a data-commentable-label region within that page), so the Library can scope itself to whatever screen the operator is on and the Process Map can lay processes out by tab.

The review pipeline​

The load-bearing concept is review status (review_status, with is_programmed kept as a denormalized mirror):

StatusMeaning
PendingA subject-matter expert wrote the process; nobody has reviewed it against the code yet. The default for new entries.
ProgrammedA reviewer confirmed the software already does this.
Needs codeA reviewer confirmed this needs new code. The wishlist, grounded in a real workflow.

The List view splits on this: "Not yet programmed" (pending + needs-code) on the left, "✓ Programmed" on the right. Every status change appends to process_review_log (prior status, new status, reviewer, notes, timestamp) so the review history is auditable.

Data model​

All in D1 (single-tenant per shop, per ADR-0003):

  • processes — name, description (Store action), software_process (Software process), region_label, region_screen, review_status, is_programmed, reviewer_notes, reviewed_at, reviewed_by_staff_id, resolved_spec (JSON, from the Interpretation Engine), last_interpreted_at, x_pos / y_pos (Process Map layout, migration 082), is_active (soft-delete), timestamps.
  • process_review_log — the status-change audit trail (migration 079).
  • process_links — typed edges between processes: link_type ∈ sequential / paired / alternative / triggers / sub_step, with label + confidence, UNIQUE(from, to, link_type) (migration 081). 79 high-confidence links seeded (migration 083).
  • process_bundles + process_bundle_members — named end-to-end workflows assembled from processes (migration 081). Ship empty; operator-created.
  • process_interpretations — the Interpretation Engine's per-call cache (see below).

Endpoints​

GET /api/processes (active, by name) · GET /api/processes.json (bulk export) · POST /api/processes · GET|PUT|DELETE /api/processes/:id (PUT writes a process_review_log row and syncs is_programmed on status change; DELETE is soft) · full CRUD under /api/process-links and /api/process-bundles (+ members / reorder) · POST /api/processes/:id/interpret · GET /api/processes/:id/interpretations · GET /api/processes/:id/export-prompt.

The UI​

A fixed bottom-right pill (ultraviolet #7c3aed) opens the Library as an 85vw × 85vh modal over whatever screen is active — it is not a nav tab. The modal header carries three views:

  • 📋 List — the two-column Programmed / Not-yet-programmed board, grouped by tab, defaulting to the active tab's processes (a "View all tabs" toggle widens it). Each card shows a status badge and opens the editor.
  • 🗺 Graph — embeds the Process Map.
  • 📦 Bundles — operator-assembled end-to-end workflows.

The editor (per process) has the two narrative textareas (Store action, then Software process), the page/area tags, a ⌖ Capture from active tab button, the review block (status radios + reviewer notes), and the connections panel.

Label lag

The pill's visible text still reads "⚙ Process Builder" even though everything it opens is the Process Library — a leftover from the reverted Builder. Per the label doctrine this is fixed in source, not at runtime.

Clarifications: the reviewer ↔ SME Q&A round-trip (v0.6.260, migration 084)​

The review pipeline left one gap: the reviewer (Tom, running a Claude-Code review pass) needed a way to ask the SME (James) a clarifying question without leaving the Library. Migration 084 adds process_clarifications with a four-state lifecycle:

open       → reviewer posted; SME hasn't answered
answered   → SME submitted an answer_body; reviewer hasn't accepted
resolved   → reviewer marked the answer as sufficient (closes the loop)
wontfix    → reviewer dismissed the question (out of scope / not needed)

The schema ships in 084; endpoints land in the same deploy. A planned follow-up auto-appends the full Q&A thread to the next 📋 Copy for review prompt so the next Claude pass sees the conversation, not just the latest spec. For now, Tom can drive the round-trip via curl against /api/processes/:id/clarifications.

The maintainer scripts (scripts/)​

The three layers are populated by Python helpers — not via the operator UI. They are maintainer-only and idempotent:

ScriptRoleOutput
scripts/extract_processes.pyWalks src/index.js for async function api* declarations + public/index.html for data-commentable-label regions; emits one row per L1 anchor with prose lifted from the JSDoc/comment blockOne SQL file, UPSERT on code_anchor
scripts/generate_l2_canon.pyGenerates L2 vertical-canonical rows from the hand-maintained bike-shop playbook; flags each row's review_status as programmed or needs_code based on whether the corresponding L1 row existsMigration 088
scripts/generate_l2_links.pyGenerates process_links rows joining L2 canon entries to each other and bridging back to the original 56 seeded entriesMigration 089
scripts/backfill_concept_keys.pyWalks every active row missing a concept_key and emits one UPDATE per row using the <region-screen-slug>:<name-slug> pattern (with numeric suffix on collision)Migration 086
scripts/pull_new_l3.pyEvidence-gathering vetter for new SME-authored L3 entries: pulls every pending L3 row from prod D1 since the last marker, surfaces every L1/L2 row that might be prior art (schema hits, handler hits, fuzzy-name matches), and emits a Markdown report for the chat-vetter (Tom + Claude) to readStdout (scripts/.vet_state.json tracks last_seen_id)

pull_new_l3.py deliberately makes no AI calls — it gathers evidence; the verdict (redundant / partial / gap) is made by a human or AI reading the report. This is the same shape as the manual-Copy-for-review loop in the Library UI: code does the deterministic part, intelligence does the judgement.

Who authors each layer​

Each layer has a different author with a different deliverable:

LayerAuthorSkill requiredTool
L1 (code)Coder, or scripts/extract_processes.pyKnowledge of the codeThe extractor walks src/index.js for async function api* declarations + public/index.html for data-commentable-label regions; idempotent UPSERT on code_anchor
L2 (vertical canon)The Knower — someone who understands the target industryKnowledge of the tradeThe vertical playbook + the L2 generator scripts
L3 (SME)The shop's owner / lead operatorKnowledge of this shopHand-authored in the Library editor; vetted by pull_new_l3.py

The Knower role​

A Knower is a distinct role from the Coder and from the Client. A Knower's deliverable is the vertical's playbook: every workflow a business in this industry is expected to do, written in plain language, with the L2 generator producing processes rows + typed process_links.

A Knower works before the first client of a new vertical (see Scenario 3) so the system already understands the industry by the time the client arrives. The bike vertical's L2 was reverse-derived from L1 in migration 085 because the bike vertical was built code-first; future verticals reverse the order on purpose — L1 by harvest, L2 by Knower, then L3 with the client.

How L2 evolves over time​

L2 is not a static authored canon. It grows over time from two signals:

Cross-instance pattern detection​

When multiple shops in the same vertical begin authoring similar L3 entries — the L3 vetter script surfaces these by flagging fuzzy-name matches across instances — that signal becomes a candidate L2 row. The new workflow is added to the vertical canon, flagged needs_code, and on the next code-ship propagates to every instance in that vertical at once.

This is the multi-vertical analogue of the discontinued tradesperson-template's "cross-client template update," re-expressed in the three-layer model. The trigger: "if N shops in the bike vertical are all authoring an L3 entry that looks like service:warranty-claim-against-supplier, that workflow belongs in the bike canon — not as N independent SME entries."

Industry-wide regulatory change​

New tax rule, new disclosure requirement, new accessibility law — the Knower updates the L2 canon, and the change ripples to every instance via the substrate propagation path (per the Substrate Line).

Both paths land at the same place: a new or updated L2 row, joined to L1 via concept_key, gap-tracked against L1 by the existing queries.

Interpretation Engine (dormant)​

The Interpretation Engine (migration 081, v0.6.253) was meant to let Claude read a process narrative and judge whether the code covers it (verdict ∈ programmed / partial / needs-code / uncertain), caching every call in process_interpretations. The schema and POST /api/processes/:id/interpret shipped — and the handler contains a real Anthropic API call — but in v0.6.254 the Anthropic UI was removed and the endpoint now returns 503 unless ANTHROPIC_API_KEY is set, which it deliberately is not. So no AI call fires today.

What's live instead is the manual loop: 📋 Copy for review puts a structured prompt on the clipboard to paste into Claude Code, and 📤 Export prompt (GET /api/processes/:id/export-prompt) emits a deterministic codegen template that cites the nomenclature doctrine. This is the codebase's first real-but-gated Claude wiring; see AI integration → Interpretation Engine.

See also​