research-stack · research.devclusterai.com
MVP · records to 2026-09-23 · every number is from the project's records, listed at the end
build log21 August → 18 September 2026ten entries · newest first

The POC, entry by entry.

The half before the MVP: eight questions answered by the tool this replaces, a kernel designed to be checkable, twenty-eight packages built in five days, and a gate of eight checks run three times in one day. Each entry is dated, says what was built or measured, gives the number it produced and names the record that holds it. One of the ten is a correction of a number this page itself carried until today.

2026-09-18BUILD-1docs/PACKAGES.json BUILD-1issue #981PR #1028

Fifty-nine test binaries, and the wait between a change and its verdict.

what was wrong

Every file under crates/kernel/tests/ is its own integration-test binary, and each one links the whole kernel. Fifty-nine of them. Nothing in that is visible in a diff; it is paid every time an agent changes one line and waits to hear whether the suite is still green — which, on this project, is the loop the work actually runs at.

what we built

The fifty-nine become one binary, tests/it/main.rs, a module per former file and mod support once. Nextest filters move with them — binary(x) becomes binary(it) & test(/^x::/) — and every insta snapshot is renamed rather than regenerated, so a snapshot that stopped being referenced shows up as an error instead of quietly passing. The same day, dev and test profiles dropped to line-tables-only debuginfo: full DWARF was most of what was being written and linked.

test build
226.7 → 82.1 s
cargo test --workspace --no-run --timings · same pod, before and after
MEASURED
unit time
856 → 197 s
total compile time across the test units
MEASURED
kernel test binaries
59 → 1
7.15 GB of linked test executables → 263 MB
MEASURED
workspace executables
8.4 → 1.5 GB
recorded in the PR as the done_when asks
MEASURED

The before and after were measured on the same pod, not on two machines, because this number is worthless otherwise. The suite is green with the same test count as main at the rebase point and every snapshot is referenced — those are the package's own done_when clauses, not a summary of them. BUILD-1 is filed as an MVP-stage ops package; it lands here because it merged the day before the MVP gate and it is the last thing in the record before that gate opens.docs/PACKAGES.json BUILD-1 — the done_when and its numbers · issue #981 · PR #1028 · commit 54d76881 (the before/after on asus2), ae0a1793 (the debuginfo profile)

2026-09-17 → 18S1 · S3 · E2 · B6 · S2agit log --mergesdocs/PACKAGES.jsonSPEC [C-210], [C-216]

The layer the side-by-side said to replace.

E1's finding was not the tally; it was that the kernel's reasoning was ahead of the old engine and its sourcing was behind, because R1 — the interim retrieval built to get the POC measured at all — was a single search API and a plain fetch. Between the gate and the MVP campaigns, that layer was replaced.

  • search and fetchReal search and fetch through the substrate, and the first of the fallback ladder. S1 #682 · S1a #693 · S3 #748 · S2a #857 · S2b #869
  • the kernelThe seams those units needed: K6 #739, K7 #740, K9 #741, K13 #812, K8 #858 — and FIX-686 #813 behind them.
  • evaluationE2 #809 — the synthesist numbers its claims by content, so a re-ask is a memo hit on every unit — and E7 #859, the evaluation doctrine E1's method was the seed of.
  • governanceG2 #810: every fresh fetch and search row carries the id of the Policy event it ran under.

Two of those merges changed what a ledger is: B6 (#811) landed Campaign-memory as the twelfth ledger and S2a (#857) Knowledge as the thirteenth. The POC's kernel admitted eleven — Question, Argument, Run, Unit, Plan, Doctrine, Budget, Policy, Decision, Eval, Fetch — and a v1 ledger file still only admits those. None of this is measured here: what it was worth is the MVP gate, three days later.git log --merges main, 2026-09-16 → 18 · docs/SPEC-kernel.md [C-210] (Memory), [C-216] (Knowledge) · docs/PACKAGES.json

what that layer then measured: the MVP build log, 19 September

2026-09-16 · 22:49X1docs/GATE-POC.mdcommit 89e22a6

Eight of eight — and the seven this page published instead.

the gate as it stands

The eight checks of the spec, each one a number the Eval ledger holds, the verdict recomputed from value and target when the document renders and never read from a stored flag. The document is written by rsk gate doc over the ledger; nothing in it is typed by hand.

check 1 · gold-8 headless
8 / 8
runs at the baseline's terminal state · 0 deviations · 72 calls invoked
PASS
check 2 · kill −9 → resume
0 repeats
resumed report hash equal to the uninterrupted one
PASS
check 3 · replay
79 / 79 · $0
0 calls, 0 reserves · 29 tables rebuild-equal
PASS
check 4 · anchor re-read
490 / 490
of 498 anchors, every resolved one re-read byte-for-byte · 0 mismatches
PASS
check 5 · reject → repair
5 pairs
7 unit outputs rejected on an invariant, 5 repaired, live
PASS
check 6 · wall p95
19.4 s
3 live runs at standard depth · max $0.031 · 1 paid call each, 24 cached
PASS
check 7 · the bench line
42 · 0 FAIL
gated rows inside the gate's own ledger
PASS
check 8 · replay = rebuild
14 / 14
172 claim rows equal · 6,130 event refs, 0 dangling
PASS

One clause of check 8 is recorded and not gated, and the document says so: the idempotency run's MUST-versus-SHOULD refutation does not reproduce on any real run (outcome=no_charter_line) — no live run ever wrote the charter line the refutation needs. Only the hand-built fixture proves the shape. That is the gate's own wording, and FIX-2-3 below is the package that made the document say it.docs/GATE-POC.md — the checks, measured 2026-09-16T22:49:15Z at commit 89e22a6, host tims-macbook-pro, release · "Failures: None"

the retraction · this page published the pass before it

The gate ran three times on 16 September, and this site has been quoting the middle pass — the 16:12 run at e41f64a, where check 8 failed. Its numbers were real; they were simply superseded ten hours later by the pass taken after FIX-590 merged, and nobody moved the page. Every figure below is the last pass.

what movedthis page saidthe record says
the eight checks7 PASS / 1 FAIL8 PASS
anchors · resolved · confirmed466 · 451 · 451498 · 490 · 490
reject → repair pairs45
live wall p9536.5 s19.4 s
most expensive live run$0.057$0.031
replay served from the ledger73 / 7379 / 79
event refs · dangling6,485 · 06,130 · 0

Nothing here moved in our favour by accident: the anchor count went up because the later pass covers more runs, and the one number that got better — the live p95 — got better because the Mac was quiet. The point of the row is that a public page carried a superseded measurement for days while the file it cites said otherwise.the superseded pass: docs/GATE-POC.md at commit be6d0d4b (2026-09-16T16:12:59Z, tree e41f64a) · the current one at a627ff4f, tree 89e22a6

2026-09-16FIX-590issue #590PR #673SPEC §4, §7

One live run would not replay.

what the gate found

The 16:12 pass is the one drawn below: seven checks pass and check 8 fails on a single run. Of thirteen replays over 182 claim rows, one came back unequal — live run d7f8f21a…, the third of the E-4 live trio.

1 PASS 8/8 · 67 calls gold-8 headless 2 PASS 0 repeats kill -9 → resume 3 PASS 73/73 · equal replay · rebuild 4 PASS 0 mismatches anchor re-read 5 PASS 4 pairs · live reject → repair 6 PASS p95 36.5 s wall p95 ≤ 180 s 7 PASS 0 FAIL of 42 bench line 8 FAIL 1 of 13 ≠ replay == rebuild #590 · a live run at concurrency 6 cited a page a sibling task had fetched replayed at concurrency 1 the same output is rejected: “A1: url not fetched in run” FIX-590 packaged: anchors resolve against the task's own retrievals · re-run at $0

The pass that found the miss, exactly as it rendered. Check 7 shows PASS here after two earlier failures on one K-1 row; check 8 is the red one. Ten hours later, with FIX-590 merged, the same eight checks all read PASS — the entry above.docs/GATE-POC.md at be6d0d4b — check 8, the named unequal run, "Failures"

what it actually was

At concurrency above one, invariant A1's URL → fetch_ref step resolved a claim's anchor against any task.retrieved row of the run, not the task's own. So a task's admission depended on which of its siblings had committed first: the live run at concurrency 6 admitted a claim whose page a sibling had fetched (docs.stripe.com/error-low-level, fetched by task de50ebbf…), and the same output was rejected invariant.A1 url not fetched in run when the run was replayed at concurrency 1. The spec already forbade exactly this — §4's input_hash and §7's hit rule make a unit's admission a function of the ledger, not of timing — A1 was simply not covered by either.

The fix is the issue's own option (a): A1 resolves against the task's own task.retrieved rows. Found by the second gate pass, confirmed by the independent audit, merged the same day as PR #673.

This is the failure the POC was run to find. A gate whose checks all pass on the first attempt has told you about your gate, not about your system.issue #590 — the run, the task, the sibling and the two options · docs/PACKAGES.json FIX-590 · PR #673 · docs/SPEC-kernel.md §4, §7

2026-09-16PERF-1 · T-BENCHdocs/BENCH-POC.mdcommit 79c3370issue #506

Forty-seven gated rows, one unmeasured, and a laptop that had to be quiet.

the line

Eleven micro-benchmarks against the targets written down in the test plan before any of them ran. Every value is a bench.result event carrying the doctrine and plan hashes of the tree that produced it; rsk bench report recomputes each verdict from value and target. Eighty lines after the latest-per-row rule, 47 gated, 0 FAIL.

benchPOC targetmeasuredverdict
K-1 admit latencyp99 ≤ 2 ms per kind, n = 10 k0.342 – 1.703 msPASS
K-2 sustained admit≥ 500 events/s5,375 – 5,649 /sPASS
K-3 rebuildhash-equal, < 60 s @ 100 k eventsequal · 4.166 sPASS
K-4 anchor resolve≤ 5 ms @ 315 KB (exact ≤ 100 µs)0.089 / 0.315 / 1.781 msPASS
K-5 footprint≤ 60 MB peak, headless @ 837.3 MBPASS
K-6 guardsp50 ≤ 10 ms per guard0.002 – 0.009 msPASS
A-6 the fold≤ 50 ms @ 1 k claims11.829 msPASS
U-4 replay100 % served from the ledger78 / 78 · 0 callsPASS
O-1 kill −920 random kills: 0 repeats, equal hash0 · equalPASS
G-2 budget0 over-cap admits, ≥ 1 refusal, cap reached0 of 400 · 30 refusalsPASS
U-2 first-try≥ 90 % (live, ≤ $2)not attemptedunmeasured

U-2 is the honest gap in this line: the live row was not attempted because the OpenRouter balance was below zero, and the one attempt of the previous pass halted on payment after $0.019 with nothing recorded. The document says unmeasured and why, rather than carrying the previous pass's figure forward — which is the same rule that makes the rest of the line worth reading.docs/BENCH-POC.md — the line, bench by bench; measured 2026-09-16T11:33–11:35Z at commit 79c3370 (clean tree, main), release, mock model server

the box it ran on

This line is timing-sensitive and it ran on a laptop, so the laptop is part of the measurement. A waiter polled every 20 s for a one-minute load at or below 8 with no rustc, cargo, ld or test process alive, and started the line on the second consecutive hit, holding the Mac's build slot for the whole run. The line opened at load 6.74 and ended at 12.16, and every K-1 row carries the host's load at its own measurement as a parameter.

Two earlier passes had failed gate 7 on one K-1 kind — task.retrieved at 2.214 ms and then 2.789 ms against a 2 ms target, both under the load ceiling. The target was not loosened. D-K6 (#506) is still open: whether K-1's gated number should be stated end to end, over the transaction alone, or at 3 ms for the heaviest kinds is a decision for the spec owner, and until it is made the number stands as measured.docs/BENCH-POC.md — "Load while measuring", K-1 per kind · docs/PACKAGES.json FIX-2-1 · issue #506 (open)

2026-09-16E1docs/E1-2026-09-16.mdbench/sets/gold-8doctrine/eval/v1

Five of eight against the engine it replaces. The bar was six.

the judgement

The old browser-driving engine's report beside the kernel's, for each of the eight questions, judged by the method written down first: a verdict with one reason and explicit flip conditions, then a mechanical source audit, then quote fidelity against the live page, then the draft attacked from four angles before the final. Better 0, not-worse 5, worse 3. The self-check moved two questions down from the first draft, because those questions ask for authoritative sources and the kernel had two among about twenty vendor blogs.

browser agents at scale deep $0.72 $0.20 92 → 12 calls NOT-WORSE liveness checks standard $0.87 $0.19 117 → 11 calls WORSE RLS as a backstop scout $0.11 $0.08 16 → 4 calls NOT-WORSE RLS super- user, practice standard $0.94 $0.17 122 → 10 calls NOT-WORSE idempotency tutorial standard $0.71 $0.15 94 → 9 calls WORSE retries and backoff guide standard $0.76 $0.16 93 → 10 calls WORSE observability and alerting standard $0.72 $0.16 99 → 9 calls no report NOT-WORSE same question re-run standard $0.14 $0.04 23 → 2 calls no report NOT-WORSE old tool · its own estimate at batch rates (real ≈ 2×) kernel · settled in the ledger, live better 0 · not-worse 5 · worse 3 → 5 of 8 · the bar was 6 · recorded as eight Decision.feedback events, actor owner

Eight questions, the old engine's cost against the kernel's, and the verdict per question. 656 paid calls and $4.97 against 67 calls and $1.15 — the kernel's calls and money as the eight reports E1 judged recorded them, which is why they do not sum to the 72 the gate counted over the re-recorded tree six hours later. The kernel's figure is settled in its own Budget ledger, while the old engine's is its own estimate at batch rates. The eight verdicts are Decision.feedback events in the gate ledger, actor owner; the document is the reasoning they cite.docs/E1-2026-09-16.md — method, the eight verdicts, the tally · bench/sets/gold-8/questions.json — the baseline runs, calls and estimated cost

the finding that mattered more than the tally

Every loss had one shape. The kernel was structurally ahead — adversarial checks, could-not-establish, honest confidence, verbatim quotes — and behind on where it read: its primary-source share roughly halved on every question where the old engine found primary sources.

primary-source share per question · old (Q1–Q6) vs kernel (Q1–Q8) Q1 47 % 36 % Q2 76 % 38 % Q3 100 % 75 % Q4 100 % 20 % Q5 92 % 11 % Q6 85 % 42 % Q7 5 % Q8 5 % of 179 fetch.done rows in the gate ledger stackoverflow.com 7 / 7 failed · 403 challenge reddit.com 3 / 3 failed · robots_disallowed yoti.com 3 not ok browser-use.com/pricing ok · 200 — the $0.02 miss is extraction, not fetching ahead: adversarial checks · could-not-establish · honest confidence · verbatim quotes behind: primary sources · locked out of the sites practitioners write on ten defects filed → the evaluation doctrine (E7)

Of 179 fetch.done rows in the gate ledger, stackoverflow.com failed 7 of 7 on a 403 challenge and reddit.com 3 of 3 on robots. The pricing page the first question needed was fetched ok/200 and the rate on it never reached the answer — that miss is extraction, not fetching, which is the same sentence the project is still writing about itself a week later. Ten defects were filed from the reading; the method became the first evaluation doctrine.docs/E1-2026-09-16.md — the ledger check, the finding, the ten defects · doctrine/eval/v1 (E7)

the same eight questions, re-judged inside the MVP campaigns: better 3, not-worse 1, worse 4 — the bar is still six

2026-09-15 → 16FIX-2-1…7docs/PACKAGES.jsonissues #494–#500PRs #562–#568

A critic read the POC against its own test plan.

Before the gate was allowed to close, a completeness pass read the built POC against the documents that describe it — the spec, the test plan, the package catalogue — looking only for places where the record claims something the code does not do. It found seven, and each became a package with an issue, a branch and a PR like any other.

packagewhat the record claimedwhat was true
FIX-2-1gate 7 PASSesone K-1 kind at 2.214 ms against a 2 ms target, measured under the load ceiling — re-run, not loosened
FIX-2-2check 5 is gatedthe target attached only to a live-recorded row, and nobody had run the live gate — check 5 read “recorded, not gated”
FIX-2-3B3's refutation reproducesunmet on every real run: only the hand-built fixture proves the shape — the doc now says so
FIX-2-4live rows run on a scheduleone test id with no evidence of ever having run, and no scheduled workflow existed
FIX-2-5snapshots are checked in CIcargo insta --unreferenced=reject named in three documents, absent from the CI job
FIX-2-6gate 1 expects seven done, one gapfour test-plan lines contradicted the baseline file, which records eight done
FIX-2-7every test id is greppable20 test-plan ids cited by no test function's doc comment

Two of the seven were resolved by changing the document instead of the code — FIX-2-3 and FIX-2-4 — and that is written down as the outcome, with the reason. The package text says it in as many words: do not leave the done_when claiming what the record contradicts.docs/PACKAGES.json FIX-2-1 … FIX-2-7 — each with the finding and its two options · issues #494–#500 · PRs #562–#568, merged 2026-09-16

2026-09-14 → 15K0 … F1docs/PACKAGES.json stage pocPRs #100–#267

Twenty-eight packages, and the harness before any of them.

what was built

Twenty-eight packages carry stage poc in the catalogue. The order is the part worth saying: the test harness merged first — a mock model server and fixture recorder, a fault-injection runner, the bench harness and the adversarial fixtures (PRs #100–#103) — and only then the doctrine, the units, the kernel's statechart and ledger, the interim retrieval, the budget stub and the MCP server. Sixteen packages merged on 14 September, each through its own pull request with CI green before the merge.

  • the harnessT-MOCK #100 · T-FAULT #101 · T-BENCH #102 · T-FIXTURES #103 — the mock server's gold-8 seed recordings cost 8 live calls and $0.0121.
  • the brainB1 #104 doctrine · B2 #105 six unit manifests · B3 #225 the argument ledger · B4 #266 the report renderer.
  • the kernelK2 #176 invariants · K3 #177 statechart · K4 #193 unit runner · K5 #224 run ledger and resume. K0 and K1 landed on main on the 11th and 12th, before the rule that every merge goes through a PR.
  • the edgesR2 #194 fetch ledger · R1 #196 interim retrieval · G0 #195 budget stub · F1 #267 MCP server.

The remaining twelve are the seven completeness fixes and FIX-590 above, and the two eval packages E1 and X1 — which produce documents, not code, and so have no merge commit of their own. Each package carries a done_when written before it was built; the workflow rules of the same week made verification block on high-severity defects and file the rest as findings.docs/PACKAGES.json — the 28 packages at stage poc, each with its done_when · git log --merges main 2026-09-14 → 16 · commits f3ff025d (PR-with-CI rule), f0b9d5f4 (the seed recordings)

2026-09-11 → 12R0 · D0 · D1 · D2 · T0docs/DESIGN-kernel.mddocs/SPEC-kernel.mddoctrine/v1

Three candidate designs, one judged, then a spec you can cite by line.

the design

One research synthesis, then three candidate kernels written in full and judged against each other — testability-replay-first, simplest-correct, libsql-perf-first — then one design, then two rounds of spec fixes, then a test plan with two coverage rounds over it. All of it before the first line of the kernel. What came out is a machine with one writer, an append-only event log, and eleven ledgers: Question, Argument, Run, Unit, Plan, Doctrine, Budget, Policy, Decision, Eval, Fetch.

Every rule that can refuse an event has a stable code and a path into the payload that caused it, so a refusal is a row and not a log line. That is what makes the gate's checks possible to state at all: a claim marked observed must carry an anchor whose quote is the exact bytes of a page the run fetched, or the event does not go in.

the doctrine

The doctrine was not rewritten for the kernel. B1 lifted it verbatim out of the old engine's prompts, and its done_when is that the generated skill file comes back byte-identical to the one the old engine shipped — so any later change to how the kernel reads is a change on the record, not a drift. Six laws, and four of them still need a reader:

  • L1Audit before you search. Never issue a query on the asker's framing alone. the ledger's rule
  • L2Evidence outranks priors — including when it contradicts the asker, or your own earlier conclusion. judged
  • L3Every claim carries a source you actually opened. A URL you did not visit is a fabrication. the ledger's rule
  • L4Label your epistemics: observed, inferred, assumed. Never let assumed wear the costume of observed. judged
  • L5Gaps get reported, not smoothed. Absence of evidence is never quietly upgraded. judged
  • L6The reframe is offered, not imposed: answer the question asked, then say what the better question is. judged

L1 and L3 are process facts the ledger already records, so the kernel decides them per finished run and reading a report for them would be judging what the rows say outright. The other four need a reader — which is the whole of the MVP's evaluation problem, and the reason six of 53 reports pass all four.doctrine/v1/DOCTRINE.md — the six laws, verbatim · doctrine/eval/v1/JUDGES.md — which are mechanical and why · docs/DESIGN-kernel.md · docs/SPEC-kernel.md §1–§3 · docs/PACKAGES.json B1, B2

2026-08-21 → 09-10gold-8bench/sets/gold-8/questions.jsonweb-research/runs

Eight questions, answered first by the tool this replaces.

The set was not invented for the kernel. Eight real questions had already been put to the old engine — a browser-driving research pipeline — over three weeks in August and September, and its runs were kept: the question as asked, the terminal state, the wall clock, the call count and the cost. On 11 September those eight became gold-8, the baseline the POC would be measured against and the fixture set its tests replay.

the set
8 questions
1 scout · 6 standard · 1 deep · built 2026-09-11 from web-research/runs
RECORDED
the old engine
656 calls
LLM calls across the eight runs · 10.6 M prompt tokens
RECORDED
its cost
$4.97
its own estimate at batch rates · the figure a live re-run doubles
RECORDED
its wall clock
60.7 min
3,644 s over the eight · 738.8 s on the slowest
RECORDED

Two of the eight never finished: the observability question and its cached re-run ended in error when the provider returned 402, which is why the side-by-side's seventh and eighth verdicts compare a kernel report against nothing. A baseline with two holes in it is still a baseline, as long as the holes are in the file.bench/sets/gold-8/questions.json — the eight questions with each run's terminal state, elapsed seconds, calls and estimated cost

21 Aug old engine 8 questions 11 Sep the design 11 ledgers 14 Sep the build 28 packages 16 Sep measured bench · gate · E1 16 Sep FIX-590 merged #673 18 Sep build time 227 → 82 s

The whole half in one line. What happens after the last stop is the MVP: twenty campaigns, fifty-three questions, and every report read.the entries above, each with its own record

next: the MVP build log — 53 questions for $8.27, 16.42 over 672 expert-written rubric items, and the four numbers we had to take back