Evidence
Three states, always: what was built, what the audit found, what has not been run.
What was built
A browser-based evaluation instrument (v3) designed and built to test whether AI delegates deliberating across an enforced context-isolation boundary produce valid synthesis — the question at the center of an AI-mediated research-collaboration platform. It is a governed agent system under test: delegated roles, a bounded turn budget, human checkpoints, action logging and provenance export. Nine authored scenarios expand into eighteen matched presets (baseline and treatment share identical contexts and checklist and differ only in settings).
| Control | Status | Note |
|---|---|---|
| Delegated roles: two delegates (A, B) and a Recorder, each with its own provider and model | implemented | Provider is held constant across the suite; model resolves to the strongest in each provider list. |
| Context isolation between delegates (split-context arm) | implemented | Each delegate receives only its own context; the single-context arm is the ablation. |
| Pre-committed grading rubrics and sealed targets per case | implemented | One case (VF-1) carries a target/chain inconsistency — D-01. |
| Ablation arms: split-context vs single-context | partly built | Split and A-alone built; the B-alone arm required by the spec is not — D-06. |
| Leakage and contamination review of case contexts | executed | Round-2 review ran on TB-2, RF-3, VF-1; the provenance export mislabels it — D-04. |
| Grader independence from delegates and Recorder | specified — build non-conformant | Both grader calls run on the Recorder — R-07; fourth-agent fix specified. |
| Human checkpoints: operator interventions mid-run; pause and stop | implemented | Intervention placement in provenance drifts after the first spawn — D-03. |
| Bounded turn budget with an explicit extra-turn request mechanism | implemented | Delegates may request an extra turn only when the setting allows it. |
| Action and turn logging; provenance export (JSON) | implemented | Two labeling defects recorded — D-04, D-05. |
| Sandbox framing control | specified — not effective | Writes a field no delegate path reads — D-02. |
| Tool use by delegates | not applicable | Delegates receive context only; document parsing happens in the browser to assemble context. |
| Post-run analysis per run | implemented | Run records are kept in the browser and exported on demand. |
Statuses are read from the code at the time of the site build (2026-09-03), not from design documents. "Specified" means the specification requires it and the build does not yet conform.
What the audit found
On 2026-09-01 the instrument went through an origin-blind adversarial review by a separate model; each finding was then verified in code, in session, and two further issues the review missed were found in the same pass. Six defects, three decisions, one documentation correction. The review was model-run; that is disclosed here and in the register.
| ID | Finding | Class |
|---|---|---|
| D-01 | VF-1 sealed target says LOWER; the chain and checklist say RAISE | QA defect (case content) |
| D-02 | Sandbox "Question/framing" control writes a field no delegate path reads | ineffective stated control |
| D-03 | Intervention turn-index drift: interventions after the first spawn are misplaced in provenance | provenance |
| D-04 | independent_giveaway_check exports not_done for TB-2, RF-3, VF-1 although the round-2 leakage review ran | provenance (understated control) |
| D-05 | Q2 meta.purpose claims the pair "flips ONLY the Verify phase"; phaseForTurn re-slices exposure when Verify is enabled | overclaim in metadata |
| D-06 | B-alone Q1 arm required by spec §4.2/§6.3 is not built (solo-A only) | spec conformance gap |
Three decisions followed: a fourth agent slot for grader independence (the Recorder currently grades its own recognition), a single-context leak probe on two cases, and this register. Read the full register — every entry carries its class, severity, status, verification date and fix specification, and entries are superseded rather than edited.
Logic harness
count withheld The Block 2 build transcript (2026-06-24) records a logic-invariant harness passing in full — covering context isolation, recorder non-self-attribution, spawn isolation, and the independence of the two grader calls (blind extraction, then reconcile). The harness and its run output do not survive as artifacts and no hash of the graded file was recorded, so the site states the invariants, which are re-checkable by reading the artifact, and not a count. Register entries N-10 and N-11.
What has not been run
- Q1 baseline replication (cases TB-2 and RF-3), split-context versus single-context. The readout is a spectrum position, not pass/fail. It will be recorded in the register either way.
- Q2 steerability — after Q1 has a recorded result.
- The B-alone arm (D-06) and the single-context leak probe (R-08).
- The grader-independence fix (R-07).
Open the instrument
The instrument runs in your browser with your own model-provider keys. Read what it does with your keys, then open it