MARKDOWN
README.md
4,554 bytes
SHA-256 f7f1bc7dbd91c31c…
KESPA-EXP-003 — Seeded Evidence Scale: Scope-Precision Correction
Original experiment date: 2026-09-02 Public archive publication: 2026-09-15 Status: Verified historical experiment Project: KESPA AI / NexLabs Studios
Research question
When a fixed evidence-acquisition batch fails primarily because broad topic-level seed URLs do not match the exact card scope, can a bounded scope-specific seed correction improve evidence quality without changing the canonical blueprint, acquisition architecture, or source trust registry?
Fixed batch
The before/after comparison used the same production Scale Batch 001:
- 120 complete cards
- 142 planned evidence slots
- 107 topics
- 15 domains
- search fallback: OFF
- LLM calls: 0
- production Chroma writes: 0
- Brain changes: 0
- KESPA database writes: 0
The v001.2 correction did not create a new batch and did not change the target set.
Baseline result
The first live seeded-evidence run collected 98 evidence excerpts from the 142 planned slots. The offline evidence audit reported:
- 41 PASS
- 33 PASS_WITH_WARNINGS
- 24 NEEDS_REVIEW
- 24 hard issues, all
no_scope_term_match - 62 / 120 cards passed the complete evidence gate (51.7%)
The observed hard failures pointed to seed precision: some approved topic-level source pages were too broad for the exact requested card scope.
Bounded correction
The correction added a configuration-driven scope-specific seed layer:
- 64 deterministic override rules
- 72 batch slots affected
- 51 topics affected
- 38 already-approved source IDs used
- 77 exact authoritative seed URLs
- 0 new source/trust families
- canonical blueprint: unchanged
- source registry: unchanged
- collector architecture: unchanged
- search fallback: OFF
The correction covered all 24 prior hard no_scope_term_match rows with a more precise scope seed. Five prior terminal fetch slots were intentionally left unresolved instead of expanding trust solely to improve the score.
Corrected rerun result
The same 120-card / 142-slot batch was rerun. It collected 97 evidence excerpts and the same offline auditor reported:
- 52 PASS
- 44 PASS_WITH_WARNINGS
- 1 NEEDS_REVIEW
- 1 hard
no_scope_term_match issue - 82 / 120 cards passed the complete evidence gate (68.3%)
Measured change
- Card evidence-gate passage: 51.7% → 68.3%
- Absolute improvement: +16.7 percentage points
- Passing cards: 62 → 82 (+20 cards)
- Hard scope-match issues: 24 → 1 (-95.8%)
- PASS evidence rows: 41 → 52
- PASS_WITH_WARNINGS rows: 33 → 44
- Evidence excerpts audited: 98 → 97
The corrected run did not improve raw fetch yield: it audited one fewer evidence excerpt. The result therefore supports a narrower conclusion — the scope-specific seed correction substantially improved the quality/gating outcome of the acquired evidence, not network reliability or source availability.
Interpretation
This experiment supports the idea that evidence acquisition can fail because of locator precision, even when the source family itself is approved and the collector is functioning. A bounded scope-aware correction improved evidence-to-card fit without broadening trust policy or redesigning the acquisition mechanism.
That matters to KESPA because permanent knowledge depends on source evidence being relevant to the atomic scope it is supposed to support. More fetched text is not necessarily better evidence.
What this does not prove
This is not a randomized experiment, and network conditions were not frozen between the two runs. The correction changed exact source URLs for affected slots, so the comparison demonstrates the behavior of this specific batch and policy configuration rather than a universal improvement rate.
The experiment also does not prove that the remaining PASS_WITH_WARNINGS rows are production-ready. The original audit explicitly required human review and prohibited card generation merely because an evidence row passed this audit stage.
Historical source
This public record was reconstructed from the original internal KESPA research archive. The public package intentionally excludes implementation scripts, raw fetched third-party evidence text, private configuration, and duplicated baseline packages. Hashes for the source summary artifacts used to reconstruct this record are included in provenance.json.
TEXT
SHA256SUMS.txt
399 bytes
SHA-256 d669a79fab051090…
f7f1bc7dbd91c31ce535def348b030a5eca19fd51e5d102f2b1d17b116c095e2 README.md
0ad0b16775458ff2f1fa9f4c9dcc8de3866487089b41a5b29a795f840522d01e experiment.json
089b217413c808b46f4531b0f6aefdf9eebc49c0f5704a7432092e42774c7d13 results.csv
acbc83af042daccbe6ac5b96845d3b805bcf67e9c2321a618fa2ba23baf23981 methodology.md
2aa1f45ecfe86b3921fad6d5ae22a09e453646f758ec65e296c4573af97a9006 provenance.json
JSON
experiment.json
2,699 bytes
SHA-256 0ad0b16775458ff2…
{
"schema": "kespa.public_experiment.v1",
"id": "KESPA-EXP-003",
"title": "Seeded Evidence Scale: Scope-Precision Correction",
"original_date": "2026-09-02",
"published_at": "2026-09-15T07:47:28Z",
"status": "verified",
"research_question": "Can a bounded scope-specific seed correction improve evidence quality on the same fixed batch without changing the canonical blueprint, collector architecture, or source trust registry?",
"fixed_batch": {
"cards": 120,
"evidence_slots": 142,
"topics": 107,
"domains": 15
},
"baseline": {
"evidence_excerpts_audited": 98,
"pass": 41,
"pass_with_warnings": 33,
"needs_review": 24,
"hard_no_scope_term_match": 24,
"cards_passing_evidence_gate": 62,
"cards_total": 120,
"card_gate_rate": 0.5166666667,
"duplicate_evidence_hashes": 19,
"duplicate_final_urls": 21
},
"correction": {
"override_rules": 64,
"batch_slots_affected": 72,
"topics_affected": 51,
"existing_approved_source_ids": 38,
"exact_seed_urls": 77,
"new_source_trust_families": 0,
"blueprint_changed": false,
"collector_architecture_changed": false,
"source_registry_changed": false,
"search_fallback": false
},
"corrected_rerun": {
"evidence_excerpts_audited": 97,
"pass": 52,
"pass_with_warnings": 44,
"needs_review": 1,
"hard_no_scope_term_match": 1,
"cards_passing_evidence_gate": 82,
"cards_total": 120,
"card_gate_rate": 0.6833333333,
"duplicate_evidence_hashes": 16,
"duplicate_final_urls": 19
},
"measured_change": {
"passing_cards_delta": 20,
"card_gate_percentage_point_delta": 16.66666666,
"relative_card_gate_improvement": 0.3225806452,
"hard_scope_issue_delta": -23,
"hard_scope_issue_reduction_rate": 0.9583333333,
"evidence_excerpt_delta": -1
},
"safety_and_isolation": {
"llm_calls": 0,
"search_fallback": false,
"full_page_mirroring": false,
"production_chroma_writes": 0,
"brain_changes": 0,
"database_writes": 0
},
"conclusion": "For this fixed batch, a bounded scope-specific seed correction materially improved evidence-quality gating and reduced hard scope-match failures without broadening source trust or changing the acquisition architecture. It did not improve raw fetch yield.",
"limitations": [
"Same-batch before/after comparison, not a randomized experiment.",
"Network conditions were not frozen between live runs.",
"Exact URLs changed for affected rows by design.",
"PASS_WITH_WARNINGS evidence remained common after correction.",
"Evidence-audit passage did not by itself authorize production card generation."
]
}
MARKDOWN
methodology.md
2,045 bytes
SHA-256 acbc83af042daccb…
Methodology — KESPA-EXP-003
Comparison design
This was a same-batch before/after rerun rather than a new benchmark sample.
Unchanged between runs:
- 120 complete cards
- 142 evidence slots
- 107 topics
- 15-domain batch composition
- canonical 759-topic / 3,982-card / 4,807-slot blueprint baseline
- evidence collector architecture
- offline auditor (
forge-kb-evidence-audit-001) - authoritative source registry
- trust families
- search fallback disabled
- no LLM-based evidence generation or judging
Changed in v001.2:
A deterministic scope-override configuration supplied more precise seed URLs for selected topic+scope rows. Overrides could only use source IDs and URL hosts already allowed by the existing source registry.
Pre-correction diagnosis
The baseline audit produced 24 hard failures. Every hard issue had the same reason: no_scope_term_match.
The correction was therefore targeted at seed-to-scope precision rather than changing the collector or loosening the evidence gate.
Correction constraints
The historical correction report records:
- 64 override rules
- 72 affected batch slots
- 51 affected topics
- 38 existing approved source IDs
- 77 exact seed URLs
- zero new source/trust families
Five previous terminal failure slots were intentionally unresolved rather than adding new trusted source families merely to improve the metric.
Evaluation
Both runs were evaluated with the same offline evidence-quality audit. The card-level gate required the selected card's evidence obligations to pass the evidence-quality rules; evidence audit success alone did not authorize production card generation.
Limitations
- The two live network runs were not conducted under frozen network conditions.
- Exact source URLs changed for affected rows by design.
- Fetch yield did not improve in the corrected run.
- PASS_WITH_WARNINGS remained common after correction.
- The result should be interpreted as a batch-specific evidence-quality improvement under a bounded scope-seed correction.
JSON
provenance.json
1,765 bytes
SHA-256 2aa1f45ecfe86b39…
{
"schema": "kespa.public_provenance.v1",
"experiment_id": "KESPA-EXP-003",
"source_archive": "legacy internal KESPA research repository supplied for public-history reconstruction",
"source_artifacts": [
{
"label": "baseline_quality_summary",
"legacy_path": "FORGE_seeded_evidence_scale_v001_1_windows_candidate/scale_batch_001_run/quality_audit/evidence_quality_summary.txt",
"sha256": "2a9d6818ef64c7c053e64323459f812959be5599d88cf7000195d807ed5b56b1"
},
{
"label": "scope_seed_correction_report",
"legacy_path": "FORGE_seeded_evidence_scale_v001_2_windows_candidate/SCOPE_SEED_CORRECTION_REPORT_v001_2.txt",
"sha256": "7811f22dd252c4c464e9775fc06d568a3fd765237a7ab34c00c7e33df767a75f"
},
{
"label": "corrected_quality_summary",
"legacy_path": "FORGE_seeded_evidence_scale_v001_2_windows_candidate/scale_batch_001_run/quality_audit/evidence_quality_summary.txt",
"sha256": "fe14b421018688306046e2cff62a6ba6d2b137b8ba39a3ab708f474a8bb2ce21"
},
{
"label": "fixed_batch_summary",
"legacy_path": "FORGE_seeded_evidence_scale_v001_2_windows_candidate/generated/scale_batch_001_summary_v001.txt",
"sha256": "82e26d28cf5d8097b7bb52ff0720b3a5d7f89fd2655113d7c2c943b18a58b27d"
},
{
"label": "v001_2_audit_changelog",
"legacy_path": "FORGE_seeded_evidence_scale_v001_2_windows_candidate/AUDIT_CHANGELOG.txt",
"sha256": "4931d31114892c6926590d028c2073992c986992592ea263b2741f62f460e5cf"
}
],
"publication_policy": {
"raw_third_party_evidence_republished": false,
"implementation_scripts_republished": false,
"private_configuration_republished": false,
"aggregate_results_and_source_artifact_hashes_only": true
}
}
CSV
results.csv
406 bytes
SHA-256 089b217413c808b4…
metric,baseline_v001_1,corrected_v001_2,change
cards_total,120,120,0
evidence_slots,142,142,0
topics,107,107,0
evidence_excerpts_audited,98,97,-1
PASS,41,52,+11
PASS_WITH_WARNINGS,33,44,+11
NEEDS_REVIEW,24,1,-23
hard_no_scope_term_match,24,1,-23
cards_passing_evidence_gate,62,82,+20
card_gate_rate_percent,51.7,68.3,+16.7 percentage points
duplicate_evidence_hashes,19,16,-3
duplicate_final_urls,21,19,-2