Experiment published Sep 3, 2026

Full Seed-Ready Evidence Scale Run

KESPA executed the seeded evidence-acquisition path across 3,128 planned evidence slots representing 2,831 cards. It collected and audited 2,146 evidence excerpts; 1,555 of 2,831 cards passed the evidence gate, while 463 evidence rows required review for missing scope-term matches. The run was deliberately held before card generation.

knowledge-factory scale-study evidence-acquisition quality-gates failure-analysis

MARKDOWN

README.md

2,882 bytes SHA-256 58272491f6935e97…

KESPA-EXP-004 — Full Seed-Ready Evidence Scale Run

Date: 2026-09-03 Status: Published historical scale study Project: KESPA AI / NexLabs Studios

Research question

How does KESPA's seeded evidence-acquisition and quality-gating pipeline behave when expanded from a bounded 120-card test batch to the full seed-ready population available at that stage?

Run scope

The historical run selected:

3,128 evidence slots
representing 2,831 cards

The run used seeded authoritative-source acquisition rather than general search fallback.

Acquisition result

KESPA recorded:

2,146 evidence excerpts collected
982 terminal SEED_FETCH_FAILED slots
0 seed-not-ready skips
441 network requests used

The original acquisition summary also recorded 1,469 fetch failures. That counter is preserved as reported, but it is not treated as equivalent to the 982 terminal failed slots in the queue status counts.

Safety rails remained active:

Search fallback: OFF
Full-page mirroring: NO
LLM calls: 0
Chroma writes: 0
Brain changes: 0
Database writes: 0

Evidence-quality audit

The audit evaluated all 2,146 collected evidence excerpts across 2,831 cards.

Source-quality outcomes:

835 PASS
848 PASS_WITH_WARNINGS
463 NEEDS_REVIEW

Card-level gate:

1,555 / 2,831 cards passed
54.9% card-level gate passage

Hard issues:

463 no_scope_term_match

Warnings:

840 weak_scope_term_match
11 navigation-boilerplate-heavy
6 suspected encoding artifacts

Duplicate signals:

161 duplicate evidence hashes
145 duplicate final URLs

Result

This was not a clean-corpus success result.

It was a scale test that showed two things at once:

1. the seeded collector could operate across thousands of planned evidence slots; and

2. scaling exposed enough evidence-quality and scope-matching problems that the pipeline could not

responsibly proceed directly to knowledge-card generation.

The next gate explicitly required review of failed and warned evidence rows.

Why this matters

KESPA's evidence lifecycle was designed to fail closed.

A weaker system could have interpreted "2,146 excerpts collected" as permission to generate thousands of knowledge cards. KESPA did not. The evidence-quality gate held the corpus because only 1,555 of 2,831 cards met the complete evidence gate in this run.

That negative/partial result is part of the research record.

What this does not prove

This experiment does not measure end-user answer quality, production reliability, or general model capability. It also does not mean that every passing card later became trusted knowledge.

It measures the behavior of the evidence-acquisition and quality-gating subsystem under a materially larger workload.

TEXT

SHA256SUMS.txt

399 bytes SHA-256 bbac561ab18bfc8b…

58272491f6935e970d068580ea93f77049f04d8595912e53a94afcda2d00c8a7  README.md
83720e4d541401b7e6e11910d15a9e07399321938c03abc923ac71f617c6f08d  experiment.json
8a8154882f73cdaf71a9de7719eee3d65fd797970cf4b0fb943a78575c200abb  results.csv
038968690233e4237eb75dde73711ff12f82d9731917f855ca65e6ad526a64b0  methodology.md
6c55ebce43d287362ca9798224724a9536011cd68b3e967a149fe633e38dfab1  provenance.json

JSON

experiment.json

2,953 bytes SHA-256 83720e4d541401b7…

{
    "schema": "kespa.public_experiment.v1",
    "id": "KESPA-EXP-004",
    "title": "Full Seed-Ready Evidence Scale Run",
    "date": "2026-09-03",
    "status": "published",
    "research_question": "How does KESPA's seeded evidence-acquisition and quality-gating pipeline behave when expanded from a bounded 120-card batch to the full seed-ready population available at that stage of the knowledge blueprint?",
    "run_scope": {
        "queue_slots_selected": 3128,
        "cards_represented": 2831
    },
    "acquisition": {
        "evidence_collected": 2146,
        "terminal_seed_fetch_failed_slots": 982,
        "reported_fetch_failures": 1469,
        "seed_not_ready_skipped": 0,
        "network_requests_used": 441,
        "status_counts": {
            "EVIDENCE_COLLECTED_PENDING_AUDIT": 2146,
            "SEED_FETCH_FAILED": 982
        }
    },
    "quality_audit": {
        "auditor_version": "forge-kb-evidence-audit-001",
        "evidence_excerpts_audited": 2146,
        "cards_represented": 2831,
        "cards_passing_evidence_gate": 1555,
        "cards_total": 2831,
        "card_gate_pass_percent": 54.9,
        "duplicate_evidence_hashes": 161,
        "duplicate_final_urls": 145,
        "source_quality": {
            "PASS": 835,
            "PASS_WITH_WARNINGS": 848,
            "NEEDS_REVIEW": 463
        },
        "hard_issues": {
            "no_scope_term_match": 463
        },
        "warnings": {
            "weak_scope_term_match": 840,
            "navigation_boilerplate_heavy": 11,
            "encoding_artifact_suspected": 6
        }
    },
    "decision": {
        "card_generation_authorized": false,
        "reason": "The evidence-quality audit required human review of warned/failed rows before card generation. No evidence row became production-ready solely from this audit."
    },
    "safety": {
        "search_fallback": "OFF",
        "full_page_mirroring": false,
        "llm_calls": 0,
        "chroma_writes": 0,
        "brain_changes": 0,
        "database_writes": 0
    },
    "interpretation": [
        "The full run demonstrated that the collector could execute at materially larger scale than the 120-card batch.",
        "The scale run also exposed substantial evidence-quality and scope-matching problems that prevented automatic promotion.",
        "The pipeline correctly stopped at the evidence gate rather than turning partially supported material into trusted knowledge."
    ],
    "limitations": [
        "The run covered the seed-ready population available at that stage, not every planned blueprint slot.",
        "A card passing the evidence gate did not mean that a final knowledge card was generated or production-approved.",
        "The acquisition summary reports 1,469 fetch failures while terminal slot status counts contain 982 SEED_FETCH_FAILED slots; the public report preserves both original measurements without treating them as interchangeable.",
        "Duplicate hashes and final URLs indicate repeated evidence reuse that required later review and deduplication analysis.",
        "The study measures evidence acquisition and gate behavior, not end-user answer quality."
    ]
}

MARKDOWN

methodology.md

1,229 bytes SHA-256 038968690233e423…

Methodology

Source experiment

This public record reconstructs the historical seed_ready_full_run from the internal FORGE_seeded_evidence_scale_v001_2_windows_candidate package.

The public package does not republish raw third-party evidence excerpts, internal scripts, private configuration, or the historical Git repository.

Acquisition stage

The collector selected the full seed-ready queue available to the v001.2 package.

Recorded acquisition outputs included:

queue results;
evidence-source rows;
terminal failure rows;
acquisition summary;
evidence-quality audit.

General search fallback remained disabled.

Quality stage

Collected excerpts were evaluated by forge-kb-evidence-audit-001.

The audit classified evidence rows as:

PASS
PASS_WITH_WARNINGS
NEEDS_REVIEW

It also calculated card-level gate passage and tracked duplicate evidence hashes, duplicate final URLs, hard scope failures, and warning classes.

Release decision

The audit explicitly instructed:

Do NOT generate cards until the selected acquisition batch passes evidence quality gates.

Accordingly, this public study treats the run as a measured scale experiment, not a production knowledge release.

JSON

provenance.json

1,422 bytes SHA-256 6c55ebce43d28736…

{
    "schema": "kespa.public_provenance.v1",
    "experiment_id": "KESPA-EXP-004",
    "historical_source_archive": "forge_ai_research.zip",
    "source_artifacts": [
        {
            "path": "FORGE_seeded_evidence_scale_v001_2_windows_candidate/seed_ready_full_run/seeded_evidence_acquisition_summary.txt",
            "sha256": "b58c35ea8859731774474b14d439e0aeb98b17c43810c124ebd04232d44ee22b",
            "bytes": 482
        },
        {
            "path": "FORGE_seeded_evidence_scale_v001_2_windows_candidate/seed_ready_full_run/quality_audit/evidence_quality_summary.txt",
            "sha256": "856c65c2431f17763ef6314bb663ec41edb258e56aaa8035a17ff9102b8e528e",
            "bytes": 948
        },
        {
            "path": "FORGE_seeded_evidence_scale_v001_2_windows_candidate/seed_ready_full_run/quality_audit/card_evidence_gate_report.csv",
            "sha256": "46a12f523bdb88fd7b778875fb85353fce15c0e8cf659ab6014ad8113970514f",
            "bytes": 513277
        },
        {
            "path": "FORGE_seeded_evidence_scale_v001_2_windows_candidate/seed_ready_full_run/quality_audit/evidence_quality_report.csv",
            "sha256": "908ed89ed22fb6fe68cb5b0377df0762b05b66cd47ea839fe32b487d61775410",
            "bytes": 963719
        }
    ],
    "publication_note": "This public package contains derived measurements, methodology, and hashes. Raw fetched third-party evidence text, private implementation scripts/configuration, and the original internal repository are intentionally not republished."
}

CSV

results.csv

737 bytes SHA-256 8a8154882f73cdaf…

metric,value,unit_or_status
queue_slots_selected,3128,slots
cards_represented,2831,cards
evidence_collected,2146,excerpts
terminal_seed_fetch_failed_slots,982,slots
reported_fetch_failures,1469,fetch_failures
network_requests_used,441,requests
quality_pass,835,evidence_rows
quality_pass_with_warnings,848,evidence_rows
quality_needs_review,463,evidence_rows
cards_passing_evidence_gate,1555,cards
card_gate_pass_percent,54.9,percent
duplicate_evidence_hashes,161,hashes
duplicate_final_urls,145,urls
hard_no_scope_term_match,463,evidence_rows
warning_weak_scope_term_match,840,evidence_rows
warning_navigation_boilerplate_heavy,11,evidence_rows
warning_encoding_artifact_suspected,6,evidence_rows
card_generation_authorized,NO,decision