Technical note published Sep 1, 2026

Legacy Corpus Recovery and Clean Rebuild Boundary

KESPA recovered 729 original subject areas from the legacy synthetic corpus, resolved all 121 ambiguous taxonomy classifications, split the recovered scope into 484 TECH_REVIEW and 245 REBUILD_FROM_SOURCE topics, preserved the ~442k-card synthetic corpus as immutable archive material, and explicitly rejected those synthetic cards for clean production indexing.

corpus-recovery taxonomy contamination-control clean-rebuild knowledge-factory

MARKDOWN

README.md

3,022 bytes SHA-256 38cdc9aee7c9a7a2…

KESPA-NOTE-005 — Legacy Corpus Recovery and Clean Rebuild Boundary

Date: 2026-09-01 Status: Published technical note Project: KESPA AI / NexLabs Studios

Why this work was necessary

KESPA had accumulated a very large synthetic knowledge corpus.

At this point in the project, the historical record reports:

442,577 synthetic cards
uneven or synthetic trust assumptions
known contamination concerns
a need to preserve useful topic coverage without treating the generated card bodies as production truth

The response was not to delete the research history.

It was to separate topic recovery from knowledge trust.

Recovered taxonomy

KESPA recovered all 729 original subject areas from the legacy corpus.

The first recovery manifest classified topics into:

TECH_REVIEW
REBUILD_FROM_SOURCE
MANUAL_CLASSIFY

There were 121 ambiguous classifications requiring explicit resolution.

Those were resolved as:

81 -> TECH_REVIEW
40 -> REBUILD_FROM_SOURCE

Final recovered taxonomy:

484 TECH_REVIEW topics
245 REBUILD_FROM_SOURCE topics
0 MANUAL_CLASSIFY
729 total topics

What was preserved

The recovery work preserved:

per-topic corpus statistics;
contamination statistics;
classification-source metadata;
the legacy synthetic corpus itself as an immutable archive.

It did not rewrite or silently sanitize the old corpus and then call it trusted.

What was rejected

The 442,577 synthetic cards were explicitly rejected for clean production indexing.

That is the key architectural boundary.

KESPA retained them as:

historical material;
research evidence;
discovery/reference material.

But the clean production direction required new evidence-backed knowledge.

Clean rebuild policy

The subsequent Knowledge Factory plan added:

authoritative source policies by domain;
topic-specific source hints;
freshness/revalidation rules;
high-stakes detection;
primary-source requirements;
independent cross-check requirements;
technical code/command validation;
evidence-based trust instead of synthetic fixed trust;
focused retrieval-text contracts.

This meant KESPA could preserve *what subjects it knew it should cover* without preserving *unverified synthetic statements as truth*.

Research significance

This recovery created the experimental boundary used by the later project:

legacy synthetic corpus -> archive / history

recovered topic taxonomy -> research plan

new authoritative evidence -> candidate knowledge

verification -> trusted knowledge

That separation is what made later clean-corpus and A/B/C experiments meaningful.

Snapshot note

The 442,577-card figure belongs to this September 1 recovery snapshot.

Later production-cutover records reference approximately 441,197 legacy cards after intervening system changes. This public record intentionally preserves the historical values rather than forcing them into one count.

TEXT

SHA256SUMS.txt

393 bytes SHA-256 c8a7e3685818efa7…

38cdc9aee7c9a7a2e091913d7b729e28878c318426cfca40164f27f1e8e5245d  README.md
5c4f3442a5b7d1d7277c90c474ab3ee23a6313dc2fa2438749c83c66804b71c7  note.json
0abfd5329f1fbf10b9172b0aed0dafaebda1cf43df60f62cbc66ee68cf344686  metrics.csv
4c6c354cfe331955e926ca1c4367b77c28b2fb3e4c3eb515f75dbe61c179c71d  methodology.md
502d3b94276c035a610151a0afe52d4141c8707bb88e9e7ec259e733bf52e87c  provenance.json

MARKDOWN

methodology.md

1,079 bytes SHA-256 4c6c354cfe331955…

Methodology

Recovery

The legacy corpus was used to recover the set of original subject areas without accepting its generated card bodies as trustworthy production knowledge.

Each recovered base topic received:

corpus/contamination statistics;
a rebuild classification;
classification-source metadata;
optional heuristic research-lead terms that were explicitly not promoted as facts.

Ambiguity resolution

The initial recovered manifest contained 121 topics requiring manual classification.

Those topics were explicitly assigned to either:

TECH_REVIEW, or
REBUILD_FROM_SOURCE.

After resolution, no MANUAL_CLASSIFY backlog remained.

Preservation rule

The existing synthetic corpus was kept immutable as archive/research material.

The recovery process did not rewrite all_cards.jsonl, production Chroma, or brain_api.py.

Rebuild rule

Production knowledge was to be rebuilt from authoritative evidence with explicit source, freshness, verification, and trust contracts.

That decision separated coverage recovery from factual trust.

CSV

metrics.csv

429 bytes SHA-256 0abfd5329f1fbf10…

metric,value,unit_or_status
recovered_subject_areas,729,topics
ambiguous_classifications_resolved,121,topics
ambiguous_to_tech_review,81,topics
ambiguous_to_rebuild_from_source,40,topics
final_tech_review,484,topics
final_rebuild_from_source,245,topics
manual_classify_remaining,0,topics
legacy_synthetic_cards_preserved,442577,cards
legacy_cards_clean_production_eligible,NO,policy
legacy_corpus_preserved_as_archive,YES,policy

JSON

note.json

2,736 bytes SHA-256 5c4f3442a5b7d1d7…

{
    "schema": "kespa.public_technical_note.v1",
    "id": "KESPA-NOTE-005",
    "title": "Legacy Corpus Recovery and Clean Rebuild Boundary",
    "date": "2026-09-01",
    "status": "published",
    "purpose": "Document the recovery of KESPA's original topic taxonomy from the legacy synthetic corpus and the decision to preserve that corpus for history/research while rebuilding production knowledge from authoritative evidence.",
    "recovery": {
        "recovered_subject_areas": 729,
        "ambiguous_classifications_resolved": 121,
        "tech_review_topics": 484,
        "rebuild_from_source_topics": 245,
        "manual_classify_remaining": 0
    },
    "legacy_corpus": {
        "synthetic_cards_preserved": 442577,
        "role": "immutable archive / discovery / research history",
        "eligible_for_clean_production_indexing": false
    },
    "classification_process": {
        "initial_manifest_classes": [
            "TECH_REVIEW",
            "REBUILD_FROM_SOURCE",
            "MANUAL_CLASSIFY"
        ],
        "final_manifest_classes": [
            "TECH_REVIEW",
            "REBUILD_FROM_SOURCE"
        ],
        "ambiguous_topics_resolved_to_tech_review": 81,
        "ambiguous_topics_resolved_to_rebuild_from_source": 40,
        "classification_source_metadata_added": true,
        "existing_corpus_statistics_preserved": true,
        "contamination_statistics_preserved": true
    },
    "clean_rebuild_direction": {
        "source_policy_by_domain": true,
        "freshness_policy": true,
        "revalidation_policy": true,
        "high_stakes_detection": true,
        "primary_source_requirements": true,
        "independent_crosscheck_requirements": true,
        "technical_validation_requirements": true,
        "synthetic_fixed_trust_rejected": true,
        "focused_retrieval_text_contract": true
    },
    "interpretation": [
        "The legacy corpus remained useful as historical/discovery material but was not accepted as a trustworthy production knowledge base.",
        "Recovering the original topic taxonomy allowed KESPA to preserve subject coverage while discarding synthetic trust assumptions.",
        "The 729-topic recovered manifest became the planning substrate for the later evidence-bound Knowledge Factory and canonical blueprint."
    ],
    "limitations": [
        "This note documents taxonomy recovery and rebuild policy, not factual validation of the legacy cards.",
        "The historical source record reports ~442,577 synthetic cards at this stage; later production records reference ~441,197 cards after subsequent system changes. These should not be treated as the same snapshot.",
        "TECH_REVIEW does not mean a topic was already production-ready; it indicates a different rebuild/validation path than REBUILD_FROM_SOURCE.",
        "Heuristic research-lead terms in the recovered manifest were not promoted as facts."
    ]
}

JSON

provenance.json

899 bytes SHA-256 502d3b94276c035a…

{
    "schema": "kespa.public_provenance.v1",
    "research_id": "KESPA-NOTE-005",
    "source_basis": "Historical KESPA changelog/Jira export supplied for public research reconstruction.",
    "source_observations": [
        "Recovered all 729 original subject areas.",
        "Resolved all 121 ambiguous classifications.",
        "Finalized 484 TECH_REVIEW and 245 REBUILD_FROM_SOURCE topics.",
        "Eliminated the MANUAL_CLASSIFY backlog.",
        "Preserved the 442,577-card synthetic corpus as immutable archive material.",
        "Rejected synthetic seed cards for clean production indexing.",
        "Defined the clean Knowledge Factory rebuild direction."
    ],
    "publication_note": "This public note contains aggregate taxonomy/recovery measurements and methodology only. The historical synthetic card bodies, contamination examples, private implementation scripts, and internal corpus files are not republished."
}