Technical note verified Sep 13, 2026

Clean Verified Knowledge Production Baseline

KESPA replaced the legacy ~441k-card production retrieval corpus with the clean 423-claim verified baseline, preserved memory collections separately, validated known-claim retrieval at rank #1, and established trusted_knowledge as the authoritative knowledge store with Chroma as a rebuildable search index.

production-cutover verified-knowledge chroma retrieval architecture

MARKDOWN

README.md

2,872 bytes SHA-256 ac7b7dc0d2510b07…

KESPA-NOTE-002 — Clean Verified Knowledge Production Baseline

Date: 2026-09-13 Status: Verified — production cutover completed Project: KESPA AI / NexLabs Studios

What changed

KESPA retired the legacy ~441,197-card corpus from live production retrieval and replaced it with the clean 423-claim verified atomic corpus.

The legacy corpus was retained as historical/research material rather than copied into the new live knowledge index.

Clean Chroma build

The clean retrieval index was built from:

D:\forge\verified_corpus\clean_verified_claim_corpus_v001

into:

D:\forge\chroma_db_clean_v001

with:

collection: forge_cards
verified claims indexed: 423
embedding model: all-MiniLM-L6-v2
distance: cosine
build result: PASS

Expected claims and indexed cards matched exactly:

423 == 423

Retrieval smoke validation

Before production cutover, the clean Chroma database was opened and queried against a known verified claim.

Observed:

clean cards: 423
expected claim ranked: #1
top similarity: 0.8409
verified metadata: true
claim verdict: SUPPORTED
expected terms present: 4/4
Chroma writes during validation: 0

This was a smoke validation, not a full retrieval benchmark.

Production cutover

The production cutover then:

retired the legacy ~441k corpus from live retrieval;
promoted the clean 423-claim index into production;
copied 0 legacy cards into the new dataset;
preserved 13 chat-memory records;
preserved 5 user-memory records;
changed no source code during the cutover itself.

Architecture boundary

After the cutover:

MySQL trusted_knowledge is authoritative. Chroma is the rebuildable retrieval index.

This is important because an embedding database should not become the sole source of truth for trusted knowledge.

Trusted knowledge can be re-indexed into Chroma while retaining its canonical provenance, verification, approval, and lifecycle state elsewhere.

First live knowledge growth

The first manually trained/promoted trusted record was indexed after the clean baseline:

423 -> 424

Subsequent operational records later observed:

423 -> 424 -> 425 -> 426

as trusted knowledge entered the live index.

Those counts are operational milestones, not a controlled quality experiment.

Research relevance

The cutover operationalized the result of the earlier corpus and retrieval experiments:

large uncontrolled accumulation was removed from the live path;
the verified atomic corpus became the starting production baseline;
later knowledge growth had to enter through the trusted-knowledge lifecycle rather than silently expanding the retrieval corpus.

This created the production boundary needed for longitudinal contamination and autonomous-learning research.

TEXT

SHA256SUMS.txt

393 bytes SHA-256 aad672b5cd8b79fc…

ac7b7dc0d2510b07d577eaade144492349521a93f810c73568f47310dc811cea  README.md
b1c9e616590d74192ef0dba71a9a63c64de9b262b5a961f0677f708343d180f0  note.json
167dc01b244b849ddd26625452c1560128739b85ec898b97ecb707a2fefd7c12  metrics.csv
014125113cb22a51a9ee3a70b456b10776a88d82caa74c0bbd4b39a4af5f5bb0  methodology.md
6a14d93fbd1ea725bf1a4624f9ad655beb8035323c0b9231e985f703d9f35d38  provenance.json

MARKDOWN

methodology.md

1,018 bytes SHA-256 014125113cb22a51…

Methodology

Build

A new isolated Chroma database was constructed from the frozen 423-claim clean verified corpus.

The build was validated by comparing expected claim count to indexed card count.

Smoke retrieval

A known verified claim was queried against the isolated clean collection. The test checked:

rank position;
similarity score;
verified metadata;
supported verdict metadata;
expected content terms;
absence of write operations.

Cutover

After the isolated build and retrieval smoke passed, the clean index replaced the legacy production knowledge collection.

Memory collections were handled separately from the retired legacy forge_cards corpus so that user/chat memory preservation did not reintroduce legacy knowledge cards.

Source-of-truth rule

The production architecture treats trusted_knowledge in MySQL as authoritative and Chroma as a rebuildable semantic retrieval index.

This separation allows index regeneration without redefining what knowledge is trusted.

CSV

metrics.csv

424 bytes SHA-256 167dc01b244b849d…

metric,value,unit_or_status
legacy_cards_retired,441197,approx_cards
clean_verified_claims,423,claims
clean_chroma_indexed_cards,423,cards
retrieval_known_claim_rank,1,rank
retrieval_top_similarity,0.8409,cosine_similarity
expected_terms_present,4/4,terms
chat_memories_preserved,13,records
user_memories_preserved,5,records
legacy_cards_copied,0,cards
first_post_cutover_live_count,424,cards
production_cutover,PASS,status

JSON

note.json

2,640 bytes SHA-256 b1c9e616590d7419…

{
    "schema": "kespa.public_technical_note.v1",
    "id": "KESPA-NOTE-002",
    "title": "Clean Verified Knowledge Production Baseline",
    "date": "2026-09-13",
    "status": "verified",
    "purpose": "Document the production cutover from the archived legacy corpus to the clean verified atomic-claim baseline and the architecture boundary between authoritative knowledge storage and rebuildable retrieval indexing.",
    "pre_cutover": {
        "legacy_card_count_approx": 441197,
        "legacy_role": "retired from live production retrieval and retained as historical/research material",
        "clean_verified_claims": 423,
        "chat_memory_records_preserved": 13,
        "user_memory_records_preserved": 5
    },
    "clean_index_build": {
        "source": "D:\\forge\\verified_corpus\\clean_verified_claim_corpus_v001",
        "target": "D:\\forge\\chroma_db_clean_v001",
        "collection": "forge_cards",
        "indexed_verified_claims": 423,
        "embedding_model": "all-MiniLM-L6-v2",
        "distance": "cosine",
        "expected_claims": 423,
        "indexed_cards": 423,
        "build_result": "PASS"
    },
    "retrieval_validation": {
        "status": "PASS",
        "clean_cards": 423,
        "known_claim_rank": 1,
        "top_similarity": 0.8409,
        "verified_metadata": true,
        "claim_verdict": "SUPPORTED",
        "expected_terms": "4/4",
        "chroma_writes_during_validation": 0
    },
    "production_cutover": {
        "status": "COMPLETE",
        "live_production_baseline_claims": 423,
        "legacy_cards_copied_into_new_dataset": 0,
        "source_code_changed_during_cutover": false
    },
    "post_cutover_growth": {
        "first_promoted_trusted_knowledge_id": 1,
        "live_count_after_first_promotion": 424,
        "later_observed_autonomous_growth_sequence": [
            423,
            424,
            425,
            426
        ]
    },
    "architecture": {
        "authoritative_store": "MySQL trusted_knowledge",
        "retrieval_index": "Chroma forge_cards",
        "index_semantics": "rebuildable search index, not source of truth",
        "private_plaintext_indexing": "blocked for promoted public knowledge path",
        "legacy_corpus_status": "archived / research history"
    },
    "limitations": [
        "The single-query retrieval validation is a production smoke test, not a broad retrieval benchmark.",
        "The post-cutover growth sequence records observed operational milestones and should not be interpreted as a controlled experiment.",
        "The legacy corpus size is reported from the historical operational record and is approximate in this note.",
        "This note documents architecture and cutover integrity rather than end-user quality improvement; quality effects are covered by separate benchmark records."
    ]
}

JSON

provenance.json

1,470 bytes SHA-256 6a14d93fbd1ea725…

{
    "schema": "kespa.public_provenance.v1",
    "research_id": "KESPA-NOTE-002",
    "source_basis": "Historical KESPA changelog/Jira export supplied for public research reconstruction.",
    "recorded_clean_corpus_hashes": [
        {
            "path": "verified_corpus/clean_verified_claim_corpus_v001/manifest.json",
            "recorded_sha256": "48764DDEDB2D094DCD621E030321653C557494C872EE9ADAC405FD58022FAAA9"
        },
        {
            "path": "verified_corpus/clean_verified_claim_corpus_v001/validation_summary.txt",
            "recorded_sha256": "30F75EC56D22C811CFC171F31F1689AF7126C3DE6E9B12F9108EA672C77A91EA"
        },
        {
            "path": "verified_corpus/clean_verified_claim_corpus_v001/verified_claims.jsonl",
            "recorded_sha256": "D0BD8E02484472263A8D38E82023DD8CBEFC4B70AC86BA31713E04A63044352B"
        },
        {
            "path": "verified_corpus/clean_verified_claim_corpus_v001/verified_evidence.jsonl",
            "recorded_sha256": "0C181ED53224EE5E14A09037A49840F3D3A171C7CFF8322E090C3C64ABF13C4A"
        }
    ],
    "hash_verification_note": "These hashes are transcribed from the historical user-environment record. The underlying corpus artifacts were not supplied with the changelog export, so this public package does not claim independent rehashing.",
    "publication_note": "This public note contains architecture, aggregate counts, validation outcomes, and recorded provenance. Private memory contents, source code, raw Chroma data, and internal database records are not republished."
}