Experiment verified Sep 2, 2026

Evidence Discovery: From Search Dependency to Source-First Discovery

Three bounded discovery iterations showed that general web search was not reliable enough to be a required evidence locator in the tested Windows KESPA runtime. A source-first design using catalog hints, first-party entrypoints, robots.txt, sitemaps, and same-host navigation produced candidates for 26 of 28 smoke slots without Google or DuckDuckGo.

knowledge-factory evidence-discovery source-first failure-analysis authoritative-sources

MARKDOWN

README.md

4,141 bytes SHA-256 2e8ed7983c28d16c…

KESPA-EXP-002 — Evidence Discovery: From Search Dependency to Source-First Discovery

Date: 2026-09-02 Status: Verified historical experiment Project: KESPA AI / NexLabs Studios

Research question

Can KESPA reliably locate candidate evidence for a fixed representative smoke set using general web search as a required dependency, and if not, does deterministic first-party source discovery provide materially better coverage without weakening source trust policy?

Fixed test context

The same representative smoke set covered:

28 evidence slots
24 cards
24 canonical topics

The larger validated blueprint behind the smoke contained:

4,807 evidence slots
3,982 cards
759 canonical topics

The smoke runs were intentionally bounded. None of these runs wrote cards, embeddings, Chroma data, Brain state, or production database records.

Iteration 1 — v001 live evidence smoke

The initial design used search/discovery followed by bounded evidence fetching.

Observed result:

Network requests: 112
Candidates: 12
Evidence excerpts: 1
NO_CANDIDATE: 25 slots
DISCOVERY_FAILED HTTP 403: 2 slots
Cards passing evidence gate: 1 / 24

Verdict: do not scale the design to the full 4,807-slot queue.

Iteration 2 — v001.1 search repair

The next revision fixed two mechanical discovery defects:

1. source-family priority could no longer make an unrelated source eligible;

2. generic scope/title language could no longer overpower canonical topic matching.

The source trust policy was not loosened.

The discovery-only rerun then produced:

Search attempts: 136
Network requests: 136
Candidates: 0
Candidate slots: 0 / 28

Provider diagnostics:

Google CSE: 68 attempts, 0 result-bearing queries, 68 errors
DuckDuckGo: 68 attempts, 0 result-bearing queries, 17 errors

Verdict: general web search was not reliable enough to remain a required locator in the tested runtime.

Iteration 3 — v001.2 source-first discovery

General search was disabled for the bounded smoke (provider=none).

KESPA instead tried candidate location in this order:

1. catalog topic-specific URL hints;

2. catalog first-party entrypoints;

3. first-party robots.txt sitemap declarations;

4. common first-party sitemap locations;

5. bounded same-host navigation/link discovery.

Observed result:

Network requests: 275
Discovery attempts: 297
Candidates discovered: 249
Slots with candidates: 26 / 28 (92.9%)
DISCOVERED_APPROVED_CANDIDATE: 18
DISCOVERED_REVIEW_REQUIRED: 8
NO_SEARCH_RESULT: 2

First-party discovery diagnostics:

Navigation: 84 attempts / 75 result-bearing / 7 errors
robots.txt: 36 attempts / 17 result-bearing / 7 errors
Sitemap discovery: 177 attempts / 76 result-bearing / 95 errors

Result

The experiment did not show that search engines are universally bad discovery tools.

It showed something narrower and more useful:

In this KESPA runtime and fixed smoke set, general web search was not reliable enough to be a required

evidence locator, while bounded source-first discovery produced candidate coverage for 26 of 28 slots

without weakening the source-trust policy.

Search could therefore remain an optional fallback rather than a critical dependency.

Trust boundaries preserved

Across the revisions:

source-policy catalog: 60 hosts
auto-fetch-approved hosts: 37
unknown/new publishers remained REVIEW_REQUIRED
locator metadata was not evidence
no source family gained automatic trust merely because discovery improved

What this does not prove

This was a bounded discovery experiment, not a full-corpus acquisition run. It does not establish:

full 4,807-slot coverage;
evidence quality for every discovered candidate;
universal superiority of source-first discovery;
general search-provider performance outside the tested runtime.

The next gate after successful source-first discovery was evidence-quality auditing, not automatic knowledge generation.

TEXT

SHA256SUMS.txt

399 bytes SHA-256 db19a080304d3de3…

2e8ed7983c28d16c81669c3c2ed5100a00b45aa842f1a1fd4dd779cfd40016ed  README.md
2828fde07e60525e5f053510e0428dffb6fcc736ef49f59a318a2086df45b762  experiment.json
1aa98aae7f7ac2dd45a62b1337c8e02bd9292fccf3508e11077fc4b9b8d42ec1  results.csv
75cd333259d95fb726468674f0af3f910df6d9827d89319078f0f5d1fc0509b5  methodology.md
d706c6381ab21fcf72f20167c45bfe1c984c254df84599d7df04188d1293d4eb  provenance.json

JSON

experiment.json

4,829 bytes SHA-256 2828fde07e60525e…

{
    "schema": "kespa.public_experiment.v1",
    "id": "KESPA-EXP-002",
    "title": "Evidence Discovery: From Search Dependency to Source-First Discovery",
    "date": "2026-09-02",
    "status": "verified",
    "research_question": "Can KESPA reliably locate candidate evidence for a fixed representative smoke set using general web search as a required dependency, and if not, does deterministic first-party source discovery provide materially better coverage without weakening source trust policy?",
    "fixed_smoke_set": {
        "queue_slots": 28,
        "unique_cards": 24,
        "unique_topics": 24
    },
    "canonical_blueprint_context": {
        "evidence_slots": 4807,
        "cards": 3982,
        "canonical_topics": 759
    },
    "iterations": [
        {
            "name": "v001_live_evidence_smoke",
            "design": "Search/discovery followed by bounded evidence fetching.",
            "network_requests": 112,
            "candidates": 12,
            "evidence_excerpts": 1,
            "failures": 2,
            "slot_outcomes": {
                "evidence_collected_pending_audit": 1,
                "no_candidate": 25,
                "discovery_failed_http_403": 2
            },
            "quality_audit": {
                "pass": 0,
                "pass_with_warnings": 1,
                "needs_review": 0,
                "cards_passing_gate": 1,
                "cards_total": 24
            },
            "verdict": "DO NOT SCALE. Repair discovery mechanics and rerun a bounded discovery-only smoke."
        },
        {
            "name": "v001_1_search_repair",
            "design": "Discovery-only rerun after fixing source-family eligibility and query mechanics while retaining Google CSE and DuckDuckGo.",
            "network_requests": 136,
            "search_attempts": 136,
            "candidates": 0,
            "evidence_excerpts": 0,
            "candidate_slots": 0,
            "queue_slots": 28,
            "provider_results": {
                "google": {
                    "attempts": 68,
                    "queries_with_results": 0,
                    "errors": 68
                },
                "duckduckgo": {
                    "attempts": 68,
                    "queries_with_results": 0,
                    "errors": 17
                }
            },
            "verdict": "DO NOT SCALE SEARCH-DEPENDENT DISCOVERY. General web search was not a reliable required locator in this runtime."
        },
        {
            "name": "v001_2_source_first",
            "design": "General search disabled. Candidate discovery used catalog URL hints, first-party entrypoints, robots.txt sitemap declarations, common sitemap locations, and bounded same-host navigation.",
            "search_provider_mode": "none",
            "network_requests": 275,
            "search_attempts": 297,
            "candidates": 249,
            "evidence_excerpts": 0,
            "slots_with_candidates": 26,
            "queue_slots": 28,
            "queue_results": {
                "discovered_approved_candidate": 18,
                "discovered_review_required": 8,
                "no_search_result": 2
            },
            "provider_diagnostics": {
                "first_party_nav": {
                    "attempts": 84,
                    "queries_with_results": 75,
                    "errors": 7
                },
                "first_party_robots": {
                    "attempts": 36,
                    "queries_with_results": 17,
                    "errors": 7
                },
                "first_party_sitemap": {
                    "attempts": 177,
                    "queries_with_results": 76,
                    "errors": 95
                }
            },
            "verdict": "Source-first discovery materially improved candidate coverage while preserving source trust boundaries; proceed to evidence-quality auditing before card generation."
        }
    ],
    "derived_results": {
        "source_first_candidate_slot_rate": 0.9285714285714286,
        "source_first_candidate_slot_percent": 92.9,
        "search_repair_candidate_slot_rate": 0,
        "search_repair_candidate_slot_percent": 0,
        "absolute_candidate_slot_gain": 26,
        "source_first_approved_candidate_slots": 18,
        "source_first_review_required_slots": 8,
        "source_first_unresolved_slots": 2
    },
    "trust_policy": {
        "catalog_hosts": 60,
        "auto_fetch_approved_hosts": 37,
        "unknown_new_hosts": "REVIEW_REQUIRED",
        "trust_policy_weakened": false,
        "new_source_family_auto_approvals_added": false
    },
    "hard_rails": {
        "llm_calls": 0,
        "card_generation": 0,
        "chroma_writes": 0,
        "brain_changes": 0,
        "database_writes": 0
    },
    "limitations": [
        "This experiment measures discovery mechanics on one deterministic 28-slot smoke set; it does not prove full 4,807-slot coverage.",
        "The v001.2 run measured candidate discovery only. Candidate URLs were not automatically treated as evidence.",
        "First-party sitemap discovery generated many errors and therefore remained only one bounded discovery mechanism among several.",
        "PDF evidence support was deliberately deferred in these revisions.",
        "The experiment demonstrates that source-first discovery was more reliable than the tested general-search dependency in this runtime, not that general web search is universally unsuitable."
    ]
}

MARKDOWN

methodology.md

1,746 bytes SHA-256 75cd333259d95fb7…

Methodology

Experimental sequence

This public record reconstructs three historical KESPA evidence-discovery iterations performed on 2026-09-02 against the same deterministic 28-slot representative smoke set.

v001

Purpose: test the original evidence-acquisition path under bounded live network conditions.

The collector could perform search/discovery and fetch approved source pages. Source-policy, SSRF/private-network blocking, redirect handling, robots.txt behavior, excerpt limits, and explicit noai/noimageai exclusions remained active.

v001.1

Purpose: isolate discovery after two mechanical ranking/eligibility defects were identified.

Changes were limited to discovery mechanics:

priority alone could no longer make a source family eligible;
canonical-topic matching replaced generic title/scope matching;
topic matching became boundary-aware;
source queries became concise;
Google CSE used native site restriction;
provider diagnostics were recorded.

No source-trust policy was broadened.

v001.2

Purpose: test whether known authoritative source families could be located without requiring general search.

The smoke used provider=none, disabling Google and DuckDuckGo. Candidate discovery used:

catalog URL hints;
known first-party entrypoints;
robots.txt sitemap declarations;
common sitemap locations;
bounded same-host navigation.

Unknown hosts remained review-required. Candidate discovery did not itself authorize evidence ingestion.

Success interpretation

The experiment compares discovery coverage, not answer quality.

A candidate URL is only a locator. It is not trusted evidence until separately fetched, audited, and passed through the evidence-quality lifecycle.

JSON

provenance.json

1,261 bytes SHA-256 d706c6381ab21fcf…

{
    "schema": "kespa.public_provenance.v1",
    "experiment_id": "KESPA-EXP-002",
    "historical_source_archive": "forge_ai_research.zip",
    "source_artifacts": [
        {
            "path": "FORGE_evidence_acquisition_v001_1_windows_candidate/V001_LIVE_SMOKE_FINDINGS.txt",
            "sha256": "7a5153ca69741670de6109d61d8ec76098bc949ebc993ac6efa292526f44971a"
        },
        {
            "path": "FORGE_evidence_acquisition_v001_2_windows_candidate/V001_1_LIVE_DISCOVERY_FINDINGS.txt",
            "sha256": "f65c02bd4bd4dabc97594dbe7e8631c165ba4846fdb99b277e37a6a4c8cd12e1"
        },
        {
            "path": "FORGE_evidence_acquisition_v001_2_windows_candidate/source_discovery_smoke_run/evidence_acquisition_summary.txt",
            "sha256": "7df344dbe4b892c6b58faec5be160eef1b1881d62147418e4087b35b9bb67c36"
        },
        {
            "path": "FORGE_evidence_acquisition_v001_2_windows_candidate/source_discovery_smoke_run/search_diagnostics_summary.txt",
            "sha256": "ac8742890471d1942543faa23d0d9ff3b4493c036efc5e7694259277b5172873"
        }
    ],
    "publication_note": "The public package contains derived measurements and methodology. Historical implementation scripts, raw candidate lists, fetched third-party text, private configuration, and the original internal repository are not republished."
}

CSV

results.csv

443 bytes SHA-256 1aa98aae7f7ac2dd…

iteration,queue_slots,candidate_slots,candidate_slot_percent,network_requests,search_attempts,candidates_discovered,evidence_excerpts,approved_candidate_slots,review_required_slots,unresolved_slots,verdict
v001_live_evidence_smoke,28,1,3.6,112,,12,1,,,27,DO_NOT_SCALE
v001_1_search_repair,28,0,0.0,136,136,0,0,0,0,28,DO_NOT_SCALE_SEARCH_DEPENDENT_DISCOVERY
v001_2_source_first,28,26,92.9,275,297,249,0,18,8,2,PROCEED_TO_EVIDENCE_QUALITY_AUDIT