Embedding model, chunking, retrieval math

The knowledge vault is the answer to "what should I know about this drift signal?" The retrieval stack is a hybrid: dense vector KNN plus sparse lexical BM25, fused by Reciprocal Rank Fusion, decayed by age, then re-ranked by graph centrality. Four small SQLite-backed processes serve one HTTP endpoint each. All math is reproducible: the indexer source — the chunking, lexical, and graph-compute modules — ships inside the downloadable kit.

The embedding model ! Immutability per cache. The embedding_dim is written into the SQLite schema at cache creation. Swapping the EMBEDDING_MODEL env var without deleting the cache crashes startup (the dimension wouldn't match). To swap models cleanly: stop the indexer → delete each domain's cache file → restart → full re-embed.

Chunking strategy ! "Tokens" here means whitespace words. The config keys are named chunk_target_tokens=600 and chunk_overlap_tokens=80, but the chunker is a deliberately naive token-approximation: it counts whitespace-split words, and no tokenizer runs at chunking time. Earlier drafts of this doc flip-flopped between "words" and "tokens" — the code is the referee.

The chunker is a routine in the vault indexer. It is heading-aware by design, with a size target and overlap counted in whitespace words (a cheap token approximation — the config keys say "tokens", the loop counts words):

The four indexer processes

Scope derivation

Each process boots with DOMAIN=<slug> and reads _domains.yaml at the vault root. From the registry it derives INCLUDE/EXCLUDE prefixes for every walk. The finance indexer's scope is legacy: true: it includes everything except other domains' subfolders (auto-derived).

# _domains.yaml (abridged)
finance:
  legacy: true
  taxonomy: _taxonomy.md
  review_queue: _review-queue.md
  embedding: BAAI/bge-large-en-v1.5
  embedding_dim: 1024
  decay:
    mode: ranked_grouped
    ladder: [1.0, 0.6, 0.45, 0.35, 0.25]
    recency_boost_days: 30
    recency_boost_amount: 0.05
    kind_overrides:
      Filings: { group_by_path_prefix: [1.0, 0.6] }
  lexical:
    weight: 0.5

fitness: { ... decay.mode: off, lexical.weight: 0.05 }
nutrition: { ... decay.mode: off, lexical.weight: 0.05 }
learning: { ... decay.mode: off, lexical.weight: 0.05 }

Cold-start tagging. When a brand-new note is ingested, a small Claude Haiku pass proposes 1–5 controlled-vocabulary tags from the domain taxonomy. It is best-effort and operator-review-gated through the domain's _review-queue.md — the model suggests, the operator confirms before tags become canonical.

The retrieval stack — four layers

Layer 1 · Hybrid Reciprocal Rank Fusion

Vector KNN and FTS5 BM25 run independently. Each produces a ranked list. RRF combines them rank-wise (not score-wise):

rrf_score(d) = Σ over retrievers r:  weight(r) × 1 / (k + rank_r(d))

# k = 60 (standard RRF constant)
# rank_r(d) = position of doc d in retriever r's ranking (1-indexed)
# weight(r) per-domain in _domains.yaml
#   finance: vector=1.0, lexical=0.5   (lexical matters for tickers/filings)
#   others:  vector=1.0, lexical=0.05  (timeless concepts → vector-dominant)

RRF is robust against score-scale mismatch between vector cosine [-1, 1] and BM25 (unbounded). By using rank position rather than raw score it works well even when one retriever is dominant.

Layer 2 · Decay (finance only by default)

The finance corpus has time-sensitive material (filings, snapshots, news). The mode is ranked_grouped — and despite earlier wording on this site, it is not a time-based half-life. Results are grouped by author, ranked within each author group by published_at descending, and the ladder [1.0, 0.6, 0.45, 0.35, 0.25] is applied by rank-position, with 0.25 as the floor past the ladder's length:

• An author's most recent item: rank 0 → ×1.0; each older item steps down the ladder
• A single-item author group → rank 0 → ×1.0 (no penalty)
• Evergreen-tagged nodes opt out — always ×1.0
• The Filings kind_overrides uses a path-prefix grouping [1.0, 0.6] — current quarter vs everything else

Why rank-within-author rather than a time half-life: authors publish on wildly different cadences (a daily newsletter vs a quarterly letter). A pure time decay would bury infrequent and evergreen sources regardless of quality; ranking within each author gives every source a fair shot at the top slot on its own cadence. (Not to be confused with the operator-attention signal, which does use a 7-day half-life — different mechanism, see the Lakshmi door.)

Fitness, nutrition, and learning use mode: off — content is timeless.

Layer 3 · Graph re-ranking

When citation edges exist (a vault note links to another), a graph re-rank applies a linear combination:

hybrid_score(d) = α × vector_score(d)
                + β × pagerank(d)
                + γ × eigenvector_centrality(d)
                + recency_bonus(d)

# α = 0.6, β = 0.25, γ = 0.15
# recency_bonus = +0.05 if modified within 30 days, else 0
# applies to ALL FOUR DOMAINS (not finance-only)
# only activates if ≥10 citation edges exist in the touched subgraph

• PageRank rewards well-cited notes — the canonical references in your reading. Computed on a directed graph of citation edges only.
• Eigenvector centrality rewards notes that are well-cited by other well-cited notes — the "load-bearing" concepts. Computed on an undirected graph of citations + wikilinks (also the input to Louvain community clustering); falls back to degree centrality if the eigenvector solver fails to converge.
• Recency boost ensures fresh thinking is not buried under decade-old classics.

Graceful degradation. If a node has no PageRank or no centrality (e.g. brand-new, uncited), that term's weight folds back into α rather than scoring it as zero — so a fresh note still ranks on pure vector similarity instead of being penalised for lacking graph history.

Iterative-deepening graph traversal

/traverse/<path>?depth=N seeds the graph search from a specific node and walks the citation graph outward. The traversal is quality-gated, not exhaustive:

• Depth cap: 4 hops (max_hops)
• Beam width: 5 — at most 5 candidates carried forward per hop
• Per-hop prune floor: 0.50 — a candidate below 0.50 cosine to the query is dropped at that hop
• Early-stop condition: stop once the pool reaches target_k = 10 AND the pool's average similarity is ≥ 0.65 (most queries converge in 1–2 hops)

The two thresholds do different jobs: 0.50 prunes individual weak nodes at each hop; 0.65 is the average-quality bar that lets the search stop early. They are not the same floor.

Typical traversal touches ~100 embeddings per query — never bulk-loads.

Deep retrieval mode — retrieve wide, filter late

The fast path above is tuned to be cheap and always-on (~100 embeddings per query). When its thesis_match comes back weak or the corpus is sparse, an opt-in deep mode runs the same graph search with the brakes off. Crucially it does not run in the always-on app — it runs on-demand in a Claude Code session (the operator runs /rx-deep-retrieve <rec-id>), where the <$2/mo and <100-embeddings/query budgets don't apply. The retrieval itself is pure Python over the local bge model + sqlite-vec — zero LLM, no API key, no billing; the curated result POSTs back to the app's write-back endpoint (/v1/rx/deep) so the panel can show it. This cost seam is what makes deep retrieval affordable (see the enrichment lane on Pipeline).

The principle is separate retrieval from filtering. Deep mode retrieves wide and attaches metadata to every candidate, then lets the judgment layer (running where LLM judgment is ~free) filter with full visibility — so nothing is dropped before it is evaluated. Each retained candidate carries its similarity, hop distance, decay_weight, a decay_zero flag, and a retain_reason (seed · kept_despite_decay_zero · above_prune_floor). Pruned candidates are also returned — each with a drop_reason (below_prune_threshold · beam_overflow) — so a dropped-but-relevant doc can be rescued by the curator instead of vanishing silently.

Citation verification — is the quote actually in the source?

A separate, deterministic, app-side check (no LLM): at rec compose and at deep-result write-back, every quoted span must be a normalized substring of the chunk it cites. Normalization is, in order: NFKC unicode → smart-punctuation fold (curly quotes → straight, en/em/minus dashes → -, non-breaking space → space) → casefold → whitespace collapse. Ellipsis-elided quotes are split and each fragment must appear in order. Quotes shorter than 12 normalized characters are marked too_short — unverifiable, and deliberately not a pass, so a trivial match can't manufacture confidence.

Excerpt extraction

After top-K chunks are picked, each one's excerpt (the snippet shown in the rec's "Sources" section) is selected sentence-by-sentence:

1. Split chunk into sentences
2. Encode each sentence with the same bge prefix
3. Cosine-score against the query embedding
4. Return the top-2 sentences in document order

This keeps citations focused — instead of a 600-word chunk wall, the operator sees the two sentences that best match the rec's framing question.

End-to-end query latency budget

Why this model + stack?

Four design choices. Click any to expand.

Caches: what's inside each domain's cache file

Endpoints — the indexer HTTP surface

TradingV's vault module proxies these endpoints from the FastAPI app to the indexer. The frontend never hits Port 1 directly — it hits /v1/vault/search which forwards.