Embedding model, chunking, retrieval math

The knowledge vault is the answer to "what should I know about this drift signal?" The retrieval stack is a hybrid: dense vector KNN plus sparse lexical BM25, fused by Reciprocal Rank Fusion, decayed by age, then re-ranked by graph centrality. Four small SQLite-backed processes serve one HTTP endpoint each. All math is reproducible from the source code in tools/vault_indexer/.

The embedding model

Chunking strategy

The chunker lives in vault.py:chunk_body(). It is heading-aware by design, with a token target and overlap:

The four indexer processes

Scope derivation

Each process boots with DOMAIN=<slug> and reads _domains.yaml at the vault root. From the registry it derives INCLUDE/EXCLUDE prefixes for every walk. The finance indexer's scope is legacy: true: it includes everything except other domains' subfolders (auto-derived).

# _domains.yaml (abridged)
finance:
  legacy: true
  taxonomy: _taxonomy.md
  review_queue: _review-queue.md
  embedding: BAAI/bge-large-en-v1.5
  embedding_dim: 1024
  decay:
    mode: ranked_grouped
    ladder: [1.0, 0.6, 0.45, 0.35, 0.25]
    recency_boost_days: 30
    recency_boost_amount: 0.05
    kind_overrides:
      Filings: { group_by_path_prefix: [1.0, 0.6] }
  lexical:
    weight: 0.5

fitness: { ... decay.mode: off, lexical.weight: 0.05 }
nutrition: { ... decay.mode: off, lexical.weight: 0.05 }
learning: { ... decay.mode: off, lexical.weight: 0.05 }

The retrieval stack — four layers

Layer 1 · Hybrid Reciprocal Rank Fusion

Vector KNN and FTS5 BM25 run independently. Each produces a ranked list. RRF combines them rank-wise (not score-wise):

rrf_score(d) = Σ over retrievers r:  weight(r) × 1 / (k + rank_r(d))

# k = 60 (standard RRF constant)
# rank_r(d) = position of doc d in retriever r's ranking (1-indexed)
# weight(r) per-domain in _domains.yaml
#   finance: vector=0.5, lexical=0.5
#   others:  vector=0.95, lexical=0.05

RRF is robust against score-scale mismatch between vector cosine [-1, 1] and BM25 (unbounded). By using rank position rather than raw score it works well even when one retriever is dominant.

Layer 2 · Decay (finance only by default)

The finance corpus has time-sensitive material (filings, snapshots, news). Decay groups results into recency tiers and multiplies the RRF score by a tier weight. The default ladder is [1.0, 0.6, 0.45, 0.35, 0.25]:

• Most recent group: weight ×1.0
• Each older group: progressively reduced multiplier
• Evergreen-tagged nodes opt out — they always get ×1.0
• The Filings kind_overrides uses a path-prefix grouping [1.0, 0.6] — current quarter vs everything else

Fitness, nutrition, and learning use mode: off — content is timeless.

Layer 3 · Graph re-ranking

When citation edges exist (a vault note links to another), a graph re-rank applies a linear combination:

hybrid_score(d) = α × vector_score(d)
                + β × pagerank(d)
                + γ × eigenvector_centrality(d)
                + recency_bonus(d)

# α = 0.6, β = 0.25, γ = 0.15
# recency_bonus = +0.05 if modified within 30 days, else 0
# applies to ALL FOUR DOMAINS (not finance-only)
# only activates if ≥10 citation edges exist in the touched subgraph

• PageRank rewards well-cited notes — the canonical references in your reading.
• Eigenvector centrality rewards notes that are well-cited by other well-cited notes — the "load-bearing" concepts.
• Recency boost ensures fresh thinking is not buried under decade-old classics.

Iterative-deepening graph traversal

/traverse/<path>?depth=N seeds the graph search from a specific node and walks the citation graph outward. The traversal is quality-gated, not exhaustive:

• Depth cap: 4 hops default
• Beam width: pruned at each hop
• Quality floor: 0.65 vector similarity to query
• Target K: 10 results
• Early stop: when average similarity plateaus

Typical traversal touches ~100 embeddings per query — never bulk-loads.

Excerpt extraction

After top-K chunks are picked, each one's excerpt (the snippet shown in the rec's "Sources" section) is selected sentence-by-sentence:

1. Split chunk into sentences
2. Encode each sentence with the same bge prefix
3. Cosine-score against the query embedding
4. Return the top-2 sentences in document order

This keeps citations focused — instead of a 600-token chunk wall, the operator sees the two sentences that best match the rec's framing question.

End-to-end query latency budget

Why this model + stack?

Four design choices. Click any to expand.

Caches: what's inside cache-<domain>.db

Endpoints — the indexer HTTP surface

TradingV's app/vault/ module proxies these endpoints from the FastAPI app to the indexer. The frontend never hits :8001 directly — it hits /v1/vault/search which forwards.