VaultRAG — Technical Deep Dive

core/config.py — Centralized configuration via Pydantic Settings with environment variable override. Single global instance. Mode-specific path isolation prevents cross-contamination between private and public databases.

• AWS credentials (optional for private mode)
• Model IDs for generation, embedding, vision, metadata per mode
• Feature flags: use_mem0_public, max_conversation_turns, bm25_max_documents
• Path getters: get_chroma_db_path(private), get_bm25_index_path(private)

core/cache.py — Lazy-loaded global singletons prevent expensive model reloading. Mode-specific caches use dictionaries keyed by {"private": ..., "public": ...}.

• SBERT embeddings (768d), Spacy NLP, BM25 retrievers, metadata schemas
• Custom NomicVisionEmbeddings: HuggingFace model, L2-normalized CLS token, 768d
• Custom CohereImageEmbeddings: Bedrock Cohere Embed v4, data URI format, 1024d

core/bedrock_client.py — Auto-detects model family from model ID prefix, normalizes request/response formats. Supports Claude, Llama, Mistral, Titan, and Cohere through a single invoke_text_model() interface.

• Text generation: unified + streaming variants
• Vision: invoke_vision_model() for image+text input (Claude only)
• Model switching requires only changing the model ID string

core/prompts.py — Separate templates for Ollama vs Bedrock. Schema-aware metadata extraction when field names are available. Output priming technique: prompts end with "Answer: The" to force direct responses without preambles like "Based on the documents...".

data_loading/text_extraction.py — Unified entry point extract_text_unified() routes by mode and file type. Private mode uses Docling (outputs Markdown preserving structure). Public mode uses Bedrock Vision for PDFs/images. Plain text files use direct read in both modes.

data_loading/chunking.py — Three strategies selectable per upload:

• Semantic — LangChain SemanticChunker with mode-specific embeddings, percentile breakpoint (0.7). Best quality, slowest.
• Spacy — Sentence-based splitting into ~3 chunks. Medium speed, good quality.
• Recursive — Character split at 600 chars with 30-char overlap. Fastest, adequate quality.

data_loading/indexing.py — Stores embeddings in mode-specific ChromaDB collections. Rebuilds BM25 sparse index from full corpus on each upload (skipped if >5000 docs). Auto-extracts metadata schema (unique field names) at zero extra cost. Image indexing uses separate collections with vision embeddings.

retrieval/metadata_extraction.py — Extracts structured filters from natural language queries. Three-tier fallback: Bedrock LLM (best accuracy, public) → Ollama (private) → Spacy NLP (regex + NER, always works). Returns single key-value pair for ChromaDB filtering.

retrieval/retriever.py — Combines up to 3 retrievers via LangChain's EnsembleRetriever with configurable weights. Chroma+filter (0.4) catches precise metadata matches, Chroma unfiltered (0.3) provides safety net, BM25 (0.3) handles exact keyword matches. Each retriever returns 15 internal candidates; ensemble re-ranks and returns top-k.

generation/generator.py — Routes generation to Ollama (private) or Bedrock (public). Integrates conversation memory, supports both text and vision generation, and provides streaming via SSE for public mode. Vision generation uses the first retrieved image with base64 encoding.

generation/memory.py — Private mode uses a simple in-memory deque per session (last 20 turns). Public mode uses Mem0 for semantic memory search (finds relevant memories by meaning, not just recency) with sliding window as resilient fallback. Public always writes to both stores. Mem0 initialization failures are non-fatal.

generation/pii.py — Reversible PII masking via Microsoft Presidio. Detects PERSON, SSN, EMAIL, PHONE, CREDIT_CARD, etc. Replaces with unique placeholders (<PERSON_1>). Currently implemented but not integrated into the API flow — ready for activation via a per-request flag.