Overview
Dual-mode RAG application demonstrating both traditional and multimodal RAG capabilities. Public mode leverages AWS Bedrock for cloud-based inference, while Private mode runs fully local with Ollama and Docling, ensuring no data ever leaves your machine. The system supports PDF, DOCX, TXT, and image files through two distinct pipelines -- text extraction with layout-aware document processing, and direct multimodal image embedding -- giving users flexibility across document formats while maintaining control over data privacy.
Architecture
Documents (PDF, DOCX, TXT, Images)
↓
Docling (Private) / Bedrock Vision (Public)
↓
Semantic Chunking → Text Embedding → ChromaDB
↓
Hybrid Retrieval (Vector + BM25 + Metadata)
↓
LLM (Bedrock or Ollama) → Answer
Pipeline 2: Multimodal Image Embedding
Image Upload
↓
Nomic Vision (Private) / Cohere Embed v4 (Public)
↓
ChromaDB Image Collection (same latent space as text)
↓
Text Query → Cross-Modal Retrieval → Original Image
↓
Vision LLM → Answer
Dual Mode Design
Public Mode uses AWS Bedrock Claude for generation, Bedrock Vision for OCR, Titan Embed v2 for text embeddings, and Cohere Embed v4 for multimodal image embeddings. Conversation memory is managed by Mem0 vector store for intelligent context retention.
Private Mode runs entirely on your local machine using Ollama for LLM inference, Docling for layout-aware document processing, and Nomic embeddings for both text and vision in a shared latent space. Conversation memory uses a sliding window approach. No data leaves the device.
Every step in the pipeline has a private/public branch -- from text extraction through chunking, embedding, retrieval, metadata extraction, generation, and memory management. The privacy trade-off is straightforward: Public mode gives you access to frontier models at the cost of sending data to AWS. Private mode keeps everything local at the cost of requiring local compute resources.
Key Features
- Docling document processing -- IBM's open-source (MIT) layout-aware document processor replaces PyPDF2, Tesseract, and python-docx with a single tool that preserves tables, headings, and document structure as Markdown.
- Multimodal image embeddings -- Images can be embedded directly into a shared vector space with text using Nomic Vision (768d, private) or Cohere Embed v4 (1024d, public), enabling text-to-image cross-modal search.
- Vision LLM at generation time -- Retrieved images are passed as original files to vision models (Ollama qwen2.5-vl or Bedrock Claude), so the LLM sees the actual image, not just OCR text.
- Hybrid retrieval -- Combines dense vector search, BM25 keyword search, and LLM-based metadata filtering with weighted ensemble for better recall.
- Multi-format support -- PDF, DOCX, TXT, and images (PNG, JPG, TIFF, BMP, WebP) through a unified extraction pipeline.
- Conversation memory -- Sliding window (private) or Mem0 vector memory (public) for multi-turn context.