PrivateDoc AI

"Your documents. Your device. Your data."

PrivateDoc AI is an offline, privacy-first on-device document intelligence and RAG (Retrieval-Augmented Generation) application built for the AI Build Challenge — Bengaluru (PS-001 Qualcomm Edge AI Track).

Designed for environments handling confidential records (medical dossiers, legal agreements, corporate intellectual property, and financial statements), PrivateDoc AI ensures that all document parsing, chunking, embeddings, vector indexing, and retrieval execute locally on the host device without external cloud AI dependencies.

---

1. Problem Statement

Organizations and individuals handling confidential documentation face a difficult trade-off:

• They require semantic search, cross-document comparison, and synthesized question-answering across complex documents.
• Public cloud AI services (OpenAI, Anthropic, Gemini, hosted vector DBs) present compliance, confidentiality, and data residency concerns.
• Cloud dependencies fail in air-gapped, field-deployed, or low-connectivity environments.

---

2. Solution: On-Device Document Intelligence

PrivateDoc AI executes every stage of the document intelligence pipeline on local edge hardware:

1. Local Text Extraction: PyMuPDF parses PDF, TXT, and Markdown files locally.
2. Page Coordinate Preservation: Every extracted passage strictly retains its source document_id, filename, and 1-based page_number.
3. Deterministic Chunking: Pages are chunked with configurable token/word boundaries (250 words, 40-word overlap) strictly bounded by page numbers to eliminate cross-page bleeding.
4. Local Embeddings: Dense 384-dimensional semantic vectors are generated on-device using local all-MiniLM-L6-v2 weights.
5. Local Vector Index: Vector indexing and cosine similarity search are executed locally via FAISS (IndexFlatIP).
6. Programmatic Citations: Citations are derived directly from deterministic retrieval metadata (filename, page number, chunk ID, exact supporting text)—never hallucinated by an LLM.
7. Strictly Grounded RAG Orchestration: Prompts enforce negative constraints ("Answer ONLY using supplied context; if absent, state 'I couldn't find that information in the indexed documents.'").
8. Multi-Document Comparison: Queries retrieve and attribute evidence across multiple documents with distinct page-level source citations.
9. Zero External API Calls: The core pipeline operates without any external network connections.

---

3. Architecture Overview

[ Sensitive Documents: PDF / TXT / MD ]
                  │
                  ▼
         [ PyMuPDF Local Parser ]
                  │ (Page metadata preserved)
                  ▼
        [ Deterministic Chunker ]
                  │ (Strictly page-bounded)
                  ▼
      [ EmbeddingProvider: Local ] ───────► [ QualcommEmbeddingProvider (Snapdragon Target) ]
                  │ (384-dim float32)
                  ▼
        [ Local FAISS Index ] ◄──────────► [ SQLite Metadata Store ]
                  │
        [ User Query / Prompt ]
                  │
                  ▼
         [ Local Retriever ]
                  │ (Cosine similarity + Threshold)
                  ▼
       [ Relevant Evidence Passages ] ───► [ Programmatic Citation Engine ]
                  │                                   │
                  ▼                                   ▼
        [ Grounded Context Prompt ]           [ Page-Level Citations ]
                  │
                  ▼
        [ LLMProvider: Local ] ──────────► [ QualcommLLMProvider (Snapdragon Target) ]
                  │
                  ▼
    [ Grounded Answer + Source Passages ]

---

4. Grounded RAG Pipeline

1. Extraction: PyMuPDF extracts text page-by-page, assigning 1-indexed page numbers and checking text density.
2. Chunking: DocumentChunker applies a 250-word sliding window with a 40-word overlap, constrained strictly within individual pages.
3. Embedding: LocalSentenceTransformerProvider computes L2-normalized 384-d vectors with HF_HUB_OFFLINE=1.
4. Indexing: FAISS IndexFlatIP indexes vectors with parallel persistence in SQLite and chunks.json.
5. Retrieval: LocalRetriever computes inner product similarity and filters by RELEVANCE_THRESHOLD (0.25).
6. Attribution: CitationBuilder formats source cards containing filename, page number, chunk ID, and exact snippet.
7. Refusal: If no chunks meet the threshold, the system immediately returns:
   "I couldn't find that information in the indexed documents."

---

5. Privacy & Local-Processing Model

Component	Execution Mode	Cloud AI Used	Telemetry
Document Storage	Local filesystem (backend/data/documents/)	None	None
Metadata Store	Local SQLite (backend/data/metadata/privatedoc.db)	None	None
Text Parsing	Local PyMuPDF (fitz)	None	None
Embeddings	Local CPU (backend/models/all-MiniLM-L6-v2/)	None	None
Vector Search	Local FAISS in-memory index	None	None
Generation	Local GGUF (when configured) / Unconfigured fallback	None	None
Network Traffic	Bound to 127.0.0.1 / Localhost only	None	None

---

6. Technology Stack

• Frontend: React 18, TypeScript, Vite, Tailwind CSS, Lucide React icons.
• Backend API: Python 3.12, FastAPI, Pydantic v2, SQLite 3.
• Document Extraction: PyMuPDF (fitz).
• Vector Search: FAISS (faiss-cpu, IndexFlatIP).
• Embeddings: SentenceTransformers (all-MiniLM-L6-v2).
• LLM Runtime Interface: llama-cpp-python / GGUF model interface.
• OCR Interface: Abstract OCRProvider (Local Tesseract / Qualcomm boundary).

---

7. Installation

Prerequisites

• Windows 10/11 (or Linux/macOS)
• Python 3.10+ (tested on Python 3.12)
• Node.js v18+ & npm

Clone / Workspace Root

The project root is:

C:\Users\ADMIN\engg\projects\qualc

---

8. Backend Setup

cd C:\Users\ADMIN\engg\projects\qualc\backend
pip install -r requirements.txt

Verify or copy the environment configuration:

cp .env.example .env

---

9. Frontend Setup

cd C:\Users\ADMIN\engg\projects\qualc\frontend
npm install

---

10. Embedding Model Setup

The embedding model is stored locally on disk at:

backend/models/all-MiniLM-L6-v2/

The application loads weights locally from this path with HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 set in the environment.

---

11. GGUF Model Setup

Note

GGUF Weights are NOT bundled with this repository to keep the repository size lightweight.

• When no GGUF file is present, the system truthfully reports:
  • System Status: LLM: NOT CONFIGURED
  • Privacy Status: Generation: LOCAL / NOT CONFIGURED
  • RAG Answer: "Local LLM is not configured. Supporting document evidence has been retrieved and cited below directly from the indexed document passages."
  • Full retrieval, source citations, document inspection, and unknown-question refusal remain 100% operational.
• To enable local generative synthesis:
  1. Download any compatible GGUF model (e.g. Qwen2.5-1.5B-Instruct-Q4_K_M.gguf or Phi-3-mini-4k-instruct-q4.gguf).
  2. Place the file at backend/models/model.gguf.
  3. Restart the backend server.

---

12. Running Tests

Run the automated pytest test suite from the repository root:

cd C:\Users\ADMIN\engg\projects\qualc
python -m pytest -q

Expected Real Result: 28 passed, 1 warning (includes comprehensive tests for documents, chunking, FAISS vector store, local embeddings, retrieval, citations, RAG queries, and on-device OCR engine).

---

13. Running the Demo

Step 1: Start Backend (Terminal 1)

cd C:\Users\ADMIN\engg\projects\qualc\backend
python -m uvicorn app.main:app --host 127.0.0.1 --port 8000

Interactive OpenAPI documentation is available at http://127.0.0.1:8000/docs.

Step 2: Start Frontend (Terminal 2)

cd C:\Users\ADMIN\engg\projects\qualc\frontend
npm run dev

Open http://127.0.0.1:5173 in your browser.

Step 3: Run 4-Document Verification Script (Terminal 3)

cd C:\Users\ADMIN\engg\projects\qualc
python scripts/verify_four_docs.py

---

14. Offline Verification Status

• Offline Execution: NOT INDEPENDENTLY VERIFIED
• Explanation: Local processing architecture is verified. Independent network-isolation/air-gap testing was not performed in this development environment.
• Local Processing Guarantees:
  • All document processing, chunking, vector indexing, and retrieval execute locally.
  • Cloud AI is not used (no OpenAI, Gemini, Anthropic, or external inference APIs).
  • Application network communication is localhost-only (127.0.0.1).
  • Zero external AI or cloud dependencies.

---

15. Snapdragon / Qualcomm Integration Boundary

The provider abstractions allow Qualcomm-specific inference implementations to be integrated later. Snapdragon hardware execution has not been validated in this development environment.

The codebase provides clean provider abstractions ready for future Qualcomm Snapdragon target integration:

• QualcommEmbeddingProvider (backend/app/services/embeddings.py): Interface for future Qualcomm AI Hub ONNX / QNN embedding integration.
• QualcommLLMProvider (backend/app/services/llm.py): Interface for future Snapdragon Genie / GenieX on-device GenAI runtime integration.
• QualcommOCRProvider (backend/app/services/ocr.py): Interface for future Snapdragon on-device OCR integration.

When integrating Qualcomm-specific runtimes in the future, only the provider implementations need to be supplied; all document ingestion, chunking, FAISS vector indexing, citation mapping, and React UI remain identical.

---

16. Known Limitations

1. Local LLM Weights: Generative text synthesis requires placing a compatible .gguf file at backend/models/model.gguf. Without it, the system delivers retrieved source passages and exact citations without generative paraphrasing.
2. OCR Availability: Local OCR uses on-device Tesseract for scanned/image-based PDFs. It dynamically discovers the binary via:
   • TESSERACT_CMD environment variable
   • settings.TESSERACT_CMD in .env (e.g. TESSERACT_CMD=C:\Program Files\Tesseract-OCR\tesseract.exe)
   • System PATH lookup (tesseract.exe)
   • Standard Windows directories (C:\Program Files\Tesseract-OCR\tesseract.exe, %LOCALAPPDATA%\Programs\Tesseract-OCR\tesseract.exe) When absent from the host, the system truthfully reports OCR: NOT CONFIGURED and safely marks scanned PDFs as Needs OCR without failing. Text-based PDFs process with full fidelity via PyMuPDF.
3. Hardware Acceleration: Currently executes using PyTorch CPU and FAISS CPU on Windows x86_64.

---

17. Future Work

• Package quantized ONNX models for Qualcomm QNN Execution Provider.
• Integrate Qualcomm Snapdragon Genie SDK for on-device LLM inference.
• Add local vision model integration for scanned document analysis.
• Provide zero-install single-executable desktop packaging using Tauri or Electron.