DocuSense

End-to-end RAG and NLP pipeline for natural-language Q&A over a private document corpus, achieving sub-second retrieval latency on a 10K+ chunk index. Implements chunking, OpenAI embedding generation, and vector indexing in Pinecone, with LangChain orchestration for retrieval, context assembly, and grounded LLM generation. Deployed as a FastAPI service containerised with Docker — production-style packaging with health checks, structured JSON logging, and secrets management.

---

Architecture

flowchart LR
    subgraph Ingest[Ingestion pipeline - python ingest.py]
        A[PDF / TXT files] --> B[Recursive chunker<br/>512 chars / 64 overlap]
        B --> C[OpenAI<br/>text-embedding-3-small]
        C --> D[(Pinecone<br/>serverless index)]
    end

    subgraph Serve[FastAPI service - app.main]
        Q[POST /query] --> R[LangChain LCEL chain]
        R --> E[OpenAI<br/>embed query]
        E --> D
        D --> F[Top-k retrieval<br/>+ score threshold]
        F --> G{Grounded?}
        G -- yes --> H[OpenAI gpt-4o-mini<br/>grounded generation]
        G -- no --> I[Fixed refusal]
        H --> J[Answer + citations]
        I --> J
    end

    HC[GET /health] --> D

Loading

Tech stack

Layer	Choice
Orchestration	LangChain (LCEL)
LLM	OpenAI gpt-4o-mini
Embeddings	OpenAI text-embedding-3-small (1536-dim)
Vector store	Pinecone serverless (AWS us-east-1)
API	FastAPI + Uvicorn (async)
Container	Docker (multi-stage, non-root) + Docker Compose
Config / secrets	pydantic-settings + python-dotenv
Logging	python-json-logger (structured JSON to stdout)

Project layout

docusense/
├── app/
│   ├── __init__.py
│   ├── main.py          # FastAPI app — /query, /health, /ingest
│   ├── rag.py           # LangChain LCEL retrieval chain
│   ├── ingest.py        # Chunking + embedding + Pinecone upsert
│   └── config.py        # pydantic-settings + JSON logging
├── scripts/
│   └── benchmark.py     # Latency harness backing the sub-second claim
├── docs/
│   └── sample-corpus.md # Bundled sample so a fresh clone is runnable
├── ingest.py            # CLI shim -> app/ingest.py
├── Dockerfile           # Multi-stage, non-root, with HEALTHCHECK
├── docker-compose.yml   # Local one-command run
├── .dockerignore
├── .env.example
├── .gitignore
├── requirements.txt
└── README.md

Quickstart

1. Prerequisites

• Docker (24+) and Docker Compose v2
• OpenAI API key — https://platform.openai.com/api-keys
• Pinecone API key — https://app.pinecone.io/

2. Configure

git clone <your-fork-url> docusense
cd docusense
cp .env.example .env
# edit .env and paste your OPENAI_API_KEY and PINECONE_API_KEY

3. Add documents

A small sample document ships in docs/sample-corpus.md, so you can run the full pipeline immediately without supplying anything. Drop your own PDF / .txt / .md files into ./docs/ to replace or extend it — everything in docs/ other than the sample is git-ignored, so your corpus stays local.

4. Build and run

docker compose up --build -d

The Pinecone serverless index is created automatically on first boot.

5. Ingest the corpus

docker compose run --rm api python ingest.py --source /app/docs

Ingestion is idempotent — re-running it over the same corpus replaces each file's vectors rather than appending a second copy, so you can safely re-ingest after editing a document or changing CHUNK_SIZE.

6. Ask a question

curl -s http://localhost:8000/query \
  -H 'Content-Type: application/json' \
  -d '{"question": "What does the document say about retrieval latency?", "top_k": 5}' \
  | jq

Example response:

{
  "answer": "The document specifies a sub-second retrieval latency target on a 10K+ chunk index.",
  "sources": [
    {
      "source": "design-notes.pdf",
      "chunk_index": 12,
      "page": 3,
      "score": 0.8421,
      "snippet": "Retrieval latency must remain under 1s at the p95 even when the index..."
    }
  ],
  "grounded": true,
  "latency_ms": 412
}

If nothing relevant is found:

{
  "answer": "I don't have enough information in the provided documents to answer that.",
  "sources": [],
  "grounded": false,
  "latency_ms": 138
}

API reference

GET /health

Liveness + Pinecone connectivity probe. Returns 503 if the index is unreachable.

{
  "status": "ok",
  "pinecone": {
    "index": "docusense",
    "namespace": "default",
    "total_vector_count": 1247,
    "dimension": 1536
  }
}

POST /query

Body: {"question": "...", "top_k": 5} (top_k optional, 1–20).

Returns {"answer", "sources", "grounded", "latency_ms"} as shown above.

Errors: 422 for malformed input, 500 for upstream failures (with a redacted message).

POST /ingest

Multipart upload of a single .pdf, .txt, or .md file. Indexes it into Pinecone on the fly.

curl -s -F file=@./docs/whitepaper.pdf http://localhost:8000/ingest | jq

{ "files": 1, "chunks": 87 }

Environment variables

Variable	Required	Default	Purpose
OPENAI_API_KEY	yes	—	OpenAI auth.
OPENAI_EMBEDDING_MODEL	no	text-embedding-3-small	Embedding model — must match PINECONE_DIMENSION.
OPENAI_CHAT_MODEL	no	gpt-4o-mini	Generation model.
OPENAI_CHAT_TEMPERATURE	no	0.0	Sampling temperature — 0 for grounded answers.
PINECONE_API_KEY	yes	—	Pinecone auth.
PINECONE_INDEX_NAME	no	docusense	Created automatically if missing.
PINECONE_CLOUD	no	aws	Serverless cloud.
PINECONE_REGION	no	us-east-1	Serverless region.
PINECONE_DIMENSION	no	1536	Must equal embedding model dimension.
PINECONE_METRIC	no	cosine	Similarity metric.
PINECONE_NAMESPACE	no	default	Logical partition inside the index.
CHUNK_SIZE	no	512	RecursiveCharacterTextSplitter chunk size.
CHUNK_OVERLAP	no	64	Chunk overlap.
RETRIEVAL_TOP_K	no	5	Default chunks returned per query.
RETRIEVAL_SCORE_THRESHOLD	no	0.2	Min cosine score for a chunk to be considered relevant.
APP_HOST / APP_PORT	no	0.0.0.0 / 8000	Uvicorn bind address.
LOG_LEVEL	no	INFO	Root log level.

Benchmark

Validate the sub-second latency claim with a real measurement:

# from the host, against the running container
python scripts/benchmark.py --n 100 --concurrency 1

Sample output:

============================================================
  requests:      100  (ok=100, errors=0)
  concurrency:   1
  wall time:     34.20 s
  throughput:    2.92 req/s
------------------------------------------------------------
  min:      118.4 ms
  mean:     312.7 ms
  p50:      289.1 ms
  p95:      612.4 ms
  p99:      781.6 ms
  max:      934.2 ms
============================================================

Pass your own questions with --questions path/to/file.txt (one per line).

Design decisions

• LCEL over RetrievalQA. The legacy from langchain.chains import RetrievalQA interface is deprecated in LangChain 0.3+. LCEL (prompt | llm | parser) is the current public API, supports ainvoke natively, and composes cleanly when you later want to add reranking or query rewriting.
• Grounding enforced at two layers. Retrieved chunks are filtered by RETRIEVAL_SCORE_THRESHOLD before the LLM ever sees them — if nothing survives, the chain returns a fixed refusal string and skips the generation call. The prompt also instructs the LLM to refuse if context is insufficient. Belt and braces.
• Health check verifies the dependency. /health calls Pinecone.describe_index_stats() rather than returning a hardcoded 200. Orchestrators get a truthful signal.
• Idempotent ingestion via ID prefixes. Vectors are keyed <filename>#<chunk_index>, and each file's existing vectors are deleted by ID prefix before re-upserting. Pinecone serverless has no delete-by-metadata-filter, so prefix paging is the supported route. Without this, re-ingesting silently doubles the index and duplicate chunks crowd each other out of top-k.
• Blocking SDK calls are off-loaded. The Pinecone and embedding SDKs are synchronous, so /health and /ingest wrap them in asyncio.to_thread. Otherwise a single large upload would freeze every concurrent /query for its full duration — and any concurrent benchmark would silently measure serialized requests.
• Multi-stage Dockerfile, non-root user. Build tooling stays in the builder stage; the runtime image ships only the venv and the app code, owned by an unprivileged app user.
• Structured JSON logs to stdout. Drop-in compatible with CloudWatch / Datadog / Loki — no per-service parsing rules required.

Local development (without Docker)

Requires Python 3.11–3.13.

python -m venv .venv
.venv\Scripts\activate                       # Windows PowerShell
# source .venv/bin/activate                  # macOS/Linux
pip install -r requirements.txt
cp .env.example .env                         # then edit
python ingest.py --source ./docs             # or: python -m app.ingest --source ./docs
uvicorn app.main:app --reload