MSH.org RAG Stack (Qdrant + Open WebUI + Entra SSO + Admin Document Manager)

This repository deploys a secure internal RAG system for Management Sciences for Health (MSH.org) on a single Ubuntu 20.04+ host without Docker. The stack includes:

• Qdrant vector database with metadata filters (department, doc type, confidentiality, etc.)
• CPU embeddings via BAAI/bge-small-en-v1.5 (fast + high quality)
• Bulk + incremental ingestion for large PDF collections
• LLM generation with citations (PDF path + page)
• Open WebUI as the staff-facing UI
• Azure Entra ID (Azure AD) OIDC login for staff
• Manual admin login for the Document Manager portal

---

Architecture

Open WebUI (SSO) → OpenAI-compatible endpoint → RAG API (rag_api)

RAG API flow:

1. embeds query (CPU)
2. retrieves relevant chunks from Qdrant (with department filters)
3. calls local LLM server (OpenAI-compatible llama-cpp-python)
4. returns answer + citations

---

Hardware + OS requirements (Ubuntu 20.04 LTS or newer)

Recommended minimum for the target deployment:

• Ubuntu 20.04 LTS or Ubuntu 22.04 LTS
• 32 GB RAM
• 10 GB NVIDIA GPU (e.g., T4 / A10) for fast LLM inference
• 500 GB SSD for PDFs, indexes, and logs

Ports that must be open on the host:

• 3000 (Open WebUI)
• 8010 (RAG API)
• 6333 (Qdrant)
• 8001 (llama-cpp-python server)

---

Installation (Docker-free)

1) Install base system packages

sudo apt update && sudo apt -y upgrade
sudo apt -y install \
  build-essential \
  cmake \
  curl \
  git \
  pkg-config \
  python3-dev \
  python3-venv \
  rsync \
  ufw \
  unzip

The helper script runs the same setup, validates dependency versions, and installs Python API dependencies:

./scripts/ubuntu_setup.sh

It enforces these minimum versions during installation:

• Ubuntu >= 20.04
• Python >= 3.8
• git >= 2.25
• cmake >= 3.16
• pip >= 20.0

After pip install -r api/requirements.txt, it also runs pip check to catch dependency conflicts before deployment.

---

2) Install NVIDIA drivers + CUDA (10GB GPU)

sudo apt -y install nvidia-driver-535
sudo apt -y install nvidia-cuda-toolkit
sudo reboot

After reboot:

nvidia-smi

You should see the GPU listed with ~10GB VRAM.

---

3) Clone the repo to /opt/msh-rag

sudo mkdir -p /opt/msh-rag
sudo chown "$USER":"$USER" /opt/msh-rag

git clone <YOUR_GITHUB_REPO_URL> /opt/msh-rag
cd /opt/msh-rag

If you want a dedicated service account, create it now and replace mshrag in the systemd units later:

sudo useradd --system --create-home --shell /usr/sbin/nologin mshrag
sudo chown -R mshrag:mshrag /opt/msh-rag

---

4) Configure environment variables

cp .env.example .env
nano .env

You must set:

• WEBUI_URL
• WEBUI_SECRET_KEY
• RAG_API_KEY, API_KEY, and OPENAI_API_KEY (keep them identical)
• ADMIN_USERNAME / ADMIN_PASSWORD
• Entra ID settings (if using SSO)
• WEBUI_DATA_DIR (keep Open WebUI data under /opt/msh-rag/data/open-webui)

Example for localhost without SSO:

WEBUI_URL="http://localhost:3000"
OPENID_REDIRECT_URI="http://localhost:3000/oauth/oidc/callback"
ENABLE_OAUTH_SIGNUP="false"
ENABLE_SIGNUP="true"

Make sure Open WebUI points to the local RAG API:

OPENAI_API_BASE_URL="http://localhost:8010/v1"
OPENAI_API_KEY="${RAG_API_KEY}"

---

5) Download a GGUF model (7B Q4 recommended)

./scripts/download_model.sh

Ensure the model ends up at:

/opt/msh-rag/models/model.gguf

---

6) Install Qdrant (binary install)

mkdir -p /opt/msh-rag/bin
curl -L https://github.com/qdrant/qdrant/releases/download/v1.11.0/qdrant-x86_64-unknown-linux-gnu.tar.gz \
  | tar -xz -C /opt/msh-rag/bin

The repo includes a config file at config/qdrant.yaml that points Qdrant storage to /opt/msh-rag/data/qdrant. Copy or edit it if your paths differ.

---

7) Install llama-cpp-python server with CUDA

python3 -m venv /opt/msh-rag/venvs/llm
source /opt/msh-rag/venvs/llm/bin/activate

export CMAKE_ARGS="-DGGML_CUDA=on"
export FORCE_CMAKE=1
pip install --upgrade pip
pip install "llama-cpp-python[server]==0.2.90"

Recommended 10GB GPU + 32GB RAM settings (set in .env):

LLM_N_CTX="8192"
LLM_N_THREADS="8"
LLM_N_BATCH="1024"
LLM_N_GPU_LAYERS="35"
LLM_N_PARALLEL="1"

---

8) Install Open WebUI (local install)

python3 -m venv /opt/msh-rag/venvs/open-webui
source /opt/msh-rag/venvs/open-webui/bin/activate
pip install --upgrade pip
pip install open-webui

---

9) Install RAG API dependencies

The setup script already creates .venv and installs api/requirements.txt. If you skipped it:

python3 -m venv /opt/msh-rag/.venv
/opt/msh-rag/.venv/bin/pip install -r /opt/msh-rag/api/requirements.txt
/opt/msh-rag/.venv/bin/pip check

---

10) Deployment verification checklist (Ubuntu 20.04)

Run the following checks after install and after each upgrade:

# OS + runtime versions
source /etc/os-release && echo "$PRETTY_NAME"
python3 --version
git --version
cmake --version | head -n 1
/opt/msh-rag/.venv/bin/pip --version

# Python dependency integrity
/opt/msh-rag/.venv/bin/pip check

# App import smoke test
cd /opt/msh-rag
/opt/msh-rag/.venv/bin/python -m compileall api/app

# Service health checks
curl http://localhost:8010/health
curl http://localhost:6333/collections

Expected result: no pip check conflicts, successful module compilation, and healthy HTTP responses from the API and Qdrant endpoints.

---

Systemd services (recommended)

To simplify deployment, use the installer script instead of manually editing service files:

./scripts/install_services.sh

Optional overrides (for non-default user/path):

SERVICE_USER=mshrag INSTALL_DIR=/opt/msh-rag ENV_FILE=/opt/msh-rag/.env ./scripts/install_services.sh

The script generates and installs all four units, reloads systemd, and starts the stack.

If you still prefer manual service management, templates remain in scripts/systemd/.

---

First-run checks

curl http://localhost:8010/health
curl http://localhost:6333/collections

Open:

• Open WebUI (staff): http://<vm>:3000
• Admin Document Manager: http://<vm>:8010/admin

---

Ingestion (Bulk + Incremental)

Bulk upload (fast)

rsync -avh --progress /local/pdfs/ user@vm:/opt/msh-rag/data/docs_incoming/

Use department folders:

/opt/msh-rag/data/docs_incoming/
  HR/
  Finance/
  SupplyChain/
  CountryPrograms/
  All/

Bulk ingest

source /opt/msh-rag/.venv/bin/activate
python -c "from api.app.ingest import ingest_pdfs; print(ingest_pdfs())"

---

Admin UI ingestion

Admins can upload new documents directly at:

• http://<vm>:8010/admin

Protected by Basic Auth using:

• ADMIN_USERNAME
• ADMIN_PASSWORD

---

Upgrade guide

1. Read the release notes and confirm whether new variables/migrations are required.
2. Back up data:
   • /opt/msh-rag/data/qdrant
   • /opt/msh-rag/data/docs_incoming
   • /opt/msh-rag/data/docs_processed
   • /opt/msh-rag/data/open-webui
3. Fetch and checkout the release tag:

git fetch --tags
git checkout <release-tag>

Or upgrade to latest main:

./scripts/upgrade_latest.sh

4. Update .env if required by the release notes.
5. Restart services:

sudo systemctl restart msh-qdrant
sudo systemctl restart msh-llama
sudo systemctl restart msh-rag-api
sudo systemctl restart msh-open-webui

6. Verify health checks:

curl http://localhost:8010/health
curl http://localhost:6333/collections

---

Troubleshooting

Open WebUI login issues

• Confirm WEBUI_URL matches the real public URL.
• Confirm redirect URI matches exactly: <WEBUI_URL>/oauth/oidc/callback.

LLM slow

• Choose a smaller GGUF model.
• Reduce LLM_N_CTX, LLM_N_BATCH, or LLM_N_GPU_LAYERS if VRAM is saturated.

Some PDFs have no text

• They may be scanned images.
• Add OCR later for those documents if needed.