🤖 Enterprise Agentic RAG with Human Feedback Loop

An autonomous, self-correcting Retrieval-Augmented Generation (RAG) system. This project moves beyond standard linear RAG by introducing an agentic state machine capable of grading its own research, accompanied by a continuous data collection pipeline for Direct Preference Optimization (DPO).

🌟 Key Features & MLOps Focus

• Agentic Cognitive Architecture: Utilizes LangGraph to create a cyclic state machine. The agent autonomously plans search queries, retrieves live web data, and strictly grades the relevance of the context before generating an answer.
• Self-Correction Routing: Implements conditional edges with Pydantic-structured LLM outputs. If retrieved documents are deemed irrelevant by the Grader Node, the system autonomously loops back to refine its research without human intervention.
• Human-in-the-Loop (HITL) Data Flywheel: Features a Streamlit UI equipped with a session-aware feedback loop. User ratings (👍 / 👎) alongside the prompt and response are automatically logged to a PostgreSQL database, generating a "Gold Standard" dataset for future model fine-tuning.
• Production-Ready State Management: Decouples the frontend UI telemetry from the backend agent reasoning, ensuring the system remains scalable, traceable, and easily debuggable.

🛠️ Tech Stack

• Orchestration & Logic: Python, LangChain, LangGraph
• Large Language Model: OpenAI API (GPT-4o-mini)
• Live Web Retrieval: Tavily API
• Database & Telemetry: Supabase (PostgreSQL)
• Frontend UI: Streamlit
• Data Validation: Pydantic

🧠 The Agentic Workflow

1. Planner: Ingests the user's prompt and formulates a targeted search strategy.
2. Retriever: Executes searches via Tavily, pulling clean, LLM-ready context from the live web.
3. Grader (The Critical Node): Evaluates the retrieved context against the original prompt using strict binary classification. Routes back to the Planner if the context is insufficient.
4. Synthesizer: Drafts a final, hallucination-free response grounded strictly in the validated context.

🚀 Local Setup & Installation

1. Clone the repository and navigate to the directory:

git clone [https://github.com/dasashreeya/agentic_rag.git](https://github.com/dasashreeya/agentic_rag.git)
cd agentic_rag

2. Install dependencies:

pip install -r requirements.txt

3. Set up your Environment Variables: Create a .env file in the root directory and add your API keys:

Code snippet

OPENAI_API_KEY="sk-..."
TAVILY_API_KEY="tvly-..."
SUPABASE_URL="[https://your-project.supabase.co](https://your-project.supabase.co)"
SUPABASE_KEY="eyJhb..."

4. Initialize the Database: Run the following SQL command in your Supabase SQL Editor to create the telemetry table:

SQL

CREATE TABLE feedback_logs (
    id UUID DEFAULT gen_random_uuid() PRIMARY KEY,
    prompt TEXT,
    response TEXT,
    rating BOOLEAN,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

5. Run the Application:

streamlit run app.py

🚧 Future Improvements

Recursion Limits (Circuit Breaker): Implement a Time-to-Live (TTL) counter in the Agent State to gracefully fallback if the Grader rejects search results multiple times in a row, preventing infinite loops on unanswerable questions.

Query Decomposition: Upgrade the Planner node to decompose complex, multi-part questions into parallel search tasks.

Multi-Agent Consensus: Introduce a secondary fact-checking node to cross-reference multiple domains for highly sensitive queries.