Change Analysis · Impact Graph · Evidence Strength · Test Planning · Graph RAG
ImpactGraph performs PR change-impact analysis using static code analysis, dependency graphs, test recommendations, and grounded Graph RAG over a deterministic software impact graph.
Traditional diff review answers what changed directly. ImpactGraph attempts to trace:
- downstream impact: What else could be affected?
- structural relationships: Why is this component considered impacted?
- supporting evidence: How strong is the structural evidence?
- affected tests: What should be tested?
- report-grounded queries: Can I ask follow-up questions grounded in this analysis?
Analyze a repository archive alongside a code diff. Analysis runs asynchronously through the background job pipeline.
|
|
Inspect changed files, changed symbols, propagated impact, evidence, risk information, and generated test recommendations.
View Report Details
Explore structural relationships across the analyzed codebase and inspect the blast radius of a change. Fullscreen mode provides a larger workspace for navigating complex graphs.
|
|
Review report-derived test recommendations or ask follow-up questions through the Graph RAG assistant with citations back to supporting report evidence.
|
|
- Diff-Aware Static Analysis: Maps changed files and changed symbols from repository and diff inputs.
- Python Symbol Indexing: Builds structural information for modules, classes, methods, and functions across the analyzed codebase.
- Dependency & Impact Graph Construction: Models structural relationships including imports, calls, containment, inheritance, and test relationships where statically resolvable.
- Impact Propagation: Traces downstream impact through deterministic graph relationships.
- Evidence Strength: Associates propagated impact with deterministic structural evidence and provenance rather than unsupported probability-like scores.
- Test Discovery & Recommendations: Connects impacted production code with existing tests and identifies missing coverage where deterministically supported.
- Typed Retrieval Documents: Converts report information into deliberate retrieval units rather than arbitrary prose chunks.
- Hybrid Lexical + Dense Retrieval: Combines lexical and embedding-based retrieval using Reciprocal Rank Fusion.
- Graph RAG: Uses the deterministic code-impact graph to expand and contextualize retrieved report evidence.
- Artifact Fallback: Surfaces authoritative report artifacts when queries require deterministic report-level context.
- Grounded Citations: The Report Assistant cites supporting report and graph evidence for generated answers.
- Async Redis/RQ Pipeline: Runs analysis jobs outside the request lifecycle through Redis-backed worker coordination.
- Evaluation Harness: Measures retrieval quality, RAG answer quality, citations, and deterministic analyzer correctness.
graph TD
A[Repository ZIP + Diff] --> B[Ingestion & Parsing]
B --> C[Symbol Index & Structural Analysis]
C --> D[Dependency / Impact Graph]
D --> E[Impact Propagation & Evidence Strength]
E --> F[Test Discovery & Recommendation]
F --> G[Report Generation]
graph TD
A[Analysis Report] --> B[Typed Retrieval Documents]
B --> C[Lexical Retrieval]
B --> D[Dense Retrieval]
C --> E[Reciprocal Rank Fusion]
D --> E
E --> F[Retrieved Evidence]
F --> G[Graph Expansion & Context Assembly]
A --> H[Authoritative Report Artifacts]
H --> G
G --> I[Grounded Prompt Context]
I --> J[Report Assistant]
J --> K[Answer + Citations]
graph TD
A[Next.js Frontend] --> B[FastAPI Backend]
B --> C[(PostgreSQL + pgvector)]
B --> D[(Redis Queue)]
D --> E[RQ Worker]
E --> F[Analysis Pipeline]
F --> C
ImpactGraph combines hybrid retrieval with a deterministic software dependency and impact graph.
Unlike GraphRAG approaches that first construct a knowledge graph from LLM-extracted entities in unstructured documents, ImpactGraph builds its graph directly from source-code structure. That graph is then used for impact analysis, retrieval expansion, provenance, and answer grounding.
Analysis results are represented as structured retrieval documents with explicit semantic roles instead of being split into arbitrary prose chunks.
This allows retrieval to distinguish report summaries, changed files, changed symbols, impacted nodes, graph evidence, test recommendations, and other analysis artifacts.
ImpactGraph combines:
- lexical retrieval for exact names, paths, symbols, and report terminology
- dense embedding retrieval for semantic similarity
- Reciprocal Rank Fusion (RRF) to combine the two rankings
The production retrieval pipeline intentionally does not add dedicated reranking, multi-query retrieval, or parent-document retrieval unless evaluation demonstrates a measurable need.
Retrieved evidence can be augmented through the deterministic impact graph.
Graph relationships provide additional context around relevant files, symbols, tests, dependencies, and propagation paths that may not be represented by the initially retrieved document alone.
When a query requires authoritative report-level information, deterministic report artifacts can be surfaced alongside retrieved evidence.
This provides report-level context without forcing the language model to infer facts that already exist structurally in the analysis.
The Report Assistant is instructed to answer from report-derived context and attach citations to supporting evidence.
The application tracks report evidence independently from generated prose so answers can be inspected against the underlying analysis.
ImpactGraph keeps structural evidence separate from unsupported probability-like confidence scores.
Evidence Strength is derived deterministically from analysis provenance and graph relationships rather than presenting arbitrary model-generated confidence percentages.
ImpactGraph evaluates three separate layers of the system instead of reducing performance to a single accuracy score.
Measures whether the retrieval pipeline surfaces relevant report evidence.
| Metric | Value |
|---|---|
| Precision@1 | 84.21% |
| Recall@5 | 92.11% |
| Recall@10 | 92.11% |
| Hit Rate@3/5/10 | 94.74% |
| NDCG@5/10 | 86.13% |
| MRR | 0.886 |
Generated-answer quality is evaluated through the RAG evaluation harness using DeepEval, with the recorded judge-scored run using gemini-3.6-flash.
| Metric | Value |
|---|---|
| Faithfulness | 94.44% |
| Answer Relevancy | 88.89% |
| Answer Correctness | 83.33% |
| Citation Coverage | 95.00% |
| Invalid Citation Rate | 0.00% |
Faithfulness is reported directly as an evaluation metric and is not converted into an artificial "hallucination rate."
Deterministic analyzer output is evaluated separately against a source-derived correctness benchmark.
| Metric | Value |
|---|---|
| Changed Symbol Recall | 100.00% |
| Changed Symbol Precision | 78.57% |
| Changed Symbol F1 | 88.00% |
| Changed File Precision | 100.00% |
| Changed File Recall | 76.92%* |
| Changed File F1 | 86.96% |
| Graph Relationship Recall | 93.75% |
| Structural Graph Defects | 0 |
* Changed-file recall is affected by changed test files that are intentionally excluded from the report's core changed-files collection.
The repository maintains backend, frontend, static-analysis, and deterministic regression gates before changes are considered release-ready.
| Suite | Automated Tests |
|---|---|
| Backend — pytest | 2,615 |
| Frontend — Vitest | 680 |
| Total | 3,295 |
Additional verification includes:
- Ruff
- mypy
- ESLint
- strict TypeScript checks
- Next.js production build
- sample ground-truth validation
- GitHub Actions CI
| Layer | Technology |
|---|---|
| Frontend | Next.js, React, Tailwind CSS |
| Backend | FastAPI, Python 3.12 |
| Database | PostgreSQL, pgvector |
| Queue / Worker | Redis, RQ |
| LLM | Gemini |
| Embeddings | gemini-embedding-001 |
| Retrieval | Lexical + Dense Retrieval, RRF, Graph Expansion |
| Evaluation | DeepEval, deterministic evaluation harnesses |
| Testing | pytest, Vitest |
| Infrastructure | Docker Compose |
-
Copy the environment template:
cp .env.example .env
-
Start the application stack:
docker compose up -d
-
Open:
http://localhost:3000
To exercise the product without preparing your own repository and diff, open the Analyze flow and run the built-in sample analysis.
- Python-first static analysis: Source analysis currently targets Python codebases.
- Dynamic references: Highly dynamic or statically unresolved calls may remain
UNKNOWN. - Static graph model: ImpactGraph reasons over source-derived structural relationships rather than runtime tracing.
impact-graph/
|-- backend/
| |-- src/impactgraph/
| `-- tests/
|-- frontend/
| |-- app/
| `-- components/
|-- docs/
| `-- readme/
`-- scripts/