IceGraph is an interactive Apache Iceberg debugging and visualization platform that provides a hierarchical, graph-based view of Iceberg metadata. It maps the DNA of your production tables - helping engineers debug complex table states, trace metadata evolution, and understand Iceberg internals visually.
Look at Live Demo! https://yanivzalach.github.io/IceGraph/
Opinionated Design: IceGraph is built exclusively for Spark Connect backends.
Table Version: Currently IceGraph officially supports Table Version 2.
- Production-Safe & Read-Only — Built for production Iceberg tables without modifying data or metadata.
- Graph-Based Visualization — Explore metadata, snapshots, manifests, data files, and delete files through an interactive graph UI. For all your table branches.
- Snapshot & Metadata Lineage — Trace table evolution, commits, schema changes, and snapshot history over time.
- Partition & File Browser — Navigate partitions and files through a familiar hierarchical view.
- Debugging & Learning Tool — Designed for both production debugging and understanding Iceberg internals.
- Python Client & CLI — Script access to tables, snapshots, and the metadata graph via
icegraph-client. See the Python Client & CLI section of the docs for details.
Recommended: In production, use a user with read-only permissions for the Spark Connect server, for extra peace of mind.
Clone the repo, and in it, go to:
cd docker_demo
Run the docker compose:
docker compose up -d
Go to http://localhost:5050 and explore table default.events and table default.logging.
To rebuild only the IceGraph image after local code changes (without restarting Spark or other services):
docker compose up -d --build icegraphThe easiest way to run IceGraph is via DockerHub
docker run -e SPARK_REMOTE=sc://<spark-connect-ip>:15002 -p 5050:5050 yanivzalach/icegraph:latestClone the repo, update the Spark Connect version in backend/pyproject.toml, then build from the project root:
docker build -t icegraph .Then run with the same command:
docker run -e SPARK_REMOTE=sc://<spark-connect-ip>:15002 -p 5050:5050 icegraphGraph jobs are tracked in each pod's memory, so if you run more than one replica, your load balancer/ingress must route all requests from a given client to the same pod (session affinity / sticky sessions) for the duration of a job, otherwise the polling requests can land on a pod that never received it.
Before contributing, read ARCHITECTURE_PHILOSOPHY.md — the design pillars IceGraph is built on.
Work is tracked as tickets of three types: Frontend Refactor, Enrichments, and Bugs, on the IceGraph Roadmap board.
- npm
- UV (python)
- Spark Connect server (Quick setup using https://github.com/YanivZalach/Docker_Spark_Connect_Iceberg)
Sync the environments:
cd backend
uv synccd icegraph-client
uv synccd frontend
npm iWe will create an .env file in the root of the backend directory:
SPARK_REMOTE=sc://localhost:15002 # Our local testing spark, If you use docker, change it to your ip.The supported environment variables, their defaults, and their descriptions are defined in the Env class.
Open one terminal in the backend directory and run:
uv run python main.pyOpen a second terminal in the front end directory and run:
npm run devGo to http://localhost:3000 and explore your tables.
CI checks Python formatting and the frontend toolchain. Format each Python project from its own directory.
Backend:
cd backend
uv run ruff format .Python client:
cd icegraph-client
uv run ruff format .Run all three frontend checks from the frontend directory:
cd frontend
npm run format
npm run lint
npm run typecheckEach one catches a different class of problem, so all three are needed.
To work on the CLI/client itself:
cd icegraph-client
uv sync
uv run icegraph --base-url http://localhost:3000 tables