copilot-trace

Parse Visual Studio / GitHub Copilot chat session logs into an AgentTrace-compatible trace store, then inspect the result in a focused multi-page UI for parser review, evaluation triage, and session health.

Why this exists

Copilot exports are messy, deeply nested, and annoying to inspect by hand. copilot-trace turns them into a cleaner trace model you can:

• ingest into SQLite
• export as JSON
• query through a small local API
• review in a browser UI with trace filters, annotations, evaluation context, and dashboard summaries

What’s in the repo

• Python parser for Okonomi-style Copilot session logs
• CLI for ingesting one or more session folders
• SQLite + JSON exporters using an AgentTrace-like shape
• Local HTTP API for traces, sessions, evaluations, and annotation updates
• React + Vite UI with dedicated Parser, Evaluation, and Dashboard pages
• Tests + scripts + Make targets for repeatable local usage

Feature highlights

• recursive decoding of nested tool payloads and ValueContainer blobs
• persisted storage config in out/copilot-trace-config.json
• API-backed filtering, paging, timeline sort order, parent/sequence metadata, and trace annotation writes
• trace-level evaluation context surfaced directly in the UI, including score bands and status explanations
• timeline-first Parser page with the selected trace docked beside the event stream for faster review
• inline payload inspection that keeps the current timeline slice and active trace visible together
• hash-based page routing (#/parser, #/evaluation, #/dashboard) so direct links survive static hosting
• sample fallback data loaded as a separate asset instead of being bundled into the main JS chunk

Project structure

copilot-trace/
├── parser/         # parser, CLI, API, storage helpers, evaluation export
├── tests/          # Python parser/API/CLI tests
├── ui/             # React + Vite frontend
├── scripts/        # install/run/ingest helpers
├── docs/           # notes and usage docs
├── screenshots/    # current UI captures for README/social use
├── out/            # generated sqlite/json/config outputs (local only)
├── Makefile        # common automation entrypoints
└── pyproject.toml  # package metadata

Prerequisites

• Python 3.11+
• Node.js 20+

Install

Fast path

cd copilot-trace
make install

Manual path

cd copilot-trace
python3 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
cd ui && npm ci && cd ..

Quick start

1) Ingest Copilot logs

cd copilot-trace
./scripts/ingest.sh --input ../copilot-logs/Okonomi/copilot-chat/539fc419/sessions

Equivalent CLI call:

cd copilot-trace
. .venv/bin/activate
copilot-trace trace \
  ../copilot-logs/Okonomi/copilot-chat/539fc419/sessions \
  --db out/traces.db \
  --json out/traces.json \
  --rotate-db

Inspect the active storage config:

copilot-trace config

2) Run the API

cd copilot-trace
make run-api

Equivalent direct command:

. .venv/bin/activate
python3 parser/api.py --db out/traces.db --host 0.0.0.0 --port 8000

3) Run the UI

cd copilot-trace
make run-ui

Equivalent direct command:

cd ui
npm run dev -- --host 0.0.0.0 --port 5173

Open http://localhost:5173/#/parser. The Parser page keeps the paged timeline front and center, with the selected trace docked beside it so payload and evaluation context stay visible while you move through the session. The trace timeline header also exposes an Export session action for the currently selected session, writing an MLflow-oriented JSON bundle to a local folder you choose.

4) Import an exported bundle into MLflow

cd copilot-trace
. .venv/bin/activate
pip install mlflow
python scripts/import_bundle_to_mlflow.py \
  /path/to/export-root/copilot-session-export \
  --tracking-uri file:$(pwd)/out/mlruns \
  --experiment-name copilot-trace

That importer creates a local MLflow run, mirrors the exported tags/params/metrics from mlflow-run.json, and uploads the whole bundle directory as run artifacts under copilot_trace_bundle/ by default.

UI routes

The UI uses hash routes so you can deep-link without needing server-side rewrite rules:

• #/parser
• #/evaluation
• #/dashboard

Selected session and filter state are mirrored into the URL query string where practical.

Automation reference

Common commands

make install      # create venv + install Python and UI deps
make test         # run Python + UI tests
make build        # build Python package + UI bundle
make package      # build Python distribution artifacts
make run-api      # run the local API server
make run-ui       # start Vite dev server
make ingest INPUT=/path/to/sessions

Script equivalents

• ./scripts/install.sh
• ./scripts/ingest.sh --input /path/to/sessions
• ./scripts/run-api.sh --db out/traces.db --host 0.0.0.0 --port 8000
• ./scripts/run-ui.sh --host 0.0.0.0 --port 5173
• python scripts/import_bundle_to_mlflow.py /path/to/exported-bundle --tracking-uri file:$(pwd)/out/mlruns --experiment-name copilot-trace

Validation

Run tests

cd copilot-trace
make test

Build distributables

cd copilot-trace
make package
make build

Publishing notes

This repo is close to “shareable side-project” territory, but a couple of publishing decisions should still be made explicitly:

• choose a license
• decide whether to keep example trace exports in-repo or generate them during release prep
• decide whether GitHub Pages, static hosting, or local-only hosting is the intended UI delivery model
• keep only the current screenshot set that matches the shipped UI

Suggested release checklist

1. Run make test
2. Run make build
3. Confirm the UI works against both live API and sample fallback
4. Refresh screenshots only when the current UI meaningfully changes
5. Pick a license before making the repo public
6. Push to GitHub and add a short project description + topics

Notes on dependencies

• requirements.txt is intentionally small and geared toward local development.
• Python runtime dependencies live in pyproject.toml.
• requirements.txt installs the package in editable mode so the copilot-trace CLI is available immediately.
• MLflow is optional and only needed when you want to import exported bundles into a local/remote tracking store (pip install mlflow).
• UI dependency versions are locked in ui/package-lock.json.

Screenshots

Current captures live in screenshots/ for the active parser/evaluation/dashboard experience.