srcMove — Move Annotation for srcDiff XML

srcMove is a C++ tool that post-processes srcDiff XML output and annotates detected “move” operations by adding a stable move id (and an xpath location) onto matching diff:delete / diff:insert regions.

This repository is being developed as part of a master’s thesis project focused on improving the interpretability of fine-grained source-to-source diffs produced in the srcML/srcDiff ecosystem.

What it does

Given a srcdiff.xml file (the XML produced by srcDiff), srcMove:

1. streams the XML and collects all diff:insert and diff:delete regions
2. filters the regions to choose good “move units” (default: leaf-only diff regions, skip whitespace-only, skip regions already marked with move)
3. hashes each region by its inner text content
4. groups deletes/inserts by content (hash + exact text equality confirmation)
5. assigns a new move id to each group that has both deletes and inserts
6. writes a new XML file identical to the input, except:
   • adds move="<id>" to the START tag of matched diff:insert and diff:delete
   • adds xpath="<path>" to those tags (even if move already existed)

Output is a new XML file (default: diff_new.xml) you can feed into downstream tooling, renderers, or visualizers.

Quick start

Prerequisites

• CMake 3.20+
• A C++17 compiler (Clang or GCC recommended)
• libxml2 development package
• A workspace that contains:
  • srcReader (sibling repository)
  • a built/installed srcML (expected at srcML-install)

Recommended: use the workspace installer

SrcMLBuildTemplate

This project is designed to live inside a srcML workspace alongside its dependencies. While you can configure paths manually, the recommended setup is to use my installer repo, SrcMLBuildTemplate, which bootstraps a reproducible workspace.

Typical layout:

srcMLBuildTemplate/    # installer root (recommended)
  srcML/
  srcML-install/      # local srcML install prefix
  srcReader/          # dependency
  srcDiff/            # to generate input XML
  srcMove/            # this repo

Configure and build with CMake + Ninja:

From the repository root:

cmake -S . -B build \
  -G Ninja
ninja -C build

If your workspace layout differs, configure with:

cmake -S . -B build \
  -G Ninja \
  -DWORKSPACE_ROOT=/path/to/workspace \
  -DSRCREADER_ROOT=/path/to/srcReader \
  -DSRCML_INSTALL_PREFIX=/path/to/srcML-install

ninja -C build

Run

./build/srcMove path/to/srcdiff.xml

By default, srcMove writes annotated XML to diff_new.xml in the current working directory. Optionally specify an output filename:

./build/srcMove path/to/srcdiff.xml out.xml

CLI reference

./build/srcMove <srcdiff.xml> [out.xml] [--results results.json] [--profile] [-v]
./build/srcMove --help
./build/srcMove --version

Arguments:

• <srcdiff.xml>: input XML produced by srcDiff
• [out.xml]: output annotated XML file; defaults to diff_new.xml

Options:

• --results <file>: write a JSON summary of detected move groups, annotated regions, candidate counts, group kinds, and match kinds
• --profile: write coarse wall-clock timings to stderr as profile.<stage>_ms=<milliseconds> lines
• -v, --verbose: accepted by the parser, but currently no pipeline behavior is gated by it
• -h, --help: print command usage
• --version: print the current srcMove version string

Example: run against a fixture

This repo includes small, focused srcDiff fixtures under test/e2e_custom/cases/.

For example:

./build/srcMove test/e2e_custom/cases/1x1_basic/input.xml /tmp/srcmove-1x1.xml

To also capture the summary JSON:

./build/srcMove \
  test/e2e_custom/cases/1x1_basic/input.xml \
  /tmp/srcmove-1x1.xml \
  --results /tmp/srcmove-1x1.json

To profile a run without changing the XML or JSON output format:

./build/srcMove \
  test/e2e_custom/cases/1x1_basic/input.xml \
  /tmp/srcmove-1x1.xml \
  --results /tmp/srcmove-1x1.json \
  --profile

For repeatable performance tracking, prefer a release build and the profiling runner:

scripts/build_release.sh

python3 scripts/profile_srcmove.py \
  --prepare-bigclonebench

The script records only srcMove --profile timings from generated case inputs. BigCloneBench setup, database loading, and case generation are run before the measured profile pass and are not included in the CSV timing columns. For BigCloneBench, the profiler reads the active bcb_t*_manifest.json so stale case directories from older runs are ignored. By default, profile runs are written under profile-results/runs/ and copied to profile-results/latest.csv with matching .txt metadata files. The default profile run requests a BigCloneBench Type-1 --limit 1000 generation batch, then profiles however many deduped cases are actually generated, repeated 3 times. In this checkout that request usually yields 915 Type-1 cases. Use --clone-type, --bigclonebench-limit, --repeats, and --label to name or reshape a run.

To profile the large premade OpenCV srcDiff fixture at examples/opencv/opencv.1_2.v000001-to-v000002.e46e13a77579-to-5e38cf8042d1.position.diff.xml:

scripts/build_release.sh
python3 scripts/profile_srcmove.py --suite opencv

Each custom fixture directory contains input.xml, expected.xml, and expected.json. See test/README.md for the normal test runner entry points, generated source-pair tests, and the larger generated BigCloneBench Type-1/Type-2 suites.

How it works

The pipeline (see src/pipeline.cpp) is intentionally simple and streaming-friendly:

• collect_all_regions (src/diff_region.*)

  • single pass over srcml_reader
  • records every diff region with nesting metadata and inner text
  • detects pre-existing move attributes and tracks the max id

• filter_regions_for_registry (src/region_filter.*)

  • selects which regions become candidates for move detection

  • defaults:

    • leaf-only diff regions
    • skip whitespace-only regions
    • skip regions that already have a move attribute

• move_registry + group builder (src/move_registry.*)

  • buckets candidates by a fast 64-bit FNV-1a hash of raw inner text
  • refined grouping splits hash buckets by exact canonical text, selects non-overlapping exact groups, recovers eligible one-to-one Type-2 groups, and emits unmatched leftovers
  • produces “content groups” that can be classified (1-to-1, many-to-many, etc.)

• annotation_plan + writer (src/writer/annotation_plan.*, src/writer/annotation_writer.*)

  • assigns a new move id per group that has both deletes and inserts
  • rewrites the XML in a second pass
  • patches only the START tags of diff:insert / diff:delete
  • writes xpath values using a streaming path builder (src/xpath_builder.*)

This design keeps memory usage predictable and avoids generating the full D×I pair explosion unless explicitly requested.

Output format

srcMove edits only the START tags of diff:insert and diff:delete elements:

• move="<id>" is added only if the element was not already marked by srcDiff
• xpath="<path>" is written even if move already existed

The produced output remains valid srcDiff XML and should be usable anywhere the original was used.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<unit xmlns="http://www.srcML.org/srcML/src" xmlns:diff="http://www.srcML.org/srcDiff" revision="1.0.0" language="C++" filename="test/simple/original.cpp|test/simple/modified.cpp"><function><type><name>int</name></type> <name>main</name><parameter_list>()</parameter_list> <block>{<block_content>
<diff:delete><diff:ws>  </diff:ws><diff:delete move="1" xpath="/unit[1]/function[1]/block[1]/block_content[1]/diff:delete[1]/diff:delete[1]"><decl_stmt><decl><type><name>int</name></type><diff:ws> </diff:ws><name>first</name><diff:ws> </diff:ws><init>=<diff:ws> </diff:ws><expr><literal type="number">123</literal></expr></init></decl>;</decl_stmt></diff:delete><diff:ws>
</diff:ws></diff:delete>  <decl_stmt><decl><type><name>int</name></type> <name>second</name> <init>= <expr><literal type="number">456</literal></expr></init></decl>;</decl_stmt>
<diff:insert><diff:ws>  </diff:ws><diff:insert move="1" xpath="/unit[1]/function[1]/block[1]/block_content[1]/diff:insert[1]/diff:insert[1]"><decl_stmt><decl><type><name>int</name></type><diff:ws> </diff:ws><name>first</name><diff:ws> </diff:ws><init>=<diff:ws> </diff:ws><expr><literal type="number">123</literal></expr></init></decl>;</decl_stmt></diff:insert><diff:ws>
</diff:ws></diff:insert>  <return>return <expr><literal type="number">0</literal></expr>;</return>
</block_content>}</block></function>
</unit>

Included developer tools

This repository also builds a few small utilities under src/tools/:

• srcdiff_render
• srcdiff_highlight
• srcdiff_highlight_pos

These are intended for debugging and inspection during development (they link against the same srcReader/srcML stack).

Project structure

• src/

  • core pipeline and move detection logic

• src/tools/

  • small inspection utilities (debug-focused)

• doc/

  • diagrams, notes, terms, and papers reviewed during thesis work

• test/

  • compact fixtures (original/modified + sample diffs)

Limitations (current)

• Matching is currently content-based (raw inner text) and does not use AST-aware similarity scoring.
• Pairing is currently a fast baseline (greedy 1-to-1 consumption inside each content group).
• Many-to-many groups are identified but not fully disambiguated into specific move pairings.
• This tool expects srcDiff XML input and is not a general-purpose diff engine.

These are active areas for improvement as the thesis work progresses.

Roadmap

Planned directions (subject to change as research evolves):

• richer move classification (copy vs move vs repeated insertion)
• better disambiguation strategies for many-to-many groups
• scoring models that incorporate structural context (xpath, node type, surrounding tokens)
• a dedicated visualization companion tool (side-by-side view with insert/delete/move highlighting)

License

GPL-3.0-only. See LICENSE.

Acknowledgements

This project builds on the srcML/srcDiff tooling ecosystem and depends on srcReader and srcML for streaming parsing and writing of srcML-derived XML formats.