Skip to content

Repository files navigation

swmmgi — reproducible watershed-scale SWMM green-infrastructure modelling

DOI License: MIT tests

Reviewers / reproducibility — start here (no ArcGIS and no external data needed):

git clone https://github.com/Sami4517/swmmgi.git && cd swmmgi
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e ".[gis,swmm,test]"                    # or: pip install -r requirements-repro.txt
python reproduce.py                                  # → "RESULT: ALL PASS"

reproduce.py runs the 23 core tests + the real-basin diagnosis (swmmgi diagnose on the shipped Leon measurements) + the diagnostic twin-validation (5/5) + the sensitivity transfer. Granular checks: python tests/run_tests.py (core) · pytest tests/ (all 29) · python tools/twin_robustness.py (detection floors, tolerance grid, noise controls) · python tools/twin_regime_sa.py (does the sensitivity ranking survive a change of hydraulic regime?) · python examples/minimal/run_example.py. Full details in docs/REPRODUCING.md.

swmmgi is an open-source, unit-tested Python tool that builds watershed-scale EPA SWMM 5.2 green-infrastructure (GI/LID) models from public and incomplete municipal data, runs a GP-emulator-accelerated global (Sobol') sensitivity analysis, and applies a transferable forcing/parameter calibration-diagnostic protocol. It is the companion code for:

Sami, F. Y. (2026). swmmgi: an open, reproducible tool and a forcing/parameter diagnostic protocol for watershed-scale SWMM green-infrastructure modelling, demonstrated in two contrasting basins. Prepared for Environmental Modelling & Software. (DOI pending.)

The reusable core runs without ArcGIS: a validated FOSS GIS path (pysheds / WhiteboxTools) reproduces the reference (arcpy) channel network to within ~1 % (see FOSS validation below).


What this tool contributes

  1. A reproducible, config-driven pipeline that builds watershed-scale SWMM-GI models from public + municipal inputs, with documented, tested handling of the data pathologies that arise at this scale — orphan-outfall reconnection, DEM-derived Strahler channel augmentation, and SWMM's easily-missed multi-LID stacking constraint (ERROR 188). These are framed as practical data-integration solutions, not discoveries.
  2. A forcing/parameter calibration-diagnostic protocol (swmmgi diagnose --input <measurements.json>): forcing-swap (synthetic → observed point-gauge → Stage-IV radar) + GP-Sobol' + a roughness sweep + a per-storm variability (conveyance-saturation) test, which localizes model error to forcing, losses, or conveyance. Implemented as the unit-tested swmmgi.diagnose module and validated on a controlled synthetic twin experiment — it correctly classifies all five known-truth cases including the no-error control (tools/twin_validation.py), with a robustness suite quantifying detection floors (rain ×0.9 / Ksat ÷3 / n ×1.5), 27/27 tolerance-grid stability, and observation-noise false-positive rates (tools/twin_robustness.py). Running it on the shipped real-basin measurements (validation/leon_diagnose_input.json) reproduces the paper's diagnosis in one command.
  3. A GP-emulator global sensitivity analysis (swmmgi sa): channel Manning's n dominates outlet peak discharge in both demonstration basins (Sobol' total-order index 0.83–0.93).
  4. A controlled two-basin build-and-diagnose demonstration plus third-basin generalization (San Antonio, TX): the same pipeline builds urban Leon Creek and peri-urban Lower Medina end-to-end (64 design-storm configurations, median continuity error < 0.3 %), and the FOSS build generalizes to a third, independent basin — Headwaters Salado Creek — with no code changes, delineating 98.5 % of its mapped area on the first run.

Companion papers. The research article for Environmental Modelling & Software covers the tool, the validated diagnostic protocol, and the sensitivity analysis (Objective 1 of the parent project). GI scenario performance and its cross-basin transferability are quantified in a separate companion paper (in preparation) — here the scenario suite serves as the tool's stress test.

Honest scope of the demonstration model

The demonstration model is uncalibrated for absolute prediction: under design-storm forcing it over-generates runoff volume (PBIAS +148–492 %) and the outlet is conveyance-controlled. All quantitative claims therefore rest on relative comparisons under a common parameter set. A continuous-record (NEXRAD) loss-and-conveyance recalibration is the documented priority next step. The tool's value is the reproducible, reusable software and the diagnostic protocol — not a production-calibrated forecast for these basins.


Install

# from the repository root
pip install -e .                 # core: numpy, pyyaml
pip install -e ".[gis]"          # + FOSS GIS path (rasterio, geopandas, shapely, pysheds, pyproj, whitebox)
pip install -e ".[swmm]"         # + run the SWMM 5.2 engine (pyswmm, swmm-toolkit)
pip install -e ".[gis,swmm,test]"  # everything + pytest

Python ≥ 3.9. The reusable core has no GIS or ArcGIS dependency; the [gis] extra provides the arcpy-free preprocessing path.

Configure (no hardcoded paths)

Project paths and parameters resolve from a swmmgi.yaml file or the SWMMGI_ROOT environment variable — there are no hardcoded user paths in the package (enforced by a unit test). See config.example.yaml and swmmgi.leon.yaml.

export SWMMGI_ROOT=/path/to/project        # or set params.root in swmmgi.yaml

Use (CLI)

swmmgi version
swmmgi info                                # resolved config + parameters
swmmgi build  --config swmmgi.leon.yaml    # FOSS DEM hydrology -> channel network + catchment
swmmgi run    --config swmmgi.yaml         # run the SWMM model (needs [swmm])
swmmgi sa     --config swmmgi.yaml         # GP-emulator Sobol' sensitivity analysis
swmmgi diagnose --config swmmgi.yaml       # forcing/parameter calibration-diagnostic protocol

CLI status: version, info, build, and diagnose --input <measurements.json> run end-to-end (diagnose executes the decision logic on a protocol-measurements file — try swmmgi diagnose --input validation/leon_diagnose_input.json). run and sa are documented workflow entry points — their reusable logic is importable now (swmmgi.metrics, swmmgi.diagnose) and demonstrated end-to-end in tools/twin_validation.py and tools/twin_sensitivity.py, whose regime arms in tools/twin_regime_sa.py show the ranking holding from a free-draining network through to 88 % flooding loss (reproduce.py runs all of them).

The reusable core is also importable:

from swmmgi import config, network, lidcap, metrics, diagnose
from swmmgi import gis_hydro, gis_foss     # arcpy-free GIS (needs [gis])

Tests

35 unit tests — 29 core + 6 GIS:

python tests/run_tests.py        # 23 core tests (no extra deps): config, lidcap, metrics, network, diagnose
pytest tests/                    # all 29, including the 6 GIS tests (requires the [gis] extra)

The core suite is verified passing under the standard ArcGIS Pro Python; the GIS suite is verified under a geopandas/rasterio/pysheds/whitebox environment.


FOSS validation (runs without ArcGIS)

tools/validate_foss_hydro.py checks the FOSS DEM-hydrology path against the published arcpy build on the Leon Creek DEM (5 km² stream threshold):

Metric arcpy (provenance) FOSS (pysheds/whitebox) Agreement
Total channel length 167.9 mi 169.0 mi +0.7 %
Drainage density 0.709 0.713 +0.6 %
Reach count 57 59 +3.5 %
Spatial overlap (≤ 2 cells) 89 %
Strahler orders 1–4 1–4 match
swmmgi build outlet catchment 237 mi² (basin) 236.2 mi² 99.7 %

The full-pipeline arcpy build is retained as the published provenance; the validated FOSS path makes the workflow runnable without a proprietary license. Detail: docs/FOSS_validation_report.md.


Repository layout

.
  swmmgi/                  the package (config-driven, no hardcoded paths)
    config.py              SWMMGI_ROOT / swmmgi.yaml resolution + DEFAULT_PARAMS
    network.py             orphan-outfall reconnection, drainage density, channel length
    lidcap.py              multi-LID stacking cap (ΣFromImp <= 100 %; avoids ERROR 188)
    metrics.py             KGE / NSE / PBIAS / peak-ratio goodness-of-fit
    diagnose.py            calibration-diagnostic decision logic (forcing / losses / conveyance)
    gis_foss.py            FOSS vector/raster adapter (geopandas/rasterio) — replaces arcpy bulk
    gis_hydro.py           FOSS DEM hydrology (fill, D8, accumulation, Strahler, delineation; foss_build)
    cli.py                 swmmgi version|info|build|run|sa|diagnose
  tests/                   35 unit tests (run_tests.py runs the 29 core; pytest runs all)
  tools/                   twin_validation.py, twin_robustness.py, twin_sensitivity.py,
                           twin_regime_sa.py, twin_masking.py, clip_salado_dem.py,
                           run_salado_build.py, validate_foss_hydro.py, audit_arcpy.py,
                           migrate_paths.py, fetch_fig1_inputs.py,
                           plot_Fig1_studyarea.py
  data/fig1/               study-area boundaries (USGS WBD HUC10, Census TIGERweb
                           counties) and the DEM-derived channel networks, so
                           tools/plot_Fig1_studyarea.py reproduces the paper's Figure 1
  validation/              shipped result artifacts: twin validation + robustness JSONs,
                           leon_diagnose_input.json (real-basin measurements),
                           salado_build_summary.json, leon_calibration_goodness_of_fit.csv
  docs/                    quickstart, diagnostic_protocol, foss_port_roadmap, FOSS_validation_report
  examples/minimal/        runnable minimal example
  config.example.yaml, swmmgi.leon.yaml
  pyproject.toml, LICENSE

The original study/provenance scripts (the Leon Scripts/00–31 and Lower Medina LM_00–LM_38 chains, and the GP-emulator/Sobol' scripts) are retained in the parent project tree as the exact provenance of the published results. Reserved follow-up analyses (CMIP6 climate resilience, SVI equity allocation) are archived under manuscript/sections_archived/ and are not part of this release's claims; they are kept for transparency and a properly-powered follow-up.

Data sources

All primary inputs are public-archive datasets:

Dataset Source URL
USGS streamflow (IV/DV) NWIS https://waterservices.usgs.gov
DEM (10 m) USGS 3DEP https://apps.nationalmap.gov
NLCD 2021 (impervious) MRLC https://www.mrlc.gov/data
SSURGO 2024 (soils → Green-Ampt) USDA NRCS https://websoilsurvey.sc.egov.usda.gov
NOAA Atlas 14 Vol 11 (design storms) NWS HDSC https://hdsc.nws.noaa.gov/pfds/

The City of San Antonio storm-water shapefile is available from CoSA Public Works via a one-time data-sharing request (non-proprietary); it is not redistributed here. (CMIP6 and CDC SVI inputs pertain only to the archived follow-up analyses.)

Citation

@software{swmmgi_2026,
  author    = {Sami, Fahmida Yeasmin},
  title     = {swmmgi: reproducible watershed-scale SWMM green-infrastructure modelling},
  year      = {2026},
  publisher = {Zenodo},
  version   = {1.2.0},
  doi       = {10.5281/zenodo.21923303},
  url       = {https://doi.org/10.5281/zenodo.21923303}
}

@article{sami2026_swmmgi,
  author  = {Sami, Fahmida Yeasmin},
  title   = {swmmgi: an open, reproducible tool and a forcing/parameter diagnostic protocol
             for watershed-scale SWMM green-infrastructure modelling, demonstrated in two
             contrasting basins},
  journal = {Environmental Modelling \& Software},
  year    = {2026},
  note    = {Prepared for submission.}
}

Contributing

Bug reports, feature requests, and pull requests are welcome — see CONTRIBUTING.md for how to report issues, set up a dev environment, and run the tests. A short software paper (paper.md) is prepared for submission to the Journal of Open Source Software.

License

Code: MIT (see LICENSE). Data: the CoSA storm-water shapefile follows LICENSE_DATA.txt; all other inputs are public-domain or under their originating-agency licenses.

Contact

Fahmida Yeasmin Sami — Department of Political Science and Geography, University of Texas at San Antonio. Email: fahmidayeasmin.sami@utsa.edu · ORCID: 0000-0002-4100-3086

About

Open-source, unit-tested toolkit for building watershed-scale EPA SWMM green-infrastructure models from public + incomplete municipal data, with a validated calibration-diagnostic protocol and GP-emulator Sobol

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages