Add AtlasNet-Organs and AtlasNet-Tumors (Couinaud liver segments) - #180
Open
porth-bot wants to merge 9 commits into
Open
Add AtlasNet-Organs and AtlasNet-Tumors (Couinaud liver segments)#180porth-bot wants to merge 9 commits into
porth-bot wants to merge 9 commits into
Conversation
AtlasNet-Organs is the organ checkpoint from the AbdomenAtlas 3.0 release (RadGPT, ICCV 2025). It segments 34 abdominal classes, twelve of which the viewer did not carry: the eight Couinaud liver segments, plus esophagus, rectum, hepatic vessel, and the portal/splenic vein. The segments are the reason to add it. Nothing else on the site can localise a finding to a liver segment, which is the unit surgeons actually plan in. Notes for review: - Named AtlasNet-Organs, not AtlasNet. The existing "Atlas-Net" entry is an unrelated 25-class model that happens to share the name, so a distinct id keeps the dispatch and the dropdown unambiguous. - The released checkpoint is a bare nnU-Net model folder rather than an nnUNet_results tree, so ATLASNET_ORGANS_MODEL_PATH points straight at the trainer directory. The runner raises a clear error if it is missing rather than failing deep inside nnU-Net. - The model has no whole-liver and no whole-pancreas class; the liver exists only as its segments. combined_labels is a flat volume, so a voxel cannot be both "liver" and "liver_segment_5". The segments win and the plain liver slot stays empty for this model. The eight are grouped under a new "Liver Segments" subsystem so they nest in the organ panel instead of flooding it. - Labels are flat 1..34 with no regions_class_order, so they map straight through. Verified against the checkpoint's own dataset.json. Tests are dependency-light (no app, DB, GPU or dataset). Besides covering the mapping itself, they assert that the Python label indices still agree with the TypeScript segmentation_categories array. Those two files have to be edited in lockstep and nothing enforced it before; if they drift, every label past the drift point renders as the wrong organ. Server setup this needs (not in this change): the unzipped AbdomenAtlasNetOrgans folder and a conda env with nnU-Net v2, pointed at by ATLASNET_ORGANS_MODEL_PATH and CONDA_ENV_ATLASNET_ORGANS.
The first pass at the Couinaud ramp ran into the blue end of the palette, where lungs, veins and postcava already sit. Measured in CIELAB, segment 8 was 4.7 dE from renal vein right and segment 7 was 5.8 from the portal vein, which is indistinguishable on screen. Ends the ramp at teal instead of blue, and darkens hepatic vessel and the portal/splenic vein so vessels running through the liver read as structure rather than as another segment. Worst remaining pair is segment 3 against intestine at 10.5 dE, which do not abut in practice. Checked visually on a real segmentation across three slices, not just by the numbers: an earlier candidate scored better in CIELAB but put the liver in the same violet family as the spleen, which sits directly against it.
The integration is inert until the weights and a conda env exist on the inference box, and that is done by whoever has server access rather than by whoever wrote the code. Writing it down so it does not have to be relayed. Covers the two env vars, the unzip layout (the release is a bare nnU-Net model folder, not an nnUNet_results tree, which is the one non-obvious part), a verification run that does not go through the website, and the measured resource cost: 91 s and 10.7 GB of GPU per scan, but ~26 GB of host RAM, which is the number worth checking against the box.
Completes the AbdomenAtlas 3.0 pair. Segments tumors in liver, kidney and pancreas alongside the organs they sit in. The important difference from Organs is that this checkpoint is region-based: dataset.json defines overlapping regions and nnU-Net converts them via regions_class_order, so the emitted labels are a sparse set (no 4, 5, 11, 12, 13) rather than a dense range. A map written assuming dense labels would paint the wrong organs without erroring, so there is a test that fails if anyone tidies the map into 1..N. Region precedence turns out to be useful rather than just a hazard. Segments are written before lesions, so a lesion voxel inside a segment is emitted as liver_lesion while the surrounding segments survive. One volume therefore carries both, which is what lets the viewer place a lesion in a named segment. Needs no new viewer labels: all 18 targets already exist, including the eight segment slots added for Organs. A test pins the two checkpoints to the same slots for those segments, since they map from different source numbering (Organs 24-31, Tumors 15-22) and nothing else keeps them in step. Not yet run on a GPU. The label set is derived from the checkpoint's own dataset.json but has not been checked against real output the way Organs was.
Same procedure, different zip and a differently named trainer directory. The part worth writing down is that a sparse label set means the opposite thing for each checkpoint: for Organs it signals a problem, for Tumors it is correct.
Ran the checkpoint on an A100 and checked every emitted label against the map. All present, and none of 4, 5, 11, 12 or 13 appeared, so the sparse set is the model's actual behaviour rather than an inference from dataset.json. 227 s and 5.1 GB peak GPU, against 91 s and 10.7 GB for Organs. Slower but about half the memory; most of the extra time is export, not the GPU pass. Still unproven: the scan carried no liver or pancreatic lesion, so those two labels never fired. Running tumour-positive cases to close that.
Six more cases through the checkpoint on an A100. Every emitted value was in the map on all seven, and none of 4, 5, 11, 12 or 13 ever appeared. All three lesion classes fired on at least one scan, so no label in the map is now unobserved. PanTS_00000411 carries several liver lesions, which exercises the reason both checkpoints exist: reading the segments adjacent to each lesion places a 9.4 mL finding in segment 6, a 2.9 mL one in segment 7, and a 3.8 mL one in segment 2. Worth noting for anyone building on this: lesions frequently straddle a segment boundary. The 9.4 mL one is 66 percent segment 6 and 34 percent segment 5, so a readout should say primarily-6-extending-to-5 rather than flatly 6.
Picks up the conda fix and Aditya's SuPreM --resume fix from main. The two AtlasNet runners predated the conda work, so they still built `conda run` by hand and would have failed on the server the same way everything else did. The guard tests caught it on merge, which is what they were for. Also corrects the setup runbook. It said to install a cu121 x86 wheel, which is wrong for bdmap1: that box is GB10/aarch64 and already has an atlasnet env with torch 2.11+cu130 built for it. Points at the existing env instead of having anyone build a broken one.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 470759c6fc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+1080
to
+1083
| nnunet_input = os.path.join(input_dir, f"{case_id}_0000.nii.gz") | ||
| if os.path.lexists(nnunet_input): | ||
| os.remove(nnunet_input) | ||
| os.symlink(input_path, nnunet_input) |
There was a problem hiding this comment.
When a user selects the explicitly supported .nii format, this creates a .nii.gz-named symlink whose contents are not gzip-compressed, so nnU-Net's image reader fails before inference. The existing _stage_nifti_gz helper was added specifically to convert this case; use it here and in the identical AtlasNet-Tumors staging block at lines 1160–1163.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pull request overview
Adds two new AbdomenAtlas 3.0 AtlasNet checkpoints (Organs + Tumors) to the app, extending the viewer’s label space to support Couinaud liver segments and exposing both models as selectable upload-time inference options.
Changes:
- Backend: add AtlasNet-Organs / AtlasNet-Tumors runners and label→viewer remap tables; extend
_VIEWER_LABELSto include 12 new anatomy slots (vessels + 8 liver segments + GI endpoints). - Frontend: extend
SegmentationCategories, organ system grouping (new “Liver Segments” subsystem), and add colors/categories for the new labels; add two new model dropdown entries. - Tests/docs: add unit tests to lock label-map correctness + TS/Python label-order agreement; add a server setup/runbook for AtlasNet weights/env.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| PanTS-Demo/src/types.tsx | Adds new segmentation category literals and a “Liver Segments” subsystem type. |
| PanTS-Demo/src/routes/UploadPage.tsx | Adds “AtlasNet-Organs” and “AtlasNet-Tumors” as selectable model options. |
| PanTS-Demo/src/helpers/constants.ts | Adds 12 new categories, colors (36–47), and groups liver segments under a new subsystem. |
| flask-server/services/auto_segmentor.py | Adds two new inference runners, dispatch branches, viewer-label slots, and label remap tables for both checkpoints. |
| flask-server/tests/unit/test_atlasnet_organs_labels.py | Adds unit tests validating mapping completeness, slot uniqueness, sparsity/precedence rules, and TS/Python label index alignment. |
| flask-server/deploy/atlasnet-organs-setup.md | Adds an operator runbook for installing weights/env and verifying label outputs. |
Suppressed comments (1)
flask-server/services/auto_segmentor.py:1163
- Same staging issue as AtlasNet-Organs: symlinking
input_pathto a*.nii.gzname will break when the uploaded file is a plain.nii(the link promises gzip bytes that aren't there). Use_stage_nifti_gzto handle both.nii.gzand.niiinputs.
nnunet_input = os.path.join(input_dir, f"{case_id}_0000.nii.gz")
if os.path.lexists(nnunet_input):
os.remove(nnunet_input)
os.symlink(input_path, nnunet_input)
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+1080
to
+1083
| nnunet_input = os.path.join(input_dir, f"{case_id}_0000.nii.gz") | ||
| if os.path.lexists(nnunet_input): | ||
| os.remove(nnunet_input) | ||
| os.symlink(input_path, nnunet_input) |
Comment on lines
+11
to
+13
| `_run_atlasnet_organs_inference` in `services/auto_segmentor.py` shells out to nnU-Net inside a | ||
| conda env. It reads two things from the environment and hardcodes nothing: | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add AtlasNet-Organs and AtlasNet-Tumors
Adds both checkpoints from the AbdomenAtlas 3.0 release (RadGPT, ICCV 2025) as selectable
models.
The reason to add them is the eight Couinaud liver segments. Every other model on the site
returns the liver as a single mask, so the most that can be said is "there is a lesion in the
liver". With these, a finding can be placed in a named segment, which is the unit surgeons plan
resections in and the phrasing RadGPT's own generated reports use.
What changed
services/auto_segmentor.py_VIEWER_LABELSslotstypes.tsxLiver Segmentssubsystemhelpers/constants.tsroutes/UploadPage.tsxtests/unit/test_atlasnet_organs_labels.pydeploy/atlasnet-organs-setup.mdNotes for review
Named AtlasNet-Organs / AtlasNet-Tumors, not AtlasNet. The existing
Atlas-Netentry is anunrelated 25-class
Dataset001_ATLASNetthat happens to share the name.Tumors is region-based. Its
dataset.jsondefines overlapping regions with aregions_class_order, so the emitted labels are a sparse set (no 4, 5, 11, 12, 13) rather than adense range. A map written assuming dense labels would paint the wrong organs without erroring,
so there is a test that fails if anyone tidies it into
1..N.That ordering is useful rather than just hazardous: segments are written before lesions, so a
lesion voxel inside a segment comes out as
liver_lesionwhile the surrounding segments survive.One volume therefore carries both, which is what allows placing a lesion in a named segment.
No whole-liver class in Organs. The liver exists only as its segments.
combined_labelsis aflat volume, so a voxel is either
liverorliver_segment_5, never both. Segments win and theplain Liver checkbox stays empty for that model. Forced by the mask format, not a preference. The
eight are grouped under a new Liver Segments subsystem so they nest rather than flood the
organ panel.
Colours were re-picked. The first set collided with existing organs in CIELAB (segment 8 sat
4.7 dE from renal vein, segment 7 3.6 dE from portal vein). The ramp now runs warm to teal and
stops short of the blues that lungs, veins and postcava already occupy.
Verification
Organs, on
bdmap1: verified by @aperson30 against the real deployment. Output labels cameback as a dense subset of 1..34, as expected.
Both, on an A100 (MSI): every emitted value landed where each checkpoint's
dataset.jsonsays it should. Tumors was run across 7 scans with no unexpected values and none of the
forbidden 4, 5, 11, 12, 13 ever appearing. All three lesion classes fired on at least one scan.
On
PanTS_00000411, reading the segments adjacent to each lesion places a 9.4 mL finding insegment 6, 2.9 mL in segment 7, and 3.8 mL in segment 2. Worth knowing for anyone building a
reporting layer: lesions often straddle a boundary, and that 9.4 mL one is 66% segment 6 / 34%
segment 5, so a readout should say primarily-6-extending-to-5 rather than flatly 6.
Still outstanding: Tumors has not been run end to end on
bdmap1yet.Server state
Weights are already unzipped on
bdmap1at the paths the code defaults to, so no env vars areneeded. Both models point at the existing
atlasnetenv (torch 2.11+cu130, built for thatbox's GB10/aarch64 hardware).
Full runbook in
flask-server/deploy/atlasnet-organs-setup.mdfor anyone setting this upelsewhere.
Depends on
Includes the merge of
main, which brings in the conda fix and the SuPreM--resumefix. BothAtlasNet runners predated the conda work and still built
conda runby hand; the guard testscaught that on merge and they now go through
_env_commandlike everything else.License: model weights are CC BY-NC 4.0, patents pending.