ProtoRRG

Official implementation of:

I Wrote This Because of That: Interpretable Radiology Report Generation via Visual Prototypes Findings of the Association for Computational Linguistics: EMNLP 2026

Marco Salmè, Federico Siciliano, Fabrizio Silvestri, Paolo Soda, Rosa Sicilia, Valerio Guarrasi

Paper · Checkpoints

---

Overview

ProtoRRG is an interpretable-by-design framework for radiology report generation that grounds generation in two complementary forms of inspectable evidence: localised visual prototypes and retrieved reports from similar cases.

Visual prototypes are matched against local image regions and their contributions are accumulated into a decomposable evidence score for each finding, providing spatially localised evidence without requiring anatomical supervision. Retrieved reports supply case-based linguistic context for generation. ProtoRRG integrates both sources within a shared architecture designed to reconcile the discriminative representations required by prototype-based prediction with the rich spatial information needed for autoregressive decoding.

---

Installation

Clone the repository and install the required dependencies:

git clone https://github.com/marcosal30/protoRRG.git
cd protoRRG
pip install -r requirements.txt

Additional requirements

The NLG evaluation metrics provided through pycocoevalcap (BLEU, ROUGE, CIDEr, and METEOR) require a Java runtime for the METEOR scorer.

Clinical efficacy evaluation requires a pretrained CheXbert checkpoint. Its path can be specified through:

--chexbert_checkpoint_path <path/to/chexbert.pth>

---

Data

MIMIC-CXR is a credentialed-access resource. You must complete credentialing and download the data from PhysioNet. The corresponding annotations, along with the precomputed text embeddings used for retrieval (text_embeddings.npy, shared across both datasets), are released on the Hugging Face repository.

IU X-Ray is openly available from Open-i. Its annotations are released directly in this repository, at annotations/iuxray_annotation_biomedclipcxr.json, as a JSON list.

---

Training

Training is organized into three phases. The configuration for each phase is provided under configs/.

Phase 0 — visual backbone warm-up

Warm up the visual backbone on the language-modelling objective alone, with the prototype-based classification objective disabled:

python main_train.py \
    --config configs/phase0_lm_warmup.yaml \
    --chexbert_checkpoint_path <path/to/chexbert.pth>

Phase 1 — Prototype-based visual learning

Train the ProtoPNet visual component using the finding classification objective:

python train_phase1.py \
    --config configs/phase1_ppnet.yaml

Phase 2 — Joint training

Jointly train the prototype-based visual encoder and report generation components with retrieval augmentation:

python main_train.py \
    --config configs/phase2_joint_rag.yaml \
    --chexbert_checkpoint_path <path/to/chexbert.pth>

Configuration values defined in the YAML files can be overridden directly from the command line.

---

Evaluation

MIMIC-CXR

python main_test.py \
    --config configs/test_mimic_cxr.yaml \
    --chexbert_checkpoint_path <path/to/chexbert.pth>

IU X-Ray

python main_test.py \
    --config configs/test_iu_xray.yaml \
    --chexbert_checkpoint_path <path/to/chexbert.pth>

The evaluation pipeline computes the report-generation and clinical efficacy metrics used in the paper.

---

Pretrained Checkpoints

Pretrained model checkpoints are released through the Hugging Face Hub:

ProtoRRG checkpoints

---

Acknowledgements

This repository builds upon the implementation and code structure of PromptMRG. We thank the authors for making their code publicly available.

---

Citation

If you find this work useful, please cite:

@inproceedings{salme2026protorrg,
    title     = {I Wrote This Because of That: Interpretable Radiology Report Generation via Visual Prototypes},
    author    = {Salm{\`e}, Marco and Siciliano, Federico and Silvestri, Fabrizio and Soda, Paolo and Sicilia, Rosa and Guarrasi, Valerio},
    booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2026},
    year      = {2026}
}