occupancy: average mIoU over the classes present, not all 17

[egeboy35]

OccupancyEvaluator.summarize() computes

iou  = self.tp / np.maximum(self.tp + self.fp + self.fn, 1)
miou = float(np.mean(iou))

A class that appears in neither the ground truth nor the prediction has union 0. The clamp turns its undefined IoU into 0/1 = 0, and the mean then divides by the full class count anyway — so the reported number is

mIoU_reported = mIoU_official × n_present / NUM_SEMANTIC

Occ3D-nuScenes averages over the classes actually present (np.nanmean over per_class_iu), so any comparison against a published number is scaled by however many of the 17 classes the scene happened to contain.

Measured

A perfect prediction — the prediction is the ground truth, so the only correct answer is 1.0:

classes present	reported	correct
3 / 17	0.1765	1.0
9 / 17	0.5294	1.0
17 / 17	1.0000	1.0

The identity above holds exactly in each row.

This matters most where the suite is weakest: a scene containing only a few classes, or a baseline that only ever predicts a subset. run_eval.py prints the depth→occupancy baseline's mIoU directly above a table of published SOTA numbers, and those are not on the same scale.

The change

Absent classes are excluded from the mean. They report nan in per_class, so a reader can tell "the model missed it" from "it was never there", and summarize() now returns classes_present / num_classes so a number can be read without guessing which scene produced it. The verbose line prints the same counts.

The "best classes" list drops nan before sorting — nan compares False against everything and would otherwise land in arbitrary positions.

Verification

Nothing changes when every class is present: the existing self-test still reports 1.000 and 0.000. A third case is added for a scene with few classes; reverting the fix turns it from 1.000 to 0.235 and trips its assert.

perfect on a scene with few classes:
  samples=1  mIoU=1.000  geometric IoU=1.000  (4/17 classes present)
OK

🤖 Generated with Claude Code