Skip to content

[stable-diffusion] Match diffusers Transformer2D GroupNorm epsilon - #1448

Open
saichowdary007 wants to merge 1 commit into
ml-explore:mainfrom
saichowdary007:fix/stable-diffusion-groupnorm-eps
Open

saichowdary007 wants to merge 1 commit into
ml-explore:mainfrom
saichowdary007:fix/stable-diffusion-groupnorm-eps

Conversation

@saichowdary007

Copy link
Copy Markdown

Summary

Fixes #1434.

Transformer2D is shared by the Stable Diffusion UNet, but its input
GroupNorm relied on MLX's default eps=1e-5. Diffusers' reference
Transformer2DModel uses eps=1e-6, and the mismatch accumulates through the
UNet. Pass the reference epsilon explicitly while leaving the UNet ResNet and
LayerNorm defaults unchanged.

Testing

  • cd stable_diffusion && PYTHONPATH=. python -m unittest test_unet -v
  • pre-commit run --files stable_diffusion/stable_diffusion/unet.py stable_diffusion/test_unet.py

The regression test constructs a Transformer2D without downloading model
weights and asserts that its GroupNorm uses 1e-6.

Implementation and test preparation used AI assistance; the submitting
contributor reviewed the changed lines and owns the behavior validation.

Fixes ml-explore#1434

Signed-off-by: sai <saikumar@sais-MacBook-Pro.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stable Diffusion: Transformer2D's GroupNorm uses MLX's default eps (1e-5) where diffusers uses 1e-6

1 participant