Skip to content

Fix T407250: build renewal exclusion from pks - #1559

Merged
jsnshrmn merged 2 commits into
masterfrom
theresnotime/T407250-2
Aug 26, 2026
Merged

Fix T407250: build renewal exclusion from pks#1559
jsnshrmn merged 2 commits into
masterfrom
theresnotime/T407250-2

Conversation

@theresnotime

Copy link
Copy Markdown
Member

Description

Fix two defects in the user_renewal_notice management command that stopped
expiry notice emails from being sent.

  1. no_email_list was built as a list of querysets and then passed to
    exclude(pk__in=no_email_list). Django compiles that to
    NOT (id IN ((SELECT ...), (SELECT ...), ...)) — a list of scalar
    sub-queries. An empty sub-query evaluates to NULL, so
    id IN (NULL, ..., 51910) is NULL for every non-matching row,
    NOT NULL is never TRUE, and every authorization was removed from the
    result set. The command now collects primary keys into a set() instead.

    With exactly one renewal application Django collapses IN (single_subquery)
    into a real sub-query, which works correctly. The defect needs two or more
    renewal applications where at least one refers to an authorization outside
    the two-week window. That is why the existing tests never caught it — each
    one had zero or one renewal application.

  2. The partners__isnull=False filter joins the partners M2M and produced one
    row per partner, so a single authorization was emailed once per partner.
    Added .distinct(). This only affects a Bundle authorization that has
    date_expires set, because model validation forbids multi-partner
    non-Bundle authorizations.

user_renewal_notice_diagnostics receives the same two query fixes. It also
now evaluates the old exclusion alongside the new one and prints
<-- DEFECT PRESENT when the two counts differ, so the defect can be confirmed
or ruled out on a running instance.

Rationale

Users stopped receiving access expiry notices. On production the command
selected 22 in-window authorizations and then emailed 0 of them; on staging
it selected 106 and emailed 0. Both are explained by defect 1 above.

Defect 2 was latent only because the query returned nothing. Fixing the
exclusion without adding .distinct() would have sent 84 emails to a single
staging user.

Phabricator Ticket

Bug: T407250

How Has This Been Tested?

Two tests added to TWLight.emails.tests.UserRenewalNoticeTest:

  • test_user_renewal_notice_unrelated_renewals_do_not_suppress — sets up two
    out-of-window authorizations that each have a renewal application, and
    asserts the in-window user still gets an email. Confirmed this test fails
    against the previous code and passes with this change.
  • test_user_renewal_notice_multi_partner_sends_one_email — asserts a
    five-partner Bundle authorization with an expiry date gets one email.

230 tests pass across TWLight.emails, TWLight.users and
TWLight.applications. manage.py check reports no issues. Files are
black-formatted.

Manual check on staging, using the command's own filter logic against real
data: 106 authorizations in the window, 28 renewal sub-queries of which 25
were empty, old exclusion returned 0, corrected exclusion returned 103.

Screenshots of your changes (if appropriate):

Not applicable. user_renewal_notice_diagnostics output serves as evidence:

[Query] authorizations the command WOULD email today: 103
  (in-window & un-reminded, before renewal-filed exclusion: 106)
  excluded because a renewal was already filed: 3
  same count with the old (defective) exclusion: 0   <-- DEFECT PRESENT

Types of changes

What types of changes does your code introduce? Add an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Minor change (fix a typo, add a translation tag, add section to README, etc.)

@theresnotime

Copy link
Copy Markdown
Member Author

Ran on staging:

twlight@staging0:/srv/TWLight$ docker compose exec twlight python manage.py user_renewal_notice --dry-run
[dry-run] would email [redacted] about [...], Women Writers Online, World Scientific (authorization 13747, expires 2026-08-27)
[dry-run] would email [redacted] about MIT (authorization 14371, expires 2026-09-05)
[dry-run] would email [redacted] about Royal Society (authorization 49785, expires 2026-09-04)
[dry-run] would email [redacted] about Newspapers.com (authorization 51888, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 51962, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 51974, expires 2026-08-31)
[dry-run] would email [redacted] about Royal Society (authorization 57725, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 57733, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 57734, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 57735, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 57737, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 57738, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 57739, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 57740, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 57741, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 57742, expires 2026-08-31)
[dry-run] would email [redacted] about Newspapers.com (authorization 57743, expires 2026-08-31)
[dry-run] would email [redacted] about MIT (authorization 57770, expires 2026-09-03)
[dry-run] would email [redacted] about Rock's Backpages (authorization 63054, expires 2026-08-26)
[dry-run] would email [redacted] about Royal Society (authorization 63150, expires 2026-09-02)
[dry-run] 20 email(s) would be sent. No data was changed.

@theresnotime
theresnotime marked this pull request as ready for review August 26, 2026 13:09

@jsnshrmn jsnshrmn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@jsnshrmn
jsnshrmn merged commit 31b201a into master Aug 26, 2026
3 checks passed
@jsnshrmn
jsnshrmn deleted the theresnotime/T407250-2 branch August 26, 2026 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants