Skip to content

Keep an added filter row visible until it gets a value - #25512

Closed
tangopium wants to merge 1 commit into
devfrom
bug/agile-432-added-filter-row-visibility
Closed

tangopium wants to merge 1 commit into
devfrom
bug/agile-432-added-filter-row-visibility

Conversation

@tangopium

@tangopium tangopium commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

Ticket

https://community.openproject.org/wp/AGILE-432

What are you trying to accomplish?

modules/meeting/spec/features/meetings_index_spec.rb:203 fails on dev, and so does every PR that merges current dev:

Failure/Error: row = page.find(filter_selector(name))
Capybara::ElementNotFound:
  Unable to find visible css ".advanced-filters--filter[data-filter-name='state']"

Behind the spec is a real defect: on the meetings index, adding the "Meeting status" filter no longer works. The row appears and is gone again by the next render, so the filter cannot be set at all. Every list filter added before it is given a value behaves this way.

Adding a filter unhides its row on the client. Every later render of the form comes from the server, which renders a row hidden unless the query holds that filter:

# app/forms/filters/inputs/base_filter_form.rb
args[:hidden] = "hidden" unless @active
# app/components/filters/filter_form_component.rb
yield filter, active_filter.present?, additional_filter_attributes(filter)

Until fd8536f an autocomplete row with no selection was still serialised, with an empty value, so the query held it and the server kept rendering it. That commit changed the parser to report no value at all:

return values.length > 0 ? values : null;

parseAdvancedFilters skips null values, so the filter stops reaching the server and the next render hides the row. The DOM snapshot CI uploads on failure shows exactly that: the row is present and carries hidden, and its entry in add_filter_select is selectable again, so the whole client-side addition had been undone by a server render.

It was not caught because AGILE-432 targeted release/17.9, where the meetings status filter does not exist (d53cc9bd440 is not an ancestor of that branch), and it reached dev through a plain Merge branch 'release/17.9' into dev, which raises no PR and no checks.

What approach did you choose and why?

The controller remembers the rows it added and restores them after each stream render, so they survive a re-render without the server having to know about them.

The obvious alternative is to send the row with an empty value again, which is what dev did before AGILE-432. That gets the row rendered, but it installs a filter the query rejects:

# app/models/queries/filters/base.rb
def validate_presence_of_values
  if operator_strategy&.requires_value? && (values.nil? || values.compact_blank.empty?)
# app/models/queries/base_query.rb
def results
  if valid? ... else empty_scope

So the meetings list would blank out between adding the filter and picking a value. That is a real defect on its own, it simply predates AGILE-432, and reinstating it to fix this one is not a trade worth making.

Restoring on the client keeps both properties: nothing invalid reaches the query, and the result set and the parameters AGILE-432 protects (page, and the Backlogs inbox all) are untouched, since no request is made when a row is merely added. The change is additive, and no behaviour introduced by AGILE-432 is altered.

A row stops being tracked as soon as the server renders it visible, which means the query holds it by then and the server owns it again.

AI involvement

Directed – I specified the requirements and AI implemented most of it; I validated via testing rather than a full line-by-line review.

Merge checklist

  • Added/updated tests
  • Added/updated documentation in Lookbook (patterns, previews, etc)
  • Tested major browsers (Chrome, Firefox, Edge, ...)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-22T11:32:30.212056Z f9456d8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown

Caution

The provided work package version does not match the core version

Details:

Please make sure that:

  • The work package version OR your pull request target branch is correct

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f9456d8b50

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// A row the user has added but not given a value to yet is still part of the form. It is
// sent with an empty value so the server keeps rendering it after the re-render, while
// `hasEffectiveValue` keeps it from counting as a change of the result set.
return values.length > 0 ? values : [''];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid submitting blank autocomplete filters

When an autocomplete row is added without a selection, returning [''] submits an active filter such as {"state":{"operator":"=","values":[""]}}. In the meetings flow, ParamsToQueryService installs that filter on the query, Queries::Filters::Base rejects the blank value for a value-requiring operator, and Queries::BaseQuery#results consequently returns empty_scope; therefore adding the status filter immediately clears all meeting results even though hasEffectiveValue treats the row as result-neutral. The placeholder row needs to be preserved without making the server execute an invalid active filter.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, and it changed the approach. [""].compact_blank is empty, so validate_presence_of_values marks the query invalid and BaseQuery#results returns empty_scope: the meetings list would blank out between adding the filter and picking a value. That is what dev did before AGILE-432 too, which makes it a pre-existing defect rather than a new one, but not something to reinstate.

The row is no longer sent at all. The controller keeps the names of the rows it added and restores them after each stream render, so the query never sees a blank filter.

@myabc
myabc requested review from myabc and a lite review from Copilot September 22, 2026 11:34
@myabc

myabc commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

@tangopium thanks for this! would you mind updating the PR description to use the standard PR template (.github/pull_request_template.md)?

@github-actions

Copy link
Copy Markdown

Warning

Flaky specs

  • rspec ./spec/features/projects/creation_wizard/wizard_from_template_flow_spec.rb[1:1]
  • rspec ./spec/features/work_packages/new/attributes_from_filter_spec.rb[1:3:1]
🤖 Ask Copilot to investigate

Copy the prompt below into a new comment on this PR to delegate the investigation to GitHub Copilot. It will look into the flakiness and open a separate pull request with you as reviewer.

@copilot The following spec(s) are flaky in CI (first seen on PR #25512, linked for reference only):

- `rspec ./spec/features/projects/creation_wizard/wizard_from_template_flow_spec.rb[1:1]`
- `rspec ./spec/features/work_packages/new/attributes_from_filter_spec.rb[1:3:1]`

Treat this as a standalone task, unrelated to PR #25512. Create a new branch from origin/dev and open a new pull request targeting dev — do not stack it on PR #25512 or reuse that branch.

Follow the playbook in docs/development/testing/handling-flaky-tests/README.md to find the root cause and fix the underlying race — do not skip, delete, or weaken the spec to make it pass; disabling is a last resort per the playbook, and only with a bug ticket. Verify the fix by running the spec(s) repeatedly (e.g. `script/bulk_run_rspec --run-count 10`).

If you cannot reproduce the flake or are not confident in a fix after reasonable investigation, do not fabricate a change or skip the spec to force CI green. Instead, leave the pull request in draft and document what you tried, the suspected cause, and any leads in its description, then assign @tangopium to take over.

Once the fix is verified, title the PR after the spec(s) it fixes, and use the PR description to explain the root cause, how the change resolves it, and the before/after results. Label the PR `flaky-spec`, assign @tangopium, and request a review from @tangopium.
On every commit, set @tangopium as the sole co-author with a `Co-authored-by:` trailer (use their GitHub no-reply email so it links to their account), so it is traceable who dispatched the fix.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved findings affect filter counting, request rollback, and parameter-reset behavior.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

This PR keeps newly added, unvalued filter rows visible after server re-rendering while separating effective filters from placeholders.

Changes:

  • Serializes empty autocomplete rows.
  • Resets pagination and expansion only when effective filters change.
  • Adds navigation regression coverage.
File Summary
frontend/​src/​stimulus/​controllers/​dynamic/​filter/​filters-form.controller.ts Updates filter serialization and effective-filter tracking. Findings: critical (2 votes) counter inconsistency; moderate (3 votes) failed-request rollback; moderate (1 vote) fresh-load baseline comparison.
frontend/​src/​stimulus/​controllers/​dynamic/​filter/​filters-form-navigation.controller.spec.ts Adds coverage for empty-row persistence and preserving existing-filter parameters.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +630 to +633
// A row the user has added but not given a value to yet is still part of the form. It is
// sent with an empty value so the server keeps rendering it after the re-render, while
// `hasEffectiveValue` keeps it from counting as a change of the result set.
return values.length > 0 ? values : [''];

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and untested: the counter specs use a text row, which returns null when empty, so the inconsistency you describe would not have shown up in CI.

The parser change is reverted entirely in the new version, so parseFilters() and the counter behave exactly as they do on dev.

})
.catch((error:Error) => {
this.sentFilters = previousFilters;
this.sentFilters = rollbackFilters;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. sentEffectiveFilters was set before the fetch and only sentFilters was restored in the failure path, so a retry would have skipped the reset of page and all.

That field is gone in the new version: nothing is sent when a row is added, so there is no second piece of state to roll back.

oliverguenther
oliverguenther previously approved these changes Sep 22, 2026
@myabc myabc mentioned this pull request Sep 22, 2026
2 of 3 tasks

@myabc myabc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tangopium thanks for taking a look at this - and sorry that AGILE-432 ended up breaking on dev.

I'm not sure about serialising the empty row as status_id = "" however, since this has three effects I'd like to avoid:

  1. Queries::Filters::Base#validate_presence_of_values rejects [""], and the Meetings query is never valid_subset!-ed, so Queries::BaseQuery#results returns empty_scope: adding "Meeting status" empties the list until a value is picked (see Codex's finding: #25512 (comment)).
  2. An unsent draft reaches URL and history via visit / replaceState. Back/Forward and reload then reconstruct a row the user never applied. This will cause an issue in an upcoming work package I'm currently specifying (AGILE-375).
  3. It needs a second "effective filters" memory next to sentFilters, which both OP-20194 (#25280) and AGILE-375 rewrite.

Normally I'd prefer to provide feedback and give you the opportunity to revise. However in this instance - since it's causing failures on CI - I've gone ahead and proposed an alternate solution #25519. Sorry if this isn't good PR etiquette!

@oliverguenther
oliverguenther dismissed their stale review September 22, 2026 14:34

Tab-switching caused me to approve the wrong PR

Adding a list filter unhides its row on the client. Every later render of
the form comes from the server, which renders a row hidden unless the query
holds that filter, so the row the user just added is hidden again. Selecting
"Meeting status" on the meetings index is impossible this way, and
meetings_index_spec.rb:203 has been failing on dev for it.

Sending the row with an empty value would get it rendered, but it would also
install a filter the query rejects: Queries::Filters::Base requires a value
for an operator that needs one, so BaseQuery#results falls back to an empty
scope and the list blanks out until a value is picked.

The controller keeps the names of the rows it added and restores them after
each stream render instead. Nothing invalid reaches the query, the result
set stays as it was, and a row stops being tracked as soon as the server
renders it visible, which means the query holds it by then.

https://community.openproject.org/wp/AGILE-432
@tangopium
tangopium force-pushed the bug/agile-432-added-filter-row-visibility branch from f9456d8 to e6492e4 Compare September 22, 2026 16:23
@github-actions github-actions Bot added the ai: Directed 🪄 A human specified the requirements and AI implemented most of it; They validated via testing. label Sep 22, 2026
@tangopium

Copy link
Copy Markdown
Collaborator Author

Description now follows .github/pull_request_template.md.

The approach changed with the force-push as well: the first version sent the added row with an empty value, which would have installed a filter the query rejects and emptied the result list until a value was picked. The controller now restores the rows it added after each stream render instead, so nothing is sent and no behaviour from AGILE-432 is altered. Details are in the description, and the review threads have the specifics.

@tangopium

Copy link
Copy Markdown
Collaborator Author

Closing in favour of #25519, which fixes the same regression and does it better.

Both fixes are client-side and neither sends a filter the query would reject, so the direction is the same. The differences that matter:

  • [AGILE-432] Keep an added filter row visible #25519 prevents the row from being hidden by cancelling the morph of hidden, rather than restoring the row after the morph has hidden it. Preventing beats repairing.
  • It handles a marker left in the DOM by a Turbo page snapshot on connect(). This version has no equivalent and would have been wrong in that case.
  • It adds an integration spec, where this one only had jsdom unit tests. That was the gap called out in the original description here.

The diagnosis in this PR stays valid and is reproduced in #25519: the server renders a filter row hidden unless the query holds it (base_filter_form.rb:84), fd8536fb2de stopped an unvalued autocomplete row from reaching the request, and the combination only met on dev because AGILE-432 was cut from release/17.9, where the meetings status filter does not exist, and arrived through a release merge that raises no checks.

@tangopium tangopium closed this Sep 22, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 22, 2026
@tangopium
tangopium deleted the bug/agile-432-added-filter-row-visibility branch September 22, 2026 16:40
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

ai: Directed 🪄 A human specified the requirements and AI implemented most of it; They validated via testing.

Development

Successfully merging this pull request may close these issues.

4 participants