Skip to content

Fix: PandasReporter with droplevels keeps one belief per event - #2595

Merged
Flix6x merged 2 commits into
mainfrom
fix/pandas-reporter-one-belief-per-event
Sep 25, 2026
Merged

Flix6x merged 2 commits into
mainfrom
fix/pandas-reporter-one-belief-per-event

Conversation

@Flix6x

@Flix6x Flix6x commented Sep 25, 2026

Copy link
Copy Markdown
Member

Description

Closes #2594.

A PandasReporter with droplevels: true drops the source level of its inputs and asserts one row per event,
so it failed with AssertionError: BeliefDataframe has more than one row per event. as soon as an input event was reported by more than one data source.
This showed up when porting a reporting job to an automation gave its results a new data source, and a downstream job reporting on a rolling window saw both sources until the window moved on.

  • With droplevels, each input is now searched with one_deterministic_belief_per_event=True, as AggregatorReporter and ProfitOrLossReporter already do.
  • This is a default, set with setdefault: an input that sets one_deterministic_belief_per_event itself keeps its value, instead of the keyword reaching search_beliefs twice (TypeError).
  • Configurations that work today are unaffected, because the assertion only passed where each event already had one deterministic belief, which the flag leaves as is.
  • The assertion stays, as a safety net.

How to test

pytest flexmeasures/data/models/reporting/tests/test_pandas_reporter.py -k droplevels

  • test_pandas_reporter_droplevels_with_several_sources uses the existing fixture data in which two sources report the same noon event. It fails on main with the AssertionError above.
  • test_pandas_reporter_droplevels_respects_input_override checks that an input's own one_deterministic_belief_per_event: false is respected. It fails with a TypeError if the flag is passed as a fixed keyword instead of a default.

The whole reporting test package and the report CLI tests pass.

🤖 Generated with Claude Code

With droplevels, the reporter drops the source level and asserts one row per event,
so an input sensor with an event reported by several sources made it fail.
It now searches such inputs for one deterministic belief per event,
unless an input sets one_deterministic_belief_per_event itself.

Closes #2594

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x added the bug Something isn't working label Sep 25, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 flexmeasures | 🛠️ Build #34754501 | 📁 Comparing bb9fc28 against latest (6dd0460)

  🔍 Preview build  

2 files changed
± changelog.html
± api/v3_0.html

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The newly added tests introduce unused local variables (likely flake8 F841) that should be cleaned up to avoid lint failures.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Low severity

Open (1)
What changed in this PR

This PR fixes PandasReporter failures when droplevels: true is used and multiple data sources report the same event, by defaulting input searches to one_deterministic_belief_per_event=True (while still allowing per-input overrides). It adds regression tests covering the multi-source overlap case and the override behavior, plus a user-facing changelog entry.

Changes:

  • Default one_deterministic_belief_per_event to droplevels via setdefault in PandasReporter.fetch_data.
  • Add tests ensuring droplevels keeps one belief per event when multiple sources overlap, and that explicit per-input overrides are respected.
  • Add a main changelog entry describing the bugfix.
File Description
flexmeasures/​data/​models/​reporting/​pandas_reporter.py Default deterministic belief selection when droplevels is enabled to prevent duplicate events after dropping index levels
flexmeasures/​data/​models/​reporting/​tests/​test_pandas_reporter.py Add regression tests for multi-source overlap under droplevels and for respecting per-input overrides
documentation/​changelog.rst Document the bugfix in the main changelog

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread flexmeasures/data/models/reporting/tests/test_pandas_reporter.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The change is narrowly scoped, aligns with existing reporter behavior, and includes targeted regression tests plus a changelog entry.

Review effort: Lite
Findings: None

Resolved since last review (1)

@Flix6x Flix6x self-assigned this Sep 25, 2026
@Flix6x Flix6x added this to the 1.1.0 milestone Sep 25, 2026
@Flix6x
Flix6x merged commit 5190c2e into main Sep 25, 2026
15 checks passed
@Flix6x
Flix6x deleted the fix/pandas-reporter-one-belief-per-event branch September 25, 2026 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working Reporting

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PandasReporter with droplevels fails when several sources report the same event

2 participants