Cover much more of the UI's JavaScript with tests

[Flix6x]

Description

#2435 made the UI's JavaScript testable. This puts that to use: the JavaScript tests go from 32 to 102, and two pieces of logic move out of templates into modules so they can be tested too.

• ui/tests/js: cover the chart data cache and fetch, the replay, date range, data and chart data helpers, the page-free helpers in ui-utils.js, and the classic scripts flexmeasures.js and map-init.js
• ui/static: move the asset tree's layout from _macros.html into asset-tree.js, and the flex-context editor's logic from assets/asset_context.html into flex-context-utils.js, and cover both
• ui: fix the bugs the new tests found (below)
• ui: embed data in scripts with tojson, and escape user-supplied text built into markup, on every asset and sensor page
• Added changelog items in documentation/changelog.rst

Closes #2452

What is now covered

Module	Newly covered
chart-data-cache.js	everything: missingRanges, effectiveResolutionMs, onSameResamplingGrid, clipToWindow, dedupeRecords, and createChartDataCache against a stand-in fetch (stepping, narrowing, widening, adjacent windows, expiry, off-grid windows)
chart-data-fetch.js	everything
replay-utils.js	everything
daterange-utils.js	thisMonth, lastNMonths, getOffsetBetweenTimezonesForDate, computeSimulationRanges (incl. across DST), encodeUrlQuery
data-utils.js	convertToCSV (timestamps, horizons, nested objects)
chart-data-utils.js	checkDSTTransitions, checkSourceMasking, checkStrictYAxisRanges
ui-utils.js	moveArrayItem, flattenErrorPayload, extractApiErrorMessage, createReactiveState, convertHtmlToElement, processResourceRawJSON, the cached getAsset/getSensor/getAccount, pollJobStatus
flexmeasures.js	numberWithCommas, currency symbols, getTimeAgo, getHumanFriendlyDeltaOrTimeStr (past moments), humanizeIsoDuration, unpackData, getLatestBeliefName, sourceIdFromKey, the timezoneFormat Vega formatter
map-init.js	computeCenteredTreeLayout
asset-tree.js (new)	the tree's hierarchy, one-or-two-row family layout, connectors, highlighting, zoom and label wrapping
flex-context-utils.js (new)	field kinds, commodity scope, cleaning before saving, currency lookup, commitment values

flexmeasures.js and map-init.js are classic scripts, so their tests put stand-ins for jQuery, Vega or Leaflet on window and load them with a <script> element. The README says how.

Refactoring kept away from open PRs

The two templates refactored here are touched by no open PR. The inline code moved nearly verbatim; the templates keep the DOM and ECharts wiring, and import the rest.

Everything else only gains tests, apart from two contained fixes in daterange-utils.js and ui-utils.js, whose hunks are far from those of #2531 and #2554. The new test file for ui-utils.js is called test_ui_utils_helpers.py, because #2531 adds a test_ui_utils.py. There are no tests of how getHumanFriendlyDeltaOrTimeStr phrases future moments, because #2554 rephrases them.

Bugs found

The flex-context editor did not load when a flex-context value held a double quote, such as a commitment named Big "flex" deal. The template embedded the flex-context as its Python representation inside a JavaScript string literal, which the quote ended early. It now embeds it with tojson. The tree and the sensors list on the same page are now embedded with tojson too, instead of as Python representations, and the sensors modal shows sensor names as text.

New commitment prices always defaulted to EUR when the flex-context priced through sensors. activeCurrency() looked up the price sensor's unit with getSensor, which the page never imported. The ReferenceError was caught by the try around the lookup, so the page fell back to EUR without a word. Checked in a browser against the rendered page: with a USD/MWh price sensor, main offers 0 EUR/MWh, and this branch 0 USD/MWh.

The asset tree highlighted assets by name. A sub-asset named like the asset whose page it is was highlighted too. Asset names are only unique among siblings, so that can happen. An asset named "Add asset" was also drawn as the green button. Both are now decided by id.

encodeUrlQuery turned the + of a UTC offset into a space, contrary to its docstring: URLSearchParams reads a literal + as an encoded space. Nothing visible broke, because AwareDateTimeField turns spaces back into plus signs on the server, but the function now does what it says.

processResourceRawJSON mangled values. It turned Python's representation of a dict into JSON with a blanket replace. 'Nonetheless' became 'nulltheless', and an apostrophe in a value made JSON.parse throw. It now translates the representation properly, and also accepts an already-parsed object, which asset_context.html now passes.

Text typed by users was embedded or rendered as markup elsewhere too. The same fixes now apply on the other asset and sensor pages. available_units and the properties page's flex-model are embedded with tojson, where they were a Python or JSON string marked safe inside a string literal. chart_options, whose download file name carries the asset's name, is now dumped HTML-safe. Names, units, graph titles and flex-model values built into markup go through a new escapeHtml in ui-utils.js, in renderSensor and renderSensorSearchResults, in the sensor cards and plot headers of asset_graph.html, and in the KPI cards of graphs.html. The sensor page renders the "dimensionless" hint in the template, so Sensor._ui_unit is gone. It was private and handed the unit to templates as markup.

These are one-line changes in templates that open PRs also touch (#2490, #2267, #2570), so they may need a trivial rebase there.

The UI guidelines in .github/ recommended both patterns, sensor._ui_unit | safe and parsing available_units | safe after swapping quotes. They now ask for tojson in scripts, and textContent or escapeHtml in JavaScript.

I checked each page in headless Chrome, rendered by the app with names, a unit, a graph title and flex-context and flex-model values seeded with quotes and markup. On main, those values stop the scripts on the context and graphs pages, and are rendered as markup on all four pages. On this branch all four pages show them as typed, and none of them logs a new console error.

Potential follow-ups

Inline JavaScript left in templates that open PRs touch, and so not moved here:

Template	Inline JS (lines)	Open PRs touching it
includes/graphs.html	~1,070	#2570, #2490, #1671
assets/asset_graph.html	~1,270	#2490
assets/asset_properties.html	~1,060	#2267
assets/asset_new.html	~430	#2531
assets/asset_automations.html	~390	#2554, #2553, #2457, #2299
sensors/index.html	~400	#2570
sensors/status.html	~270	#2554, #2474
accounts/account.html	~420	#2524
users/user.html	~140	#2214

The untouched templates with inline JavaScript left (dashboard.html, users/users.html, includes/toasts.html and a few smaller ones) are mostly DataTables and DOM wiring, with little logic to test.

How to test

pytest flexmeasures/ui/tests/js

102 tests. They skip without selenium or Chrome, as before.

To see the page-level fixes, open the context page of an asset whose flex-context has a commitment with a " in its name. On main the editor stays empty.

🤖 Generated with Claude Code

[read-the-docs-community]

Documentation build overview

📚 flexmeasures | 🛠️ Build #34674961 | 📁 Comparing d411582 against latest (4aed0c3)

  🔍 Preview build  

4 files changed

± changelog.html
± genindex.html
± _autosummary/flexmeasures.data.models.time_series.html
± api/v3_0.html

[Copilot]

Copilot review overview

🟢 Approval recommended

The changes are cohesive (test expansion + targeted refactors), include tests for the new modules and bugfixes, and the template safety adjustments are consistently applied in the touched areas.

Review effort: Lite
Findings: None

What changed in this PR

This PR substantially expands test coverage for the UI’s JavaScript (pytest-driven, headless-browser execution) and refactors previously inline template logic into importable JS modules so it can be unit-tested. It also includes several UI robustness/security fixes around embedding server data into scripts and rendering user-supplied strings as markup.

Changes:

• Add extensive JS test suite coverage for UI helpers, chart data logic, replay/date-range utilities, and classic scripts (flexmeasures.js, map-init.js).
• Refactor template-embedded JS into new modules (asset-tree.js, flex-context-utils.js) and update templates to import/consume these modules.
• Harden UI rendering and script embedding by using tojson/HTML-safe JSON dumps and escaping user-controlled strings before inserting into innerHTML.

File	Description
flexmeasures/​ui/​views/​assets/​views.py	Stop passing pre-serialized JSON/markup for flex-model and sensor unit; pass raw data for safer tojson usage in templates
flexmeasures/​ui/​utils/​view_utils.py	Use HTML-safe JSON dumping for chart_options embedded into scripts (prevents script-breakout via asset names)
flexmeasures/​ui/​static/​js/​ui-utils.js	Add escapeHtml, unitHtml, robust pythonReprToJSON, and safer processResourceRawJSON; escape user-supplied values rendered via innerHTML
flexmeasures/​ui/​static/​js/​flex-context-utils.js	New testable module containing flex-context editor decision logic (field kinds, cleaning, currency inference, commitment parsing)
flexmeasures/​ui/​static/​js/​asset-tree.js	New testable module containing asset tree hierarchy + layout option construction for ECharts
flexmeasures/​ui/​static/​js/​daterange-utils.js	Fix encodeUrlQuery to preserve literal + (UTC offsets) when using URLSearchParams
flexmeasures/​ui/​templates/​_macros.html	Move asset tree layout logic out of template into asset-tree.js; switch to tojson; highlight by asset id
flexmeasures/​ui/​templates/​assets/​asset_context.html	Embed sensors/flex-context safely with tojson; render sensor names/values via textContent; import flex-context utilities
flexmeasures/​ui/​templates/​assets/​asset_properties.html	Replace unsafe JSON parsing pattern with tojson; pass flex-model object; escape unit option HTML
flexmeasures/​ui/​templates/​assets/​asset_graph.html	Escape user-supplied names/labels; use unitHtml; avoid inserting raw values into markup
flexmeasures/​ui/​templates/​includes/​graphs.html	Escape KPI titles/values/units used in HTML templates
flexmeasures/​ui/​templates/​sensors/​index.html	Remove `_ui_unit
flexmeasures/​data/​models/​time_series.py	Remove Sensor._ui_unit (UI now handles dimensionless rendering safely)
flexmeasures/​ui/​tests/​js/​*	Add many new JS test modules covering the refactored/new JS code paths
flexmeasures/​ui/​tests/​js/​README.md	Document how classic scripts are tested and when logic must be moved into modules
documentation/​changelog.rst	Add changelog items for expanded JS testing and multiple UI bug/security fixes
.github/​instructions/​ui-terminology.instructions.md	Update UI/template safety guidance to prefer tojson and escaping/textContent patterns
.github/​agents/​ui-specialist.md	Align UI specialist checklist with the updated template safety guidance

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Note

This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.