Conversation
* implement most of issue 1 * update cp_surface * correct period meaning * update docstrings * finish issue 1 * minor comment updates * remove turbine_key mirrors change in resgroup/hill-of-towie-open-source-analysis#53 * address PR comments * Update hill_of_towie.py
* issue 2 initial effort * Simplify docstring Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * add example_hot_study.py * address PR comments * improve kwargs enforcement --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* issue 3 WIP * Update example_v0_study.py * fix md has different turbines than cfg.asset.wtgs * write out tdf * add missing save_plots guards * fix lint * remove HOT details from era5.py * fix format * fix test_explicit_mplbackend_is_respected * address PR comments
* update issue 4 * WIP * cache per year, turbine * format * improve naive plots * improve naive plots * improve issue 4 descr * rename example prepost * rename example prepost * rename example prepost * run fast methods first * fix TypeError * address PR comment * address PR comment
* refactor out scadadf * improve e2e tests * address PR comments * fix test flake
* update issue text * RLearner WIP * RLearner WIP * RLearner tested against naive * update gitignore * add overnight scripts * fix typo Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * improve overnight scripts * add findings.md * address PR comments --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* add inspect_prepost_hard_case.py * add diagnostics modules * mandatory availability filter * fix naive plot bug * fix docstring Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix docstring Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * update findings.md * address PR comments --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* power_model * update docstrings * improve histograms * address PR comments
* add study_power_model_compare.py * add study_power_model_compare baseline * uplift by condition WIP * update power_model baseline results * add by condition diagnostic plots * add by condition diagnostic plots * update findings.md * plan issues 6-8 * address PR comment
* add prediction clip * add condition dependent comparison * add _select_profiles * add --accept-candidate * update baseline * address pr comments
* add ERA5 future work * issue 8 substantially complete * address PR comments * accept new conditional uplift method * update issue 8 text * address PR comments
* issues 9-11 * address PR comments * explore feature removal * improvements from feature removal * address PR comments
* issue 12 * address PR comments
* issue 13 complete * further investigate half life * address PR comments * skip rlearner e2e tests * address PR comments
* issue 14 WIP * add 1 and 2 months to baseline * accept matched count 50 * issue 14 full benchmark * address PR comments
* issue 15 WIP * re-number issues * re-number issues * fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* prune opt-in behaviour * update docstring
* update findings.md * add power conditional WIP * add power conditional WIP * improve conditional diagnostic plots * add power plot * rerun study_power_model_compare * address PR comment
* Update naive_ratio.py * Update method.py * make benchmarking importable * use shared build_toggle_df * fix log bug * address PR comments
* make columns required * address PR comments
* add ToggleSpecialistMethod * Update test_naive_ratio.py * Update toggle_specialist.py * complete issue 17 * feat: add campaign_weeks grid + toggle-methods regression harness Adds a weeks-based campaign-length grid alongside the existing months grid, and a study script that regression-tests the two toggle-capable methods (toggle_specialist, power_model) on Hill of Towie. The weeks grid is additive: StudyConfig takes exactly one of campaign_months / campaign_weeks, and leaderboard/plot_campaign_curves take a length_col defaulting to campaign_months. Every existing months-based call is therefore bit-identical, so the committed power_model baseline stays valid. months-only accessors (CampaignWindow.months, StudyConfig.max_activity_months) raise on a weeks study rather than silently reporting weeks as months. study_toggle_methods_compare.py scores a placebo plus a symmetric +/-2% Cp pair over 1/2/4/8 weeks -- the small-signal, short-campaign regime a real toggle campaign lives in. It reports raw deltas against a committed benchmark: ground truth is deterministic in (config, seed), so an unchanged method must diff to exactly 0.0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * add validate_conditions * Create study_toggle_methods_compare_baseline.json Recorded on f6b509b's method code. The stamp reads 04f36d6-dirty because validate_conditions (a pure, unwired addition) landed mid-run and test files were uncommitted; neither method's source differed. * enforce kwargs * add binning to toggle specialist * Update density.py * update vocab * update study_toggle_methods_compare_baseline.json * address PR comments * Update study_toggle_methods_compare.py * Update study_toggle_methods_compare_baseline.json * V1 ts uncertainty (#127) * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * clean up baseline loose end * address PR comments * add labeled_rows * make power bins symmetric * address PR comments * format --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* exclude_row WIP * address PR comments
* WIP * WIP * WIP * address PR comments * address further PR comments * add inspect_wake_steering_case.py * create issues_campaigns.md
…133) Drop the rlearner package entirely, carrying forward only the shared piece power_model still needs. - Relocate make_outcome_model (+ its _COMMON params and lazy lightgbm import) verbatim from rlearner/nuisance.py into power_model/fitting.py; repoint power_model/method.py and inspect_era5_matching_importance.py. Estimator construction is byte-identical, so power_model behaviour is unchanged. - Carry over the factory's direct unit test into test_power_model_fitting.py (TestMakeOutcomeModel) so relocating it does not drop its unit coverage. - Delete the rlearner package, its seven tests, and the rlearner-only inspect_prepost_feature_ablation.py. Rename the "rlearner" fixture labels in test_plots.py (they were plain strings, no import). - Drop the now-stale rlearner references in era5_sync.py and method.py docstrings. Acceptance: poe all-fast green; power_model reads UNCHANGED against both committed benchmarks (study_toggle_methods_compare max delta 0.06 pp; study_power_model_compare 0 material moves), i.e. identical to within LightGBM's same-machine noise floor. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…nt drivers (#134) Post-C7 housekeeping ahead of the campaigns tranche. - Reword all "R-learner" references left behind by the rlearner package deletion (C7) to describe current behaviour. Notably the overnight studies advertised "oracle + naive + R-learner + v0" but now run power_model. - Strip finding-number citations (F1/F5/F13/F30/...) from source and tests per the CLAUDE.md rule; the rationale stays in docs/v1/findings.md. - Move spent one-off / manual inspection drivers into benchmarking/baselines/old/: migrate_toggle_baseline_v2_to_v3, inspect_era5_matching_importance, inspect_short_campaigns, inspect_naive, inspect_v0_run (no tests, not imported by the active tree). poe all-fast green; no scored-method or benchmark behaviour changes. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* W0: src/ layout — rename legacy wind_up to wind_up_v0, add v1 wind_up skeleton Adopt a src/ layout so the v1 package can claim the `wind_up` import name while the legacy tool is retained as `wind_up_v0` (distribution stays `res-wind-up`; only import names change). - move legacy `wind_up/` to `src/wind_up_v0/`; add a minimal `src/wind_up/` v1 skeleton (docstring + __version__ + py.typed) for W1 to fill - repoint every importer (v0_binned baseline, benchmarking, tests, examples, and the smarteole notebook) to `wind_up_v0` - constants.py PROJECTROOT_DIR parents[1] -> parents[2] (repo root is one level deeper) - pyproject packaging (where=["src","."]), coverage source, ruff paths, CODEOWNERS - document Git LFS as a prerequisite; gitignore the smarteole example download Behaviour-preserving: the v0 output-schema key "wind_up_version" is unchanged, poe lint is green (mypy 120 files), 819 tests pass, and the smarteole/wedowind example plots and numbers are byte/pixel-identical before and after. benchmarking/ stays packaged temporarily (imported by a separate project); dropping it from the release artifact and deleting the legacy config/input_data/cache root folders are deferred to W2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * update v1 README * W0 review: fix 3.10 mypy, repoint README to wind_up_v0, harden baseline guards Address PR #135 review feedback: - benchmarking/synthetic/upgrades.py: pin shape-agnostic npt.NDArray[np.float64] annotations on cp_ratio/ws_factor/nacelle_delta and cast the _is_waked argument to float64, so mypy passes under the numpy resolved on Python 3.10. This lint failure predates W0 (it came in with the wake-steering work and the v1 branch was already red on 3.10); the fix is type-only, no runtime change. - README.md: repoint the quick-start imports from `wind_up` to `wind_up_v0`. The `wind_up` package is now the empty v1 skeleton, so the old snippets raised ModuleNotFoundError. - src/wind_up_v0/py.typed: PEP 561 marker for the retained, fully typed legacy API. - test_naive_ratio / test_toggle_specialist: the independence guards now reject both `wind_up` and `wind_up_v0`, so an "independent" baseline cannot quietly import either. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* begin C1 work * C1: campaign declaration + runner + farm uplift + placebo campaign Stands up the whole declaration -> runner -> farm-uplift -> reporting pipeline on a placebo (zero injected uplift) whole-farm campaign, so a campaign is declared rather than hand-wired. - src/wind_up/farm.py: farm_uplift(), the pure headline function, with the capacity-factor cap and non-negativity floor, per-turbine guard flags and spread. - benchmarking/campaigns/: SyntheticCampaign (private, holds the injected upgrades) derives CampaignSpec (public facts only); CampaignRunner; the report; and the two placebo campaigns with their driver. - true_farm_uplift(): the N-turbine generalisation of true_net_uplift. - conditional_truth_vs_estimate moved from inspect_prepost_hard_case into harness/plots.py, so campaigns/report.py does not import the v0 pipeline for a frame-shaping helper. Re-exported, so existing callers are unchanged. Consumers read per-turbine facts through spec.timing_for()/usable_mask() and the mode through spec.mode, so C8's per-turbine change histories change accessor bodies rather than every call site. Results on real Hill of Towie SCADA are logged in docs/v1/findings_campaigns.md (CF1-CF5): truth is exactly 0 in both modes; toggle beats prepost by an order of magnitude; the farm result reaches +0.148% with six test turbines; and T06 is identified as the best failure-mode fixture turbine. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Review: drop "treated" from src, and "headline" -> "result" PR #136 review. Two terminology changes in the product surface: - TurbineUplift.treated_energy -> actual_energy. The codebase already pairs "actual" with "counterfactual" -- power_model documents its headline as ``sum_actual / sum_counterfactual - 1`` -- and farm.py already had counterfactual_energy, so "treated" was the odd one out. The docstring formula now reads the same way in both places. "actual" is also neutral for the analyses C8 covers that are not upgrades: confirming stable performance, or quantifying a loss event. - "headline" -> "result" throughout, since a farm is not necessarily the top level of aggregation. src/wind_up/ no longer contains "treated". The benchmarking layer keeps it (including the shared treated_mask helpers) for C8 to sweep once the neutral umbrella term is settled, rather than leaving a third vocabulary in play. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Review: guard non-finite energy and unusable ratings in farm_uplift Copilot review on #136. The guards accepted non-finite actual energy and non-finite or non-positive rated power, and each failed in its own way: - NaN energy passed with used=True and no guard, then vanished from the result entirely, because the summation skips NaN. The farm number looked clean and n_guarded reported 0. - Infinite energy produced an infinite farm result. - A negative rating made the capacity cap clip the counterfactual to a *negative* value, violating the documented non-negativity floor, and gave a result below -100%. - A NaN rating silently disabled the cap. All four are now dropped with explicit guard reasons (non_finite_energy, invalid_rating), so they are visible in n_guarded and the per-turbine detail rather than silent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Review: pool each method's farm truth over the turbines it actually used Copilot review on #136. farm_uplift can drop a turbine (a non-finite estimate, say), so a method's farm estimate may cover a subset of the campaign while its truth was pooled over every upgraded turbine. signed_error then subtracted one estimand from another -- and it did so precisely when a method misbehaved, which is when the number matters most. Each farm row's truth is now pooled over that method's used turbines. n_guarded flags the rows where turbines were dropped, since a method that dropped some is not directly comparable with one that used them all. CampaignResult.truth_farm_uplift is unchanged: it remains the all-turbine campaign truth. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Review: excluded turbines are not candidate references; ERA5 only when needed Copilot review on #136, two independent placebo fixes: - candidate_references included excluded turbines, so the spec simultaneously offered a turbine as a reference and forbade its use. Behaviour was correct because the runner drops those rows, but C3 is "automatic reference selection from the CampaignSpec" and would have read the contradictory field. - build_hot_v0_context fetches ERA5, and only the power model reads it, so include_power_model=False still paid the network dependency. It is now built only when the power model is enabled, which makes the fast path genuinely offline. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Review: plot only the conditions a method actually reported Copilot review on #136. The report built truth for every condition whenever a method reported any conditional output, so a method reporting one axis still got plots for the others. toggle_specialist reports only "power", so every toggle run wrote ws and ti charts whose method series was entirely NaN while being labelled method-vs-truth. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * C1 scope: v0 is out of scope for the placebo The placebo is a whole-farm campaign and v0 enumerates test/reference combinations per turbine, so a whole-farm v0 run is not tractable. Recording the decision rather than leaving an unmet acceptance criterion: Copilot correctly flagged on #136 that the scope asked for v0 as an optional slow method and the driver did not provide one. The seam still accepts V0BinnedMethod unchanged, so a later campaign over a small turbine subset can include it. C4 still expects v0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * C8: add the harness "window" disambiguation; track the robustness design C8 scope gains two items that belong with its vocabulary sweep: - Disambiguate "window" in benchmarking/harness/campaign.py, which uses it for two different spans in one docstring: CampaignWindow is the whole baseline-plus-activity span, while its prose says "post window" / "activity window" for the treated part alone. That is the ambiguity C1 renamed `window` to `analysis_period` to escape, so a reader who knows the harness will misread the spec field. - Retire "treated" from the benchmarking layer in the same pass (443 uses, 64 of them the shared treated_mask helpers). src/ is already clear of it. Also drops "treated period" as the example neutral fallback, since the same issue retires that word. The robustness design note was untracked and had picked up the C1 naming pass ("brief" -> "campaign spec"), so those edits existed only in the working tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Mark C0, W0 and C1 done in the campaigns issues list C1 merged as PR #136. C0 and W0 were already complete but unmarked, so the list did not show where the tranche stood. Each now carries a Status line in the same shape C7 uses, and the suggested order marks the finished issues and names C2 as next. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * add spec * C2: campaign context on the method seam Methods learn their candidate references and row validity from a CampaignContext the runner derives from the CampaignSpec, instead of each deriving "every turbine except the test one" -- nine such derivations across four methods, and a declaration nothing read. CampaignContext (harness/context.py) is a narrow per-test-turbine view, so methods are not coupled to the analyst-facing declaration that C8 generalizes and W2 promotes to public API. context_for (campaigns/context.py) is the one translation point, and the one place to audit that no truth reaches a method. Declared validity (exclusions, C8 histories, C5 wake gating) rides the context; screened validity (R3) stays method-internal. valid_for_uplift is named for its purpose: data can be invalid for uplift and still valid for a northing analysis. MethodInput gains a lazily-built default context that is today's implicit contract, so existing call sites are unchanged. Adds ColumnSchema.northed(role) for R1's derived column. Behaviour change: the campaign path now honours the declared candidate_references, so the placebo's six upgraded turbines stop acting as each other's references. Placebo CF1-CF5 in findings_campaigns.md are stale as a result. The study path is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * C2 review: enforce context invariants, cover declared turbines Addresses the PR 137 review. MethodInput now rejects two silent-failure modes: an input with neither a campaign_context nor an upgrade_timing (which built a context with timing=None and failed later in resolve_toggle), and a context whose test_wtg differs from the input's (which would estimate one turbine while reading another's references and row validity). context_for now covers every declared turbine present in the frame, not just the test turbine and its references, and select() raises rather than silently keeping the rows of a turbine it has no validity for. A method co-analysing several turbines keeps them via select(also=...); those rows were previously exempt from declared validity. Also hoists a set() rebuilt per column in the two wide-format methods. Not changed: build_toggle_df already de-duplicates its index internally and documents itself as indexed by unique timestamps, so passing a long-format index is correct and unchanged from before this branch. Docs: each C-issue with a real Hill of Towie counterpart now runs it alongside the synthetic campaign to test the shape of the declaration (no ground truth, not scored); C3/C4/C5 name their counterpart. W2 gains a YAML-declared CampaignSpec, with those three real campaigns as the acceptance test for "easy to use". C2's done-when records the placebo re-record as its closing step. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * CF6: re-record the placebo under the C2 reference rule C2 made the campaign path honour the declared candidate_references, so the six upgraded placebo turbines no longer serve as each other's references. Re-runs both placebo campaigns and records the result as CF6, controlled against the same driver run from the pre-C2 commit a1f96af. The control reads +0.1485%, matching the +0.148% CF3 recorded, so the comparison is a like-for-like A/B of the reference rule. power_model's prepost farm error falls from +0.148% to +0.039%, while its mean per-turbine absolute error rises slightly (0.481 -> 0.515 pp): the headline gain is cancellation across turbines, not better individual estimates. The mechanism is explicitly not established. Truth is still exactly 0.0 in both modes. CF1 is marked superseded; CF2-CF5 carry a note that they predate the reference rule and are not reproducible as written, their sweeps not having been re-run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Re-record the toggle compare baseline (stale since #126/#128) The portable baseline dated from de85f84 (2026-07-15). #126 (V1 toggle specialist) and #128 (custom filter to toggle specialist) then added ~270 lines to toggle_specialist.py without re-recording it, so toggle_specialist had been reading MOVED on every run since: 67 of 84 cells, worst 1.52 pp, against a band of 1e-5 pp. The drift is confined to the conditional per-bin cells (condition == "power"); no overall row moved, so the headline P50 was never affected. That is consistent with those PRs reworking the conditional path deliberately and forgetting the re-record. Not caused by the C2 work on this branch: an A/B with the context calls neutralised is bit-identical, and a control sweep on the pre-C2 commit a1f96af produces the same 19 moved cp_0pct cells to the digit. Recorded from a clean tree at 2e13ac3 on linux, then verified with a fresh compare: power_model UNCHANGED (max delta 0.0531 pp, band 0.1) and toggle_specialist UNCHANGED (max delta 5e-07 pp, band 1e-5) — its effectively bit-exact band is usable again as the sharp instrument for small regressions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* R1: shared northing step, fast DP norther, and the northing fault fixture
Replace v0's ruptures-based hill-climb with an exact dynamic-programming
changepoint search in the new v1 package, wire it in as a shared
preprocessing step upstream of every method, and add the fault injector
and fixture that show a northing step biting and then being closed.
Estimator (src/wind_up/northing.py)
- Daily circular-median aggregation, O(1) prefix-sum segment costs, exact
DP, local refinement, closed-form per-segment offsets. numpy only.
- Two passes preserved: reanalysis anchors the farm in absolute terms
(a uniformly-wrong farm is self-consistent and invisible to pass 2),
the farm consensus then supplies precision.
- Knobs are physical: min_step_deg, changepoints per year, min_segment.
- circular_math moves to wind_up with a wind_up_v0 re-export, so the
releasable v1 package does not depend on the legacy one.
Two effects found by measurement, not foreseen in design
- Reanalysis carries its own direction drift (~4 deg common-mode at
Homer), indistinguishable from every turbine shifting at once, so
against_reanalysis floors min_step_deg at 10 deg.
- Site veer makes a turbine's residual depend on which directions the
wind blew from, so a shift in the direction mix reads as a step. Fixed
at the cause: sector-normalise the residual for detection only, and
iron out small self-cancelling excursions. A step above
max_transient_step_deg is never ironed out, because real
recalibrations do sometimes reverse (T16: 98, 9, 7, 89 deg, net +11).
Effort tiers were built, measured and removed: the speed/quality trade
did not exist (3.1s to 5.2s across a 21-turbine farm-year, in a ~40s run)
and the cheap tier was worse for no saving. One NorthingSettings remains.
Evidence
- Homer ported tests: identical median yaw and max northing error to the
old implementation on all three offsets.
- Hill of Towie, 21 turbines, 2017-2018: 9.9x faster (389.9s -> 39.3s),
and it rediscovers v0's changepoints exactly - {T01: 2, T05: 2,
T16: 3} - plus one on T13, whose worst-case error improves.
- Fixture (T06 + T15/T10/T08, AeroUp uplift, 40 deg step on T15):
prepost bites +1.331 pp and is closed to +0.135 pp, no harm +0.032 pp;
toggle does not bite (-0.025 pp), recorded rather than assumed. Clean
arms discover 0 changepoints, faulted arms exactly 1.
ruptures is dropped from the dependencies, the mypy overrides and the
lockfile.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* R1: fix the record-edge false changepoint, add northing plots
Two findings from comparing against v0's published Hill of Towie table,
plus the plots that make either visible to a user.
Edge false changepoint (fixed)
- The estimator reported a +3.5 deg step on T13 at 2018-12-20 that v0's
table does not have. A window sweep settled it: the step exists only
when the record ends on 2019-01-01, twelve days later. Extend the
record by two days and it is gone.
- Cause: with twelve days after it, the "after" level is a veer-dominated
estimate that landed 3.36 deg from the "before" level, just over the
3.0 deg threshold.
- Fix: scale the required step with the record supporting it. A segment's
level is veer-limited rather than sample-limited and veer averages out
no faster than 1/sqrt(span), so the threshold grows as
sqrt(confident_segment / span), capped at max_transient_step_deg so a
large late jump stays findable.
- A flat "near an edge demand more than 10 deg" rule was tried first and
rejected: it broke T16's genuine +9.0 deg step, which has only 30 days
before it. The scaled rule requires 5.2 deg there and keeps it.
Outage artefact (documented, not fixed)
- Nearly every turbine gains self-cancelling pairs at 2019-11-11/19 and
2020-06-12/19. Those weeks are farm outages: v0's add_wf_yawdir
silently fills a missing farm direction with reanalysis (35% and 51% of
rows; turbine count falls to a median of 2 against the >=3 rule), and
where it does not, the farm median is taken over a different subset of
turbines, which is a different quantity because veer differs per
turbine. The v1 path is already correct on the first mechanism. A
strict-xfail test records the second so it announces itself when fixed.
Tests, written before the fix
- A real-data fixture (git-lfs, 15 MB): timestamp, raw yaw, farm
reference and a reference-fallback flag for six turbines over
2016-2020 -- only what the estimator consumes, so cases run in ~0.3s.
- Known changepoints that must keep being found: T01 x2, T05 x2, T16 x3,
and T07/T11 staying clean.
- Edge artefacts: T13 clean in every window, including the four that
previously produced the false step.
- Synthetic: a large jump ten days from either end is still found; the
same small step is kept well inside the record and dropped near the edge.
Farm-scale result is now an exact match with v0 over 2017-2018: 7
changepoints, same turbines, same dates, steps within 0.65 deg.
Plots (src/wind_up/northing_plots.py)
- Per device: 14-day median residual before and after with the fitted
steps drawn on, and residual by direction sector, where what remains
after correction is site veer.
- Farm overview, and a conditions plot of residual mean and spread
against direction, wind speed and power.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* R1: fix the farm-wide outage artefact -- it was the first pass
Over 2016-2024 nearly every turbine gained self-cancelling changepoint
pairs at 2019-11, 2020-06 and 2023-06: 143 changepoints against v0's 28,
with 111 of the 127 extras in those three months.
Three hypotheses, two of them wrong
- Silent reference substitution (v0's add_wf_yawdir filling a missing
farm direction with reanalysis): removes the Aug 2020 pair and nothing
else.
- Changing reference composition: plausible but measured false. After the
first pass every device's long-run offset from the farm median is
within -0.3 to +0.5 deg, and centring the devices moves the median by
0.00 deg in every window. An attempt to level the reference per sector
made the healthy case worse and was reverted.
- The first pass. During the June 2020 excursion northed-minus-farm is ~0
for every turbine while the raw residual is +20, so the correction the
first pass applied *is* the excursion; it had inserted 2-6 changepoints
per turbine across 2020.
Cause and fix
Reanalysis carries its own direction-dependent bias, so a spell of
unusual wind (that week was easterly, a sector HoT rarely sees) moves
every turbine's residual against it together by tens of degrees. The
first pass corrected for that, writing the excursion into the northed
directions and hence into the farm consensus the second pass trusts. The
outage correlates only because both are weather.
The first pass may now act only on a gross recalibration
(ANCHORING_MIN_STEP_DEG, 30 deg) -- above reanalysis' own excursions and
below a real one (HoT's are 36-177 deg). Everything finer is left to the
second pass, which works against the clean consensus and estimates from
the raw direction, so nothing is lost. Blocking the first pass entirely
was tried and rejected: with four devices one uncorrected 40 deg step
drags the median enough to break a real detection. A quorum (a strict
majority of the farm, not a floor of three) was added alongside; it earns
its place on the subset cases rather than this one.
Measured, 21 turbines, 2016-2024
- changepoints 143 -> 19 (v0 has 28); not in v0's table 127 -> 2
- v0's changepoints recovered 16/28 -> 17/28
- 99-case subset sweep (3 groups x 33 windows, 3 months to 9 years):
cases finding extras 40/98 -> 19/98; total extras 287 -> 32
Tests
The real-data fixture now stores raw inputs rather than a precomputed
farm reference -- the old one had been built with this very bug, so the
tests could not see it. Cases run north_farm end to end over two 2-year
windows (the search costs about the cube of record length, so this covers
the same events for a quarter of the runtime): every v0 recalibration
found, every other turbine silent, no step during either outage, T13
clean wherever the record stops, and west/east halves agreeing with the
whole farm. 52 real-data tests in 31s.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* R1: apply the first-pass fix to v0 too, and clear the scar tissue
v0 still had the outage bug
The v0 adapter's first pass called against_reanalysis (10 deg) while the
v1 path had moved to anchoring_only (30 deg), so v0 kept re-detecting the
farm-wide outage excursions the previous commit removed from v1. Exactly
the divergence a shared core exists to prevent. Both now anchor the same
way; the v0 second-pass fallback keeps against_reanalysis, which is right
for it -- that pass still does changepoint work, just conservatively.
Simplification
- anchoring_only wrapped against_reanalysis and then overrode min_step
with 30, which is already above the 10 deg floor, so the inner call was
dead. It now sets the one field it means to.
- sector_signature was made public for the reference-levelling fix that
was measured, found to make the healthy case worse, and reverted. Its
only caller is veer_normalised, so it is private again.
- _farm_quorum sat among the module constants, pages from its only
caller; moved next to _farm_direction.
- The two pruning passes were the same loop written twice -- score every
changepoint, drop the worst, re-estimate, repeat -- and their call
sites shared six of eight arguments. The loop is now _prune_while and
the two rules are small named predicates (_worst_transient,
_worst_unsupported) that read side by side.
west__year_2021 investigated, not fixed
All six extras are T11, whose data coverage in 2021 is healthy
(1.4k-3.9k usable rows a month) and which is the turbine v0 itself
adjusted most: seven entries in its table, every one between 2.2 and 8.6
degrees. Our answer there varies with the window because the evidence is
genuinely ambiguous at that scale, not because of a defect I can point
at. Making the persistence horizon local was tried as a fix and reverted:
it produced byte-identical results on every affected case, so it was a
parameter for nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Add a Greenbyte source adapter (Kelmarsh, Penmanshiel) and road-test northing on both
Two open Cubico datasets published on Zenodo, exported from Greenbyte in
a shared CSV layout. Both are simpler than Hill of Towie -- six and
fourteen Senvion turbines against HoT's twenty-one -- which makes them a
second and third site for anything that must not be tuned to one farm.
The adapter returns the same long, source-native shape the rest of the
benchmarking layer speaks, with GREENBYTE_COLUMNS in the usual
ColumnSchema vocabulary. Two source quirks are absorbed here, where
source-specific knowledge belongs: availability is published as a
fraction of the period (converted to seconds, as HoT reports it) and
Penmanshiel's static CSV carries a trailing blank row. Zips are found by
globbing rather than by published filename, so a year split across two
files works and so do the shorter names a manual download leaves behind.
Road test of the northing solution (2017-2018, no ground truth; the
question was whether it runs on other data without much effort and
whether it wanders)
turbines load north_farm changepoints monthly range
Kelmarsh 6 8s 1.5s 0 0.5-1.4 deg
Penmanshiel 14 25s 3.9s 0 0.5-4.5 deg
No changepoints on either farm, which is right -- neither has a
documented recalibration in the period -- and a meaningful negative
result given how readily the estimator invented them on HoT before the
recent fixes. No veer-like wandering: Kelmarsh is flat to within 1.4 deg
month to month, and eleven of Penmanshiel's fourteen are under 2 deg.
Penmanshiel's worst (T11, 4.5 deg) is veer being sampled rather than the
corrector moving: its corrected trace sits inside the +/-1 deg band for
two straight years, while its residual by direction sector swings from
-6.8 deg at 135 deg to +2.0 deg at 285 deg, which no single offset can
remove.
Tests build the export layout rather than needing the published zips,
which are hundreds of megabytes: the comment preamble and commented
header, turbine-number padding, UTC index, the availability conversion,
status files ignored, a year split across two zips, multi-year ordering,
the trailing blank metadata row, and the two failure messages.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Land the northing subset-consistency study as a driver
The 99-case sweep that found the outage and edge artefacts only existed
in a session scratchpad, so the exercise could not be repeated after a
change -- which was the point of defining it. The subsets are now
declared in the module (3 turbine groups x 33 windows, 3 months to 9
years, straddling the known recalibrations and the known farm outages),
so two runs are directly comparable.
It reports, per case, how many changepoints were found, how many matched
the full-record run over the same window, how many were invented and how
many lost. On the shipped estimator that reads 19 changepoints for the
full 2016-2024 run against v0's 28, with 19 of 98 cases inventing
anything, total extra 32.
The input frame is built and cached a year at a time: nine years of
21-turbine SCADA at once needs far more memory than the northing itself,
while the handful of columns the estimator reads compresses to tens of MB.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* R1: stop veer normalisation being defeated by its own de-stepping
The sector signature was measured on a residual de-stepped by the first
detection pass. Under veer that pass over-detects, so de-stepping removed
the very sector levels the signature describes -- leaving nothing to
subtract, and the spurious splits survived the pass meant to remove them.
Search the normalised residual first with no step structure assumed, then
re-measure the signature around only the confident steps that search found.
Same number of searches. A real recalibration still dominates the first
search and is still de-stepped; a speculative split is not.
Subset study (99 cases, HoT 2016-2024): extra 32 -> 27, matched unchanged
at 211, runtime 820s -> 541s, reference case byte-identical. The whole
improvement is T11 in 2021 -- six self-cancelling changepoints collapse to
one 3.95 degree step, the window-dependence R1 had left as ambiguity.
Also records CF7: a turbine is inside the farm consensus it is northed
against, which pins pass 2 to pass 1 on small odd farms. Leave-one-out was
built, measured and reverted (better detection, more spurious changepoints);
the regression test stands as xfail(strict).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Stop tracking development-phase docs (docs/superpowers, CLAUDE.md)
These are local working documents -- design notes, plans, and the Claude
Code guidance file -- not part of the published project. Add both to
.gitignore and remove docs/superpowers from the index; every file stays
on disk.
Tracked files under docs/v1/ still cite these specs by path, so those
citations now resolve only on a development machine. W2 carries a scope
item to resolve that before release: fold what is still true into
docs/methodology.md or docs/v1/, and drop the rest as development history.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* R1: north the study path too, and turn the direction feature on by default
The shared northing step ran only in CampaignRunner, so the study drivers behind
the frozen benchmarks had no northed column and power_model's direction feature
had to ship opt-in. Both paths now north.
The study path norths per replicate, discovering for itself rather than being
handed a table discovered once on the base SCADA. The cheaper variant was
rejected: supplying a prior table is the unrealistic help the benchmark exists
to rule out, and it stays valid only so long as no study profile injects a
direction fault. Northing runs after generation, so a direction-moving upgrade
stays consistent with its northed companion. Measured cost is ~11 s per
replicate on the 4-turbine HoT subset.
The step moves to benchmarking/harness/northing.py -- campaigns imports harness,
so the reverse would be a cycle -- and north_campaign_scada becomes north_scada,
taking the two fields it read (north_offsets, rated_power_kw) instead of a
CampaignSpec the study path does not have. northing_fixture's private
_era5_direction is promoted to era5_direction, the recipe both paths use.
Flipping direction_feature broke 36 tests, all in test_power_model_method.py and
all at __post_init__'s require_roles("nacelle_position"). Three fixture edits
recovered every one: _COLUMNS names the role, and _toy_scada / _shrinkage_scada
carry a site-wide direction each turbine reports through its own miscalibration
plus the northed companion that removes it. No test deleted, no tolerance
loosened.
study_power_model_compare_baseline.json is NOT yet regenerated, so the committed
benchmark is stale against this code. The placebo also moves: it built its
methods without direction_feature, so it has been running with the feature off.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* R1: address PR 138 review -- circular_math tests and typing, src comment style
Migrate the circular-math unit tests off the v0 re-export and into tests/wind_up:
test_math_funcs -> test_circular_math, plus the two rolling-window modules, all now
importing wind_up.circular_math. Fills a gap while there: circ_median's axis
parameter was documented and had its own apply_along_axis branch but no test, so
add axis=0 and axis=1 cases on data straddling 0/360.
Type circ_diff and circ_median the way numpy functions are typed -- ArrayLike in,
ndarray-or-scalar out -- which covers float scalars and pandas Series. The explicit
list type goes, and with it the isinstance branch in circ_diff: np.subtract handles
lists directly and keeps Series in, Series out. Behaviour is unchanged.
Strip markdown bold from src/wind_up/northing.py and northing_plots.py, and cut the
comments that justify rather than describe -- the three min-step constants carried
four-line rationales each, including a measured claim that will drift. What a caller
needs to use the API correctly stays: offsets are absolute so tables compose, and
the two pruning passes are order-dependent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* R1: address Copilot review on PR 138
Five findings, each verified against the code before acting on them.
north_scada now raises when a declared table's first offset for a turbine begins after
the data starts. apply_north_table extends the first value backward, so such a table
silently corrected the earliest rows with a later offset. A turbine with no declared
offset is still left alone; the vendored Hill of Towie table starts every turbine at
2016-01-01, so nothing that supplies it is affected.
CampaignRunner always calls north_scada. The _should_north predicate skipped northing
when north_offsets was None and no ERA5 was supplied, which contradicts None meaning
"discover" and deferred the failure to whichever method wanted the northed column.
north_scada already raises an actionable error for that case.
The Greenbyte loader's default column set omitted gen_rpm, although the adapter declares
the role and generate_dataset reads it unconditionally -- so default-loaded data could
not enter the synthetic pipeline. Verified the column is published for Kelmarsh, and
that a default load now feeds generate_dataset.
The real-data northing tests skipped on Path.exists(), which is true for an unsmudged
git-lfs pointer; they then tried to parse the pointer as Parquet. Check the Parquet
magic bytes instead.
Delete baselines/old/inspect_short_campaigns.py: it builds a power model and calls
score_study without era5_wd, so it cannot run now the direction feature defaults on.
The other four modules under old/ do not construct a PowerModelMethod and still work.
Note it is still cited by docs/v1/issues.md and findings.md as the provenance of past
measurements; those citations now dangle, which W2's documentation cleanup covers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Mark the heavy integration tests slow so the fast gate stays quick
`pytest -m "not slow"` had grown to a few minutes, which is too slow to run between
edits. Marked from measured durations rather than guesswork:
- tests/benchmarking/campaigns/test_placebo_end_to_end.py, at module level -- eight
full campaign runs and by far the largest single block. Matches the existing
treatment of the other *_end_to_end modules.
- the heavy real-data northing sweeps, and the plot-writing diagnostics cases.
- the wake-steering example driver, and four stragglers in the v0 tests.
Coverage the fast gate keeps deliberately: real Hill of Towie data still runs there
(the outage, edge and single-turbine-against-reanalysis cases), and so do five of the
thirteen diagnostics tests. The intent was to defer long integration runs, not to stop
exercising real data between edits.
Timings were taken while a benchmark sweep had the machine, so the absolute target
still needs confirming on an idle box; the deselected count went from 25 to 92.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* R1: add the natural-case northing probe
Sizes the naturally occurring instance of the failure mode, with nothing injected. T06's
nearest neighbour T05 carries real step changes in its reported yaw direction across
2017-2018, so using T05 as a reference exercises a real northing fault; the probe runs the
same campaign with the shared step off and on and reports how far the answer moves.
Two reference sets, because dilution is part of the answer: T05 alone, where it drives the
reference direction entirely, and T05 alongside the fixture's three stable neighbours, where
its effect is diluted. The AeroUp uplift shape is injected so truth is non-zero and the number
reported is an error rather than placebo drift, which also makes it comparable to the
fixture's clean column.
v0 is out of scope here -- the new norther has already been shown to track v0 across the
farm-scale HoT comparison and SMARTEOLE, so the probe runs power_model with naive_ratio as a
control that reads no direction and should not move between arms.
Kept separate from northing_fixture rather than parameterising it: that module produced R1's
acceptance evidence and hardcodes its turbine set in several places.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* R1: the natural probe's minimum reference set needs three devices
The first run died on the T05-alone arm: north_farm forms its consensus from at least three
devices, and the test turbine plus one reference is two. Replace that set with T05 plus one
stable neighbour, which keeps the dilution contrast the probe is for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Re-record the toggle power_model benchmark for the direction feature
R1 turned power_model's reference-direction feature on by default and made the study path
north per replicate, so this benchmark no longer described the model it was recorded from.
Re-recorded at 7e1c77a.
Checked before accepting. The headline is effectively unmoved: over the twelve `overall`
cells the mean change in score is +0.005 pp, worst cell 0.27 pp. Across the sixty
`power`-condition cells in real power bins it is -0.023 pp. The whole of the "MOVED" verdict
comes from the twelve cells in the near-zero-power bin (-230, 230] kW, which average +0.845 pp
and hold the worst cell -- the bin where the turbine is barely generating and the counterfactual
model has least signal to go on. That mirrors the sparse-bin behaviour in the power_model
benchmark and is not a headline regression.
The portable baseline is deliberately untouched: toggle_specialist reads no direction signal,
and the run confirmed it at a maximum change of 5e-07 pp, so the script left that file alone.
Only the per-platform linux file, which holds power_model's non-portable cells, is rewritten.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Define R5 (northing refinement) and record how to update the frozen benchmarks
R5 captures the two places R1's norther is known to fall short, both found while doing R1
rather than guessed at. Part A: north one or two devices with pass 1 alone, since the farm
consensus refuses below three devices and a two-device campaign therefore cannot be northed at
all. What limits pass 1 is reanalysis' own direction-dependent bias, which is why only gross
steps may be attributed against it; the development loop is unusually good, because Hill of
Towie offers 21 turbines with a published table to score a one-turbine-at-a-time answer
against. Part B: a third pass that nudges the solution toward absolute truth using apparent
wake nadirs, whose direction is fixed by layout geometry and owes nothing to a reanalysis
model. It runs after the changepoints and relative steps are settled, and outputs an offset
only -- it must not quietly change which rows downstream analysis treats as valid.
The benchmark section records what this session cost to learn. There are four frozen
baselines rather than one, and a shared feature-engineering step moves all of them. A
candidate recorded from a dirty tree is refused, so commit before sweeping. Diffing against a
baseline recorded weeks ago measures every commit since, so isolate the change with a
feature-off re-run before accepting -- here that retired a real worry in under an hour. The
MOVED verdict is blunt: degenerate bins dominate the means while their medians sit at zero.
The comparison CSV is in fractions while the logs are in percentage points. And a method that
reads none of the changed signal is a free control worth predicting in advance.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Re-record the power_model benchmark for the direction feature
R1 turned power_model's reference-direction feature on by default and made the study path
north per replicate, so the benchmark recorded on 2026-07-16 at 4f07d64 no longer described
the model it was measured from. Re-recorded at cbe3297.
The change was attributed before being accepted, rather than inferred from a diff against a
seven-week-old baseline. The same sweep was run twice on the same code and seeds, once with
direction_feature on and once with --method-overrides '{"direction_feature": false}'; northing
runs in both arms, so the only difference is whether the model reads the northed direction.
The feature-off arm reproduced the July baseline to within float noise -- mean absolute change
over the thirty-five headline cells of 0.00001 pp prepost and 0.00000 pp toggle, worst single
cell 0.00013 pp. So the six intervening commits that touched power_model, C7's outcome-model
relocation and CampaignContext among them, were behaviour-preserving, and the whole of the
movement is the feature.
The feature itself is neutral on the headline: mean change in score over the overall cells is
-0.024 pp prepost and +0.016 pp toggle, both inside the script's 0.1 pp materiality band.
Within prepost it is not uniform -- one-month campaigns improve by 0.668 pp and two-month ones
worsen by 0.417 pp -- but it nets out near zero. The large per-condition means are sparse bins:
the TI and wind-speed conditions average +8.0 and +2.3 pp against medians of -0.09 and 0.00,
and every large cell sits in TI 0.4-0.5 or wind speed 0-2 m/s, hitting all seven profiles at
the same campaign length, which is the signature of a near-empty bin rather than a regression.
R1 turns the feature on for robustness to a northing fault, which the fixture 2x2 demonstrates.
What this establishes is the other half: it costs nothing on clean data.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Mark R1 done, and close the example re-runs as redundant
Records the two Done-when items settled by decision rather than built. v0's arm is dropped:
the fixture never ran V0BinnedMethod, accepted because the norther tracks v0 through the
21-turbine farm-scale comparison, the SMARTEOLE road-test and the natural probe's rediscovery
of v0's published T05 table.
The example re-runs are closed as redundant rather than left as debt. Flipping
optimize_northing_corrections would add no coverage: v0's auto path already has six tests
through the adapter including injected changepoints, plus those three comparisons, and the
supplied-table path the examples actually ship is covered by the SMARTEOLE and WeDoWind
end-to-end tests.
W2's example item gains the consequence: northing changes shape in the migration, because the
v0 examples pin a pre-computed table while v1's shared step discovers by default, so a migrated
example should show discovery rather than port the pinned table across.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Placebo discovers its own northing, and the shared step draws its plots
The placebo supplied the vendored Hill of Towie table, so it applied a known answer and never
exercised the norther. It now declares no table and discovers, which is what a placebo on real
data should demonstrate.
Wire the northing plots in while doing it. src/wind_up/northing_plots.py existed but had no
callers and no tests, so R1's "the tool shows its working" was not actually delivered: the
functions were there, nothing invoked them, and a user running a campaign got no plots.
north_scada now takes out_dir and, whenever it discovers, writes the farm overview plus one
plot per device; CampaignRunner passes northing_out_dir through.
Measured on the real placebo window, 21 turbines over 2017-2018: the norther found seven
changepoints and the vendored v0 table holds seven inside that window -- the same seven, on the
same turbines (T01 twice, T05 twice, T16 three times), three of them to the exact ten-minute
record and none more than 12h40m out, with every offset within 0.75 degrees. Given no prior
table at all, it reproduces the published one across a whole farm.
ERA5 is now fetched whether or not the power model runs, since it is the anchor discovery needs.
The end-to-end placebo test is unaffected: its fixture ships no nacelle position, so the step
returns early on the absent role rather than reaching the reanalysis requirement.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Record that the campaigns test v1 wind-up, not a field of methods
The "methods carried forward" ground rule listed oracle, naive_ratio, power_model and
toggle_specialist flat, as if they were peers. They are not: the C-series exists to exercise
the thing being shipped, so a campaign has to run it as a user would get it -- power_model on,
and northing discovered rather than supplied. naive_ratio is a deliberately simple yardstick
and never a candidate for the shipped method; oracle is a sanity anchor. toggle_specialist
stays TBD, to be settled with evidence in W1.
The rule now says so, and says the consequence: a campaign that turns power_model off, or that
supplies a north table, is testing something other than v1 wind-up and its result should be
read that way. Tests may switch power_model off to avoid the ml dependency; drivers should not.
The include_power_model docstrings say the same where someone would reach for the flag.
Also removes _north_offsets from placebo.py a second time. It came back after the previous
commit with its imports gone, so it referenced undefined names and had no callers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Northing needs a real anchor, and writes the table it discovered
Two findings from review on PR 138.
north_farm accepted a reanalysis series that was entirely missing. Pass 1 then fell back to a
zero offset per device with only an INFO line, and pass 2 went on to produce perfectly
plausible relative corrections from the farm consensus alone. Reproduced on a three-device
farm: an all-NaN reanalysis recovered the injected +25 and -40 exactly, anchored on nothing. A
farm uniformly wrong would come back all zeros and look flawless, which is the failure the
two-pass design exists to prevent, and a misaligned ERA5 index reaches it by accident. It now
raises unless some device has a usable row where the reanalysis is finite, and warns when
fewer devices than the quorum do -- "there is an anchor" rather than "the array is non-empty",
since a series that is 95 percent NaN is just as unanchored.
The shared step also discarded the tables it discovered, so nothing downstream could see what
had been decided. It now writes northing_corrections.yaml beside the plots, in the format v0
wrote and both north_offsets and v0's northing_corrections_utc read, so an analyst can inspect
it, hand edit it, and supply it back as a prior. Verified on the real placebo farm: the file
parses to the same list-of-three shape as the vendored table and round-trips into
(turbine, Timestamp, float) tuples.
C5 gains the third finding rather than a fix. WakeSteering moves the reported nacelle position
on treated rows and yaw_usable screens only on power and downtime, so steered rows enter the
northing fit. Excluding treated rows is the obvious answer and is wrong in general: in prepost
they are half the record, and a north step inside the campaign is exactly R1's fault, so
excluding them would make it undiscoverable. The note records what limits the damage today,
that the bias is unmeasured, and what has to be settled.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* CF9: record the placebo results with power_model after R1
The placebo numbers on record (CF6) predate R1, so they describe a power_model that read no
direction and a campaign handed the vendored north table. Re-run on the same configuration
with what v1 wind-up now is: the direction feature on, and northing discovered.
Mean per-turbine error falls from 0.515% to 0.312% prepost and from 0.328% to 0.255% toggle.
naive_ratio and toggle_specialist reproduce CF6 to every recorded digit on every turbine in
both modes, so nothing but power_model moved and the comparison is clean.
T06 lands at +0.044% prepost, which matters beyond the average: it is the R-series fixture
turbine, picked in CF5 for accuracy and stability, and it is now essentially exact on a real
placebo.
The prepost farm number drifts the other way, +0.039% to +0.076%, which is the mirror of CF6:
better individual estimates leave less residual to cancel. Both sit well inside the ±0.2%
target. Toggle improves on both axes.
Worth recording because the frozen benchmarks called the direction feature neutral. They
measure four-turbine synthetic campaigns; the placebo is 21 real turbines whose record
contains real northing faults, which is where a north-calibrated direction can contribute.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Drop the unused northing diagnostics, and put v1 in the coverage gate
circular_spread, plot_residual_conditions and their two helpers had no caller
anywhere -- leftovers from the R1 investigation.
The coverage run only ever named wind_up_v0, so nothing in src/wind_up was
measured. Adding it showed northing_plots.py at 13%: no test passed out_dir, so
the plots never ran. Smoke-test the two survivors the way v0's plots are tested,
and omit the module from the report the way v0's plots are omitted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* A sector level must belong to the row that reads it
_sector_signature filled `sector` only where the value was finite, but indexed it
for every row with a finite reference, so a row whose value was NaN silently took
sector 0's level. Every present caller masks those rows out, so nothing moved --
the real-data regressions are unchanged -- but the next caller would have been
bitten. Derive the sector from the reference alone.
The two prune call sites splatted `**bounds` and `**rule`, which mypy checks not
at all: renaming a key produced no error. Spell the arguments out. Binding the
sector width to a local retires the type: ignore alongside.
Docstrings that argued rather than described are cut back. The one piece of
measured evidence among them -- the dropped low-effort tier -- is recorded as
CF10 rather than lost.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* Say when the farm reference falls back, and stop hardcoding the outage count
The fallback log was gated on the settings object changing, so it stayed silent
whenever min_step_deg was already at the reanalysis floor -- the fallback had
happened and nothing said so. Gate it on the condition itself.
v0's northing YAML writer duplicated write_north_table_yaml; delegate to it. The
output is unchanged but for a trailing newline, which nothing reads.
test_the_outage_years_are_quiet asserted a literal 4 that had to be kept in step
with EXPECTED by hand. Derive it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D
* tidy docstring
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* R2: unstable-sensor faults, measured on the placebo fixture Define the R2 failure mode and measure what it costs wind-up today. No power_model change: the headline is already immune, and that is the finding. Adds SensorGainStep and SensorGainDrift, which scale a schema role family (the anemometer channels by default) by a constant or a linear ramp. Mean and SD scale together, so turbulence intensity is invariant and only the wind-speed axis moves. Power is untouched, so ground truth never moves. The sensor_fixture driver runs 28 arms on a placebo of T06 plus T15/T10/T08 (truth exactly 0): two shapes x gains x1.5/x0.5 x two targets x both modes, plus an exposed set carrying reference anemometry as features. Supporting passthroughs: faults= on placebo_campaign, reference_stat_cols= on carried_forward_methods. Results (CF11): power_model moves at most 0.214 pp in prepost and exactly zero in toggle. The one nonzero arm is the ERA5 lag sweep tipping one row (10 min) under a x0.5 reference gain, which is why step and drift agree to six decimals. The exposed arm moves 34 pp, so the standing no-reference-anemometer-features rule is now measured rather than assumed. The conditional ws axis moves hard (it is the test turbine's own anemometer); left alone deliberately while that machinery is nascent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * R2 review: dedupe resolved fault columns so a repeated role cannot square the gain `roles` is public and a ColumnSchema may name the same column for two roles, so `roles=("wind_speed", "wind_speed")` scaled that channel twice and applied gain**2 rather than the documented gain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Fix the Python 3.10 lint and test failures on v1 Both are numpy-version artefacts: uv resolves numpy 2.2.6 for 3.10 and 2.4.6 for 3.13, which is why only the 3.10 jobs fail. lint: under 2.2.6 `np.full(n, nan)` types as ndarray[tuple[int], ...] while `np.where` returns ndarray[tuple[int, ...], ...], so the accumulator assignment is rejected; 2.4.6 widens it. Annotate the accumulator. test: with identical turbines naive_ratio divides two mathematically equal sums, and 2.2.6's summation order puts the quotient an ULP off 1.0, so the estimate reads 2.22e-16 rather than 0.0. The assertion demanded exact float equality; compare against a 1e-12 bound instead, which is ten orders of magnitude below any uplift worth reporting and still catches a real bias. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* R3: invalid references — fault, fixture and reference-validity screen Phase A. `ReferenceCpChange` injects an undeclared Cp change into a reference turbine. It changes real power, so `faults.py`'s contract is re-cut around what actually holds for all four faults: injected after the declared upgrades, invisible to the analyst, never moves the ground truth. That last part is enforced rather than documented — `Fault` gains `changes_power` and `generate_dataset` refuses a power-changing fault aimed at a test turbine, which would otherwise be absorbed into the truth derived for it. `benchmarking.campaigns.reference_fixture` declares the arms: a 3-reference pool (clean, +5%, -4%) and a 5-reference pool (clean, two references at -4%). The degradation is the realistic case — a reference is likelier to pick up a problem of its own than an unannounced improvement — so the larger pool uses it. In prepost the change lands at the changeover; in toggle, halfway through the test period, since a change predating a toggle test is common-mode across its blocks. The fault bites: at 3% it moved `power_model` -0.69 to -1.14 pp in prepost and -0.31 to -0.52 pp in toggle (CF12). Phase B, in progress. The screen estimates each candidate reference as if it were a test turbine against the others, and rules out clear outliers from what is normal for this farm over this period — not turbines that changed. A pack that drifts together reads ~0 on each other, correctly, since the test turbine would have drifted too. Deviation from the pool's median, an absolute floor, worst one dropped per pass. A screened reference is made *power-free* rather than removed: it keeps its direction features and gains a `waking` boolean, so the wake information its operating state carries survives while the channels a performance change corrupts do not. Screening always uses a prepost contrast — a reference is not the thing being toggled, so an on-block/off-block question about it answers nothing. Still open: `screen_floor` is a placeholder pending calibration, and the remediation A/B (drop / direction-only / direction+waking) has not been run. Committed at the user's request without waiting for the full suite; lint is clean and the touched test modules pass (40 fixture, 28 features, 26 screening, 11 method-screen). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: report reference uplift and screening detail; fix waking dtype The waking column was bool, which collapses to object dtype once reindexed onto the full timestamp index, and the outcome model rejects that — it crashed the fixture run. It is now tri-state as a float: waking, not waking, or unknown where the reference has no record. Unknown is the honest third state; asserting not-waking there would invent a fact. `power_model` now reports what the screen did and what the references read. `MethodOutput` gains two additive optional frames: `screen_passes` (per pass, per turbine: estimate, deviation, whether it was dropped) so an analyst can see and disagree with the screen's decisions, and `reference_uplifts` (per reference: uplift, actual energy, record count, whether screened). Reference uplift is a standard campaign sanity check — a healthy campaign's references should read near 0% — so it is computed under the campaign's own timing, not the screen's prepost contrast, and combined through `farm_uplift` by energy sums, the same path the test turbines take. It is reported *post-screen*: a ruled-out reference is listed and flagged, so the drop stays visible, but is excluded from the combined number and from every reference pool. A healthy prepost campaign reuses the screen's final pass instead of refitting the pool, since the contrast and pools are then identical. Any drop, or a toggle campaign, refits — dropping a reference changes what every survivor reads. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: stop screening once a drop leaves too few references to vote The "needs at least 3 candidate references" guard was applied only on entry, not per pass, so a three-reference pool that dropped one kept screening the remaining pair. Two references are always equidistant from their own midpoint, so the rule is degenerate there: it flagged one arbitrarily and then hit the majority guard, crashing the fixture with a farm-wide-problem error on a pool whose only real fault had already been found. Found by the fixture rather than by a unit test — the toy estimator happened to put the surviving pair inside the floor, real data did not. The test that pins it asserts no pass ever runs on fewer than three turbines, which is the invariant rather than the symptom. Two existing tests encoded the old behaviour (expecting a second pass on a three-reference pool) and now use a five-reference pool, which still has a majority after one drop and so genuinely re-screens. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: prepost-only screen, floor calibrated on clean pools The screen no longer runs for toggle campaigns. Toggle's contrast is between on- and off-blocks within the test, so a reference change predating the test is common-mode and cancels — which is why the fault bit -0.69 to -1.14 pp in prepost against -0.31 to -0.52 pp in toggle. The screen cannot see one there in either construction: the on/off contrast is blind to it by design, and splitting the test in half leaves ~3 seasonal months a side whose noise swamps the signal (the five-reference injected arm read *below* its own clean baseline). Ruling out a good reference costs more than leaving a mild bad one in, so it does not run. `power_model` is nonetheless still exposed in toggle, and that is left open deliberately: its fit reaches into the pre-campaign baseline under `adaptive_time_decay`, which straddles a reference change at the campaign boundary, reintroducing a bias the toggle contrast would otherwise cancel. That is a fit-window exposure, not a toggle-design one, and closing it re-opens the `toggle_campaign_only` knob Issue 16 pruned — its own decision, documented in the design note to revisit. `screen_floor` is now 0.025, calibrated rather than chosen. Clean placebo pools spread up to 1.19 pp with nothing injected, so 2.5 pp leaves about a factor of two before a healthy reference is at risk. The previous placeholder of 1 pp fired on healthy turbines in three separate places, and on the clean fixture cell that false positive cost 0.88 pp of error — the screen made a good campaign worse. At 2.5 pp the clean cells are unchanged to five decimal places. The five-reference arm now injects -8% rather than -4%: two references moving together drag the pool median toward themselves and roughly halve every deviation the screen sees (3.48 pp for one bad, 1.81 pp for two), so it takes about double the magnitude to clear the same floor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: carry the Greenbyte active-power minimum; add the screen road-test driver `power_model` requires the `active_power_min` role, and the Greenbyte adapter did not carry it, so it could not run on Kelmarsh or Penmanshiel at all. The export publishes `Power, Minimum (kW)` — the adapter was simply dropping it — so this is a column added, not a signal invented. The synthetic zip the tests build now carries it too, and a test asserts the minimum never exceeds the mean. `benchmarking.campaigns.screen_roadtest` runs the shipped configuration with the screen on over placebo campaigns on four reference sets: Hill of Towie whole farm, Hill of Towie west (T01-T15), Kelmarsh and Penmanshiel. Nothing is injected, so every reference the screen rules out is either a genuine find or a false positive, and both are worth seeing before this ships. It reports the test headline, the post-screen reference overall uplift and which references were ruled out. ERA5 comes from the farm's own mean coordinates, so the driver is not Hill of Towie specific. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: do not screen a campaign too short to screen reliably Running the frozen benchmark sweep with the screen on removed references — which it should not have, since nothing is injected there. The removals were not marginal: one read T04 at -5.902%, 8.02 pp from its pool median against a 2.5 pp floor, with T01 at +2.12% and T03 at +4.24%. A 10 pp spread across three references on a placebo. The cause is campaign length, not a bad reference. That case had n_up ~ 4167 records, about 29 days. Measured on clean Hill of Towie pools across four treatment-start windows, the worst deviation by campaign length is: 1 month 2.87 pp 3 months 1.51 pp 2 months 2.47 pp 6 months 1.43 pp 12 months 0.95 pp At 1-2 months the clean spread reaches the floor, so no floor separates a bad reference from a good one; from 3 months it sits at 1.51 pp or below, 1.65x under the floor. So the fix is a minimum-data gate rather than a different floor: `screen_min_campaign_days` defaults to 90, and a shorter campaign is not screened and says so. Counted from records rather than calendar span — a campaign with gaps carries less data than its span suggests, and it is the data the estimate actually has that decides how noisy it is. The screen also now logs the estimate, the deviation and the floor it acted on, because the first sweep gave no way to tell a real find from a false positive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: record the screen's calibration and limits as CF13 Covers what the runs established: T17 is a genuine Hill of Towie bad reference (+4.67% against a 20-reference pool whose median is +0.26%, reproduced on an unrelated pool, and visible directly in the SCADA as a 0.735 -> 0.804 step in its power ratio to its neighbours across 2017/2018); the campaign-length constraint that makes short campaigns unscreenable at any floor; the clean-spread calibration of the floor and the 0.88 pp cost of the false positive the earlier placeholder caused; the halving of signal when two references go bad together; why toggle is not screened and where power_model stays exposed there anyway; the runtime cost; and the three bugs real runs caught that the unit tests did not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: fix LightGBM feature names; make the reference report skippable in sweeps The road test crashed on Kelmarsh and Penmanshiel with "Do not support special JSON characters in feature name". Their source tags carry a comma ("Power, Minimum (kW)") and a degree sign ("Nacelle position (°)"), so `power_model` could not fit on either farm — a portability bug that only a second and third data source could surface. Fixed at the fit boundary rather than in the adapters: `model_safe_features` renames columns positionally for fit and predict, which sidesteps the whole class of rejected characters instead of guessing which ones this LightGBM objects to. Nothing is lost — importances come back positionally and the diagnostics carry the original tags — and the design's rule that source-native names survive verbatim into diagnostics is preserved. `report_reference_uplifts` (default True) turns off the per-reference uplift report. It costs a model fit per reference on every estimate, which is what made the frozen benchmark sweep roughly 12x slower, and a sweep scores estimators rather than reporting campaigns. All four sweep drivers now pass False. It is a report, not an input: a test asserts turning it off does not move the headline, and that the screen still runs and is still reported. The toy screen tests set `screen_min_campaign_days=0`: their campaigns are a fortnight long and they exist to exercise the outlier rule, not the minimum-data gate, which has its own tests at realistic campaign lengths. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: raise the screening gate to 150 days, set by the benchmark sweep At 90 days the sweep still produced false positives on 3-month campaigns: a reference read -2.9%, about 3.1 pp from its pool median against a 2.5 pp floor, and ruling it out made the benchmark *worse* — prepost overall score +0.103 pp and spread +0.126 pp at 3 months. That is the failure the whole design is meant to avoid, since removing a good reference costs more than leaving a mild bad one in. The sweep isolates the driver cleanly. Every other cell was exactly unchanged: 1- and 2-month campaigns (the gate already skipped them), 6- and 12-month campaigns (the screen runs and correctly finds nothing), and every toggle cell (the screen does not run there at all). Only 3 months moved. The same three- reference pool is clean at 6 and 12 months, so campaign length is the driver rather than pool size. 150 days sits clear of both boundaries: a 3-month campaign (~90 days of upgraded data) is not screened, a 6-month one (~182 days) is. My own four-window calibration had put 3 months at 1.51 pp worst; the sweep samples far more windows and found 2.9-3.25 pp, so the sweep is the better calibration and this takes its number rather than mine. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: stop the reference-uplift pass recursing through its own clones Adding `report_reference_uplifts` in 4accc6b changed the guard on the reference pass from `self.reference_screen` to `self.report_reference_uplifts`. The clones that estimate each candidate reference set `reference_screen=False`, which had been stopping the recursion by accident; they did not set the new flag. So every clone launched its own full reference pass, and its clones launched theirs, down to a pool of one. The work is combinatorial rather than linear, and the road test showed it directly: the same 21-turbine case that took 39 minutes before the change was still running after 9 hours, logging a method named `power_model_screen_reference_reference_...` with seventeen nested suffixes. The benchmark sweep was unaffected because it sets the flag to False. Both clones now come from one `_pass_clone` helper that turns off both passes, so the two cannot drift apart again, and a test asserts the invariant on each clone rather than on the symptom. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: record the road test and benchmark verification in CF13 The road test covers four reference sets on placebo campaigns. The floor was calibrated on Hill of Towie alone, and it fires on the one farm with a genuinely bad turbine (T17) while staying silent on three that do not — including Kelmarsh and Penmanshiel, which it had never seen. Reference overall uplift reads -0.26% to +0.39% throughout, so the sanity check behaves everywhere. hot_west is the control: it excludes T17, screens nobody, and gives essentially the same estimate as the whole farm. The frozen benchmarks are unchanged: zero references ruled out, zero MOVED verdicts, every delta zero in both modes at every campaign length. Also records the fourth bug the runs caught — the reference-pass recursion — and why only a large-pool run with reporting on could have found it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: make the screen's remediation configurable for the A/B Three options for a ruled-out reference: "drop" removes it from the features entirely, "direction" keeps its north-calibrated direction (which a performance change does not corrupt) so its wake geometry survives, and "direction_waking" (the default) also gives it a waking boolean in place of the power it loses. Which is best is an empirical question, and F13's lesson is that a physically-compelling feature argument can still lose to the benchmark — so the choice is made against the fixture rather than asserted. A test pins that remediation changes only what happens after detection, never what is detected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: default the screen's remediation to direction-only, per the A/B Movement from each pool's clean cell on the prepost fixture arms [pp], truth zero: remediation +5% -4% 2x-8% mean worst no screen 1.185 1.137 3.198 1.840 3.198 drop 0.376 0.276 1.332 0.661 1.332 direction 0.314 0.288 1.272 0.625 1.272 direction+waking 0.197 0.398 1.408 0.668 1.408 Screening cuts mean error from 1.84 pp to 0.63 pp and recovers up to 3.2 pp on the two-bad case, so the screen clearly earns its place. Keeping a ruled-out reference's direction beats dropping it outright on both mean and worst case, which is the point of power-free rather than removal: its wake geometry is not corrupted by a performance change. The waking boolean does not earn its place. It has the single best result but is worst on the other two arms and has the worst spread. The margin is thin — 0.04 pp separates the three means across three arms on one farm — so this is a weak preference, taken partly because direction-only is also the simpler mechanism. F13's lesson repeating. The other two remediations stay available. This cannot move the frozen benchmarks: they screen nobody, and every arm leaves the clean cells byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: keep direction + waking as the only remediation, prune the alternatives The A/B put direction-only marginally ahead, but the three remediations sit within 0.04 pp of each other (0.625 / 0.661 / 0.668 mean movement) across three arms on one farm and one test turbine. That is not evidence the waking boolean costs anything — it is evidence the fixture cannot separate the three. Against that, waking carries information the fixture does not exercise: a reference that stops producing entirely still tells the model it has stopped waking its neighbours, which neither dropping the turbine nor direction-only can express once the power channel is gone. So it stays, and the two alternatives are pruned rather than left as dead knobs — `screen_remediation` and the `include_waking` parameter are gone, and a ruled-out reference always keeps its direction and gains a waking boolean. The A/B numbers stay recorded in CF13 and the design note, since the reason for keeping a marginally-behind option is exactly the thing a later reader will want. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: address the PR 140 review All nine comments were valid; two were worse than the diff could show. **Power minimum now follows the mean** (comment 1). `active_power_min` is an unconditional model feature, and `ReferenceCpChange` moved the mean while leaving it behind — an impossible channel mismatch the screen could key on instead of the Cp shift, which would have made the detection and A/B results measurements of an artefact. It was documented as a known limitation without following through on what it meant for the conclusions. Positive minima now scale with the mean; negative ones are left alone, since a negative minimum is parasitic draw rather than production and a Cp change does not scale it. **power_free no longer removes availability** (comment 2). It is not a power channel, and the documented contract says only power channels go. Moot under the shipped `availability_feature=False`, but wrong for anyone who enables it. **The gate now uses the worst-covered candidate** (comment 3). It counted the frame's timestamp index, so a 150-day frame in which one candidate had a fortnight of campaign data passed the gate and then estimated that candidate in exactly the short-data regime the gate exists to avoid. **Pass clones write no diagnostics** (comment 4). `out_dir=None` does not suppress diagnostics — it makes a temp directory per run, and there are O(N) runs per estimate. The machine had accumulated 5845 `/tmp/power_model_*` directories totalling 151 MB. `write_diagnostics=False` requires `conditions=()`, since the conditional step writes into the run directory. **The screen calibration is prepost only** (comment 5); its toggle default would have recorded nothing now that toggle is not screened. **The run config records the screen settings** (comment 6), so a run is reproducible in whether screening was on and at what floor and gate. **CF13 corrected** (comments 7-9): the heading claimed 3+ months when the gate is 150 days and a 20-reference pool when it is 19; the 90-day gate was left standing in one section and contradicted in another; and the reference-reporting decision was recorded as unresolved after this PR resolved it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: record that the power-minimum fix leaves the conclusions intact Re-ran the fixture with the minimum scaling consistently. The screened results are identical to three decimal places (+0.197 / +0.398 / +1.408 pp) and the same references are detected; the unscreened bite strengthens about 4% now that the minimum carries the change too. There is a structural reason the screened numbers cannot move: a ruled-out reference loses its power columns, mean and minimum alike, so the consistency of a channel no longer in the feature matrix cannot matter. The artefact could only ever have affected detection, and detection is unchanged — so the R3 results stand on evidence rather than on the assertion that the artefact was harmless. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: state the benchmark result precisely rather than rounding it to zero The earlier claim that "every delta is zero" was true of the overall cells, which were checked, but was written as though it covered every cell. It does not: the largest movement anywhere is 0.013 pp in the degenerate 2-month wind-speed conditional cell, whose own score is 14.5 pp. That residual is deterministic (identical across two sweeps), predates the PR-140 review fixes, and is not the positional feature renaming — checked directly, which gives bit-identical LightGBM predictions. It is consistent with the known thread-order reproduction floor. The headline P50 cells are exactly unchanged; the degenerate conditional tail is not quite, and the finding now says so. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqubHpec4LhzD39d2QUGdc * R3: survive a reference the screen cannot estimate, and mark R3 done Two passes ran a sub-estimate per reference with nothing catching it, so one candidate too degenerate to fit took the whole campaign down -- including reference_uplifts, which runs after the headline is already computed and is documented as never moving it. The screen now reads an unestimatable candidate as NaN, which ranks worst and rules it out; the report records NaN and carries on. worst_outlier gave a non-finite estimate an infinite deviation, which cleared every floor including an infinite one, so screen_calibration's observe-only pass could drop a reference and run a second pass, mixing post-drop deviations into the clean spread the 2.5 pp floor is calibrated from. A non-finite floor now rules nobody out; a NaN estimate is still ruled out at any real floor. Also: reference_uplifts returns its documented columns when it has no rows, screening_timing reads the campaign's own rows like its sibling helpers, summarise drops include_groups (pandas 2.2, against a 2.0 floor), and the two guards _validate_model_config now reaches first are removed rather than left dead behind it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012CtbcQA3mRMixnFXVfuFVE * CI: cache the Zenodo datapack, and survive Zenodo dropping a 1.4 GB download The test job pulls several 1.3-1.5 GB Hill of Towie year zips from Zenodo on every run, and Zenodo drops them: the last four red runs were IncompleteRead, 504 Gateway Timeout, and a BadZipFile that turned out to be a cascade of the first two. Cache the download directory, keyed on the record id, so it stays off the wire after the first green run. Four defects behind the flakiness: - HTTPError was not retryable, so a 504 aborted at the first attempt; 429 and 5xx now retry, 4xx still fails fast. - A stream ending short was accepted, so a truncated zip could be cached. Bytes on disk are now compared against Zenodo's recorded size. - ensure_hot_data_files tested is_file() alone, so a 545 MB partial 2017.zip counted as cached and reached ZipFile -- the BadZipFile. A file shorter than its recorded size now counts as missing and resumes via Range. - raise_for_status() sat outside `with result`, so a 4xx/5xx leaked the streamed socket to GC and tripped filterwarnings=error in an unrelated test as an unraisable ssl.SSLSocket ResourceWarning. Attempts go 5 -> 10 with the backoff capped at 30s, since resume does make progress against Zenodo, just slowly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012CtbcQA3mRMixnFXVfuFVE * R3: address the second Copilot review Screening: a pool with more references the screen cannot estimate than the majority allowance now raises instead of dropping one alphabetically and reporting success with the rest still in the pool. Checked after the drop decision, so an infinite floor still observes without acting. Download cache: a file longer than Zenodo records it is corrupt, not complete -- a resume that re-appended bytes leaves one. The size check is now exact and an oversized file is discarded inside the retry loop, since resuming it would send a Range past the end and earn a 4xx not worth retrying. Screen gate: _campaign_days counted every row carrying a power reading, while a screening fit also drops unavailable and stuck rows. It now counts through the same NormalOperationFilter _select_rows uses, so a turbine with a year of readings but a fortnight of available ones is judged on the fortnight. This moved the wind-speed column guard ahead of the screen -- the gate reads that column now, so a missing one surfaced as a bare KeyError rather than the configuration error it is. Road test: every farm is still attempted and its results still written, but a run with a failed farm now raises rather than exiting clean on a partial table that reads like a completed four-farm validation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012CtbcQA3mRMixnFXVfuFVE --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The committed baseline was recorded on 2026-09-03 at cbe3297, before the reference-validity screen merged, so the file asserted a provenance that no longer described the shipped code. Re-run on v1 with a clean tree so the stamp names a commit a reader can check out. Both modes are neutral against the old baseline: prepost 0 better / 0 worse of 1029 cells and toggle 0 of 1001, on spread, score and |bias| alike. The largest delta in either mean-over-profiles table is 0.001 pp. R3's availability-aware screen gate therefore changes no benchmark cell -- the same 56 long-campaign prepost cells screen as before (the gate skips exactly the 84 one-, two- and three-month cells) and they still rule out no reference. The diff is large and the numbers are not: 3790 of 6300 cells move, none by more than 0.005338 pp, against a 0.1 pp materiality band. That is same-machine LightGBM reduction-order noise, and re-baking it is the price of a stamp that says when the benchmark was last actually reproduced. Sweep: 2h 06m for 280 estimates; prepost 1h 22m, toggle 44m. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012CtbcQA3mRMixnFXVfuFVE
* power_model: take the reference pool from the campaign, not the frame build_reference_features and reference_mean_wind_speed derived the pool as "every turbine present except the test turbine". That agreed with the campaign only because context.select() runs first; select(also=...) would have enrolled co-analysed test turbines as references, and check_reference_only only bars the one test_wtg named. Both now take an explicit references pool. PowerModelMethod._candidate_references derives it from context.references_among(), so n_refs is the declared pool size rather than nunique() - 1. Behaviour is unchanged for every current caller. Also renames a local in estimate() that shadowed the pool with the reference-uplifts frame. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012CtbcQA3mRMixnFXVfuFVE * R4 stage 1: a probe for which missing-data shapes crash and which stay silent An information-gathering driver, not a fixture: 19 arms over the field failure modes (whole-farm outage, whole-turbine outage, single-signal outage, ERA5 columns lost, ERA5 holes), each in the two shapes that take different code paths -- a column absent from the frame, and a column present but NaN. Runs the real campaign path so the shared northing step sees the fault too, and attributes a failure to the deepest frame in this repo's own code, which separates a northing failure from a power_model one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012CtbcQA3mRMixnFXVfuFVE * Report missing data as the missing column, not as somebody else's problem The R4 probe found five missing-column failures, none of which named its cause where the analyst reads it. - era5_direction and northing discovery raised bare pandas KeyErrors; both now raise naming the column and what it is for. - The reference screen reported "a farm-wide problem" when every candidate was unestimatable for one reason: a column that was not there. The cause was logged as a warning while the misleading verdict was what stopped the run. It now records why each candidate failed and appends those causes to its own verdict, chained to the original error. - A missing conditional matching column took the headline down with it, after the fits had already paid for it. Following the era5_exclude idiom, the untouched default matching_vars is now skip-if-missing (warn, drop the conditional breakdown, keep the P50) while an explicitly-set one keeps the strict guard -- and the check runs before the fits, so that raise is immediate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012CtbcQA3mRMixnFXVfuFVE * Say when the reference pool shrinks, and wrap up R4 The pool is derived once from the campaign's candidate references intersected with the turbines the frame carries, and the screen and the reference-uplift report now take that pool instead of the raw declared list. A campaign that declares a reference the delivery never contained previously reached the stuck-sensor filter with an empty frame and died as "IndexError: iloc cannot enlarge its target object"; it is now dropped with a warning naming it and the pool size the estimate actually ran on. The stage-1 probe never saw this because its context is derived from the frame, so declared and present always agreed there. Also threads a seeds parameter through the probe, which is what measured the 0.127 pp estimator-noise floor CF14 reports. Findings in CF14. R4 is marked done with two corrections stated as corrections: it was not power_model-internal (two of five failures were in the shared northing step), and signal discovery was measured and deliberately not built -- every NaN-shaped outage already returned a number, so adapting would have turned honest raises into silent estimates. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012CtbcQA3mRMixnFXVfuFVE * Act on Copilot's review of PR 141: three real defects, all confirmed - The conditional guard sat after screen_references, so an explicitly-named missing matching column was only caught once the screen's clone fits had already run. It is now decided before the screen, judged on the raw ERA5 columns: era5_feature_frame passes every raw column through, and _validate_model_config already refuses an era5_exclude that would remove a matching var, so the two are equivalent. - The pool-shrinkage warning could never fire. context_for intersects the declared candidate references with the turbines present in the frame, so a turbine the delivery lacks never reaches mi.context.candidate_references. The announcement moves to context_for, where the intersection happens and the declared pool is still known. - _campaign_days iterated the declared references and returns 0.0 for one with no rows, so any missing declared reference silently disabled screening even with a full pool of delivered ones. The gate now judges the same pool the rest of the estimate uses. The last two are latent on the runner path today for the same reason the second one describes, but they are real and the fixes are small. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012CtbcQA3mRMixnFXVfuFVE --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
docs/superpowers/ is gitignored by convention; force-added here purely so this spec (with the reference-stability and reference-selection review updates) can be pulled on another machine. To be git rm'd once no longer needed there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v0 is a single-method tool: it measures turbine-upgrade uplift with a binned power-curve, test-vs-reference method. v1 turns wind-up into a platform in which alternative uplift methods are pluggable and objectively benchmarked on synthetic datasets with known ground truth. The driving goals are accurate and precise results from short campaigns and richer conditional information about how an upgrade performs (uplift by wind speed, uplift in wakes vs free-stream, day vs night, by direction/stability, etc.).
Metrics:
Design considerations: