Vector (array) output support for fzr/fzo, plus xpath:// multi-node fix - #75
Merged
Conversation
…node fix fz already lets a model output entry evaluate to a Python list through several extraction forms (python://grep(all=True), csv_file(column=...), hdf5_file(dataset=...), jq:///yq:// filters selecting an array, or a plain shell command printing a JSON array). This change makes that first-class for fzr/fzo, fixes a real bug in the xpath:// form, and documents/tests the whole picture. - fz/outparsers.py: fix evaluate_xpath_output() -- an xpath:// expression matching more than one XML node used to concatenate all matched nodes' text with no reliable separator (xmllint's own behavior), returning one garbled string instead of a vector. It now detects multi-node matches via count(expr) and fetches/casts each node individually, returning a proper list. Single-node and zero-node matches keep their existing scalar behavior (backward compatible). - tests/test_vector_outputs.py: end-to-end coverage of vector outputs across fzo() and fzr(), for every extraction form (bash JSON array, python:// grep(all=True)/csv_file/json_file, jq://, xpath:// including the fix above), including fzr cases with different-length vectors and fzr/fzo coherence for vector-valued columns. Also documents the one asymmetry worth knowing: the legacy plain-shell-command path still simplifies a single-element array to a scalar (existing cast_output() behavior), while python://, jq://, yq:// and xpath:// never do. - examples/vector_outputs_example.md: runnable walk-through (a toy simulation producing a time series, extracted 4 different ways, run across several fzr cases, read back with fzo). Registered in tests/test_examples_scripts.py's markdown-syntax-check list. - Docs: doc/model-definition.md gets a new "Vector / array outputs" subsection (also linked from doc/INDEX.md); README.md's "Output Type Casting" section gets a matching subsection; skills/fz/reference.md's fzo entry gets a short note; NEWS.md gets an "Unreleased" entry. fzd (design of experiments / optimization) still expects a scalar objective per case -- vector-output support there is intentionally left for a follow-up change.
CI on windows-latest (Python 3.10/3.11/3.12) failed:
tests/test_vector_outputs.py::test_fzo_bash_json_array_output_is_a_list
AssertionError: assert False
+ where False = isinstance('[1, 2, 3, 4, 5]', list)
Root cause: the test used the implicit-default ("bash://") output form
with `"echo '[1, 2, 3, 4, 5]'"`. fz's FZ_SHELL_PATH mechanism remaps
common Unix utility names (grep, head, cat, ...) to real executables
bundled with Git for Windows, so those behave like their bash counterparts
even without a full bash invocation -- but apparently no standalone
echo.exe ships there, so "echo" isn't remapped and falls through to
cmd.exe's own builtin echo, which (unlike bash) does not strip single
quotes: the raw stdout is literally `'[1, 2, 3, 4, 5]'` (quotes included),
which cast_output()'s json.loads() rejects, and ast.literal_eval() then
parses as a quoted *string* literal rather than a list -- hence the test
receiving the plain string "[1, 2, 3, 4, 5]" instead of a parsed list.
test_fzo_with_mixed_python_jq_bash_outputs (tests/test_python_outputs.py)
already exercises "bash://grep ... | head -1" successfully on this same
Windows CI job, confirming grep/head genuinely work there; "cat" is in
the same remappable-commands list, so switching the two affected tests
from "echo '[...]'" to writing the JSON to a file and reading it back
with "cat" avoids the echo-specific quoting difference entirely, on any
platform.
Verified locally (12/12 in tests/test_vector_outputs.py); the Windows CI
run will be the real confirmation since the difference is Windows-only
shell behavior this sandbox cannot reproduce directly.
yannrichet-asnr
added a commit
that referenced
this pull request
Jul 22, 2026
…reductions Part 2 of vector-output support (part 1: #75, fzr/fzo). fzd's algorithms (sampling, optimization, ...) always need a single scalar objective per case, but the underlying model output can now be a vector (e.g. a time series, per #75). This change makes output_expression -- the place where that reduction happens -- actually usable for it. - fz/algorithms.py: evaluate_output_expression() gains sum(), len(), sorted(), mean(), median(), stdev(), variance() in its safe eval namespace, alongside the pre-existing math functions and indexing/ slicing (which already worked, e.g. "series[-1]", "max(series)"). Referencing a vector-valued output without reducing it used to fail with a bare "float() argument must be a string or a real number, not 'list'" TypeError; it now raises a ValueError naming the offending output(s) and suggesting a concrete reduction (e.g. "mean(T_series)"). fzd's per-case error handling already treats any evaluate_output_expression failure as a failed point (output value None) rather than aborting the run, so this is a message-quality fix, not a behavior change there. - tests/test_fzd_vector_outputs.py (new, 14 tests): unit coverage of the new reduction helpers (including a realistic "RMS over a vector output" expression) and the improved error message (names the vector-valued key(s), leaves scalar outputs out of the blame list, keeps the old undefined-name error unchanged); plus two end-to-end fzd() tests with a real vector-output model (python://json_file time series) -- one with a correctly reduced output_expression, verified against the manually computed mean for every sampled point, and one with a deliberately unreduced expression, verifying the run completes with every point reported as a failed evaluation rather than crashing. - Docs: doc/core-functions.md and examples/fzd_example.md get a new "Vector-valued outputs as objectives" subsection (with a working RMS example: "sqrt(sum(v**2 for v in T_series) / len(T_series))"); README.md and skills/fz/reference.md get matching notes; doc/INDEX.md links added; NEWS.md entry. fzd's own objective is still a single scalar per case -- this is about reducing a vector-valued model *output* to that scalar, not multi- objective/vector-objective optimization (out of scope here).
yannrichet-asnr
pushed a commit
that referenced
this pull request
Jul 23, 2026
…e algorithm Part 3 of the vector-output series (#75: fzr/fzo vector outputs; #76: reductions of vector outputs to a scalar objective). #76's scope note explicitly leaves multi-objective optimization out; this commit adds it: - fzd()'s output_expression now also accepts a list of expressions; each case yields a list of scalars (one per expression, same order) passed as-is to get_next_design()/get_analysis(). A plain string keeps the legacy single-scalar behavior byte-for-byte unchanged. - New evaluate_output_expressions() (fz/algorithms.py); log formatting, XY DataFrame and Y_<iteration>.csv gain one column per objective; function-model mode supported; errors in any expression fail the whole point (None), consistent with existing per-case error handling. - examples/algorithms/nsga2.py: NSGA-II (Deb 2002) at the fzd plugin format. Batch-parallel generations (population returned as a batch so fzd spreads it across calculators), SBX + polynomial mutation, (mu+ lambda) elitist selection, failed cases treated as dominated. Pareto front written to nsga2_pareto.csv and returned in analysis data. - tests/test_fzd_multiobjective.py (8 tests): unit coverage of evaluate_output_expressions; end-to-end fzd+NSGA-II on the Binh-Korn problem validated against its analytic Pareto front in objective space (normalized deviation < 3%); scalar-expression backward-compat; partial objective failure -> whole point None, run completes. - Docs: doc/core-functions.md 'Multi-objective (vector) objectives' subsection, skills/fz/reference.md note, NEWS.md entry. No regressions: test_fzd.py, test_fzd_vector_outputs.py, test_algorithm_options.py, test_algorithm_plugins.py -> 85 passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
fz already lets a model
outputentry evaluate to a Python list through several extraction forms (python://grep(all=True),csv_file(column=...),hdf5_file(dataset=...),jq:///yq://filters selecting an array, or a plain shell command printing a JSON array). This PR makes that first-class forfzr/fzo, fixes a real bug in thexpath://form, and documents/tests the whole picture.First step of a two-part change (
fzdvector-output / vector-objective support is intentionally left for a follow-up PR, as discussed).Changes
fz/outparsers.py: fixesevaluate_xpath_output()— anxpath://expression matching more than one XML node used to concatenate all matched nodes' text with no reliable separator (this isxmllint's own behavior), returning one garbled string instead of a vector. It now detects multi-node matches viacount(expr)and fetches/casts each node individually, returning a proper list. Single-node and zero-node matches keep their existing scalar behavior (backward compatible, verified against the pre-existing xpath tests).tests/test_vector_outputs.py(new, 12 tests, all passing): end-to-end coverage of vector outputs acrossfzo()andfzr(), for every extraction form (bash JSON array,python://grep(all=True)/csv_file/json_file,jq://,xpath://including the fix above), multi-case aggregation with same- and different-length vectors, andfzr/fzocoherence for vector-valued columns. Also documents the one asymmetry worth knowing: the legacy plain-shell-command path still simplifies a single-element array to a scalar (existingcast_output()behavior), whilepython://,jq://,yq://andxpath://never do.examples/vector_outputs_example.md(new): a runnable, verified walk-through — a toy time-series "simulation", extracted 4 different ways, run across severalfzrcases, read back withfzo. Registered intests/test_examples_scripts.py's markdown-syntax-check list.doc/model-definition.mdgets a new "Vector / array outputs" subsection (linked fromdoc/INDEX.md);README.md's "Output Type Casting" section gets a matching subsection;skills/fz/reference.md'sfzoentry gets a short note;NEWS.mdgets an "Unreleased" entry.Testing
pytest tests/test_vector_outputs.py— 12/12 passed.test_python_outputs.py(31 passed / 5 skipped for optional deps),test_fzo_fzr_coherence.py,test_readme_snippets.py,test_examples_scripts.py,test_skill_static.py, plus a broad ~870-test sweep of the rest of the suite (excluding the SSH/SLURM/example-notebook/Funz-protocol files CI itself excludes).examples/vector_outputs_example.md's code blocks were executed end-to-end (not just syntax-checked) to confirm the printed example values are exactly right.🤖 Generated with Claude in Cowork mode.