Vector (array) output support for fzr/fzo, plus xpath:// multi-node fix

NEWS.md

@@ -42,6 +42,32 @@
 - The `python://` prefix also works with `fzo --output-cmd NAME="python://..."`
   on the CLI (as do `jq://`, `yq://`, `xpath://` and `bash://`).
 
+### Vector (array) output support in fzr/fzo
+
+- Output entries can resolve to a Python list, not just a scalar — a
+  natural fit for time series, per-node profiles, spectra, etc. Supported
+  via `python://grep(..., all=True)`, `csv_file(column=...)`,
+  `hdf5_file(dataset=...)`, `jq://`/`yq://` filters selecting an array,
+  `xpath://` matching several XML nodes, or a plain shell command printing
+  a JSON array. `fzr`/`fzo` store the full list per case unmodified — no
+  flattening, truncation or padding, so cases can have vectors of
+  different lengths.
+- Fixed `xpath://`: an expression matching more than one XML node used to
+  return a single string with the matched nodes' text concatenated by
+  `xmllint` with no reliable separator, instead of a vector. It now
+  returns a list of per-node values (each cast like `grep`'s default);
+  single-node and zero-node matches are unaffected (same scalar behavior
+  as before).
+- New `tests/test_vector_outputs.py` and `examples/vector_outputs_example.md`
+  cover vector outputs end-to-end across `fzo`/`fzr`, all extraction forms,
+  and fzo/fzr coherence. See `doc/model-definition.md` ("output" → "Vector /
+  array outputs") for the full write-up, including the CSV/JSON
+  persistence caveat (`to_csv()` stringifies lists; prefer `--format json`,
+  `to_pickle`, or `to_parquet` for a lossless round trip).
+- `fzd` (design of experiments / optimization) still expects a scalar
+  objective per case; vector-output support there is tracked as a
+  follow-up.
+
 ## Version 1.1 (2026-06-15)
 
 ### CLI argument aliases (README forms now work)

README.md

@@ -1811,6 +1811,36 @@ results = fz.fzo("output_dir", model)
 4. Keep as string
 5. Single-element arrays → scalar
 
+### Vector (array) Outputs
+
+An output entry does not have to be a single number: `python://`, `jq://`,
+`yq://` and plain shell commands can all return a full list (a time series, a
+per-node profile, a spectrum...), which lands in the results DataFrame as one
+Python list per case — `fzr`/`fzo` never flatten, truncate or pad vector
+outputs, and cases are free to produce vectors of different lengths:
+
+```python
+model = {
+    "output": {
+        # whole JSON array file -> plain Python list
+        "T_series": "python://json_file('series.json')",
+        # every regex match -> list
+        "T_series_grep": "python://grep(r'T=(\S+)', 'log.txt', all=True)",
+        # jq/yq filter selecting an array
+        "T_series_jq": "jq://.temperatures results.json",
+        # xpath:// returns a list too when the expression matches more
+        # than one XML node (e.g. several <value> siblings)
+        "T_series_xpath": "xpath://'//value/text()' output.xml",
+    }
+}
+```
+
+Note the single-element-array simplification above (rule 5) only applies to
+the legacy plain-shell-command form; it does not apply to `python://`,
+`jq://`, `yq://` or `xpath://` outputs, so a length-1 vector stays a
+one-element list with those forms. See `examples/vector_outputs_example.md`
+and `doc/model-definition.md` ("output" → "Vector / array outputs") for more.
+
 ### Progress Callbacks
 
 Monitor execution progress in real-time with custom callback functions:
@@ -3234,6 +3264,7 @@ Practical examples in the `examples/` directory:
 - **examples/examples.md** - Overview of all examples
 - **examples/fzd_example.md** - Iterative design of experiments (fzd) examples
 - **examples/dataframe_input.md** - DataFrame input for non-factorial designs
+- **examples/vector_outputs_example.md** - Vector/array-valued outputs with fzr and fzo
 - **examples/algorithm_options_example.md** - Algorithm options format guide
 - **examples/r_interpreter_example.md** - R interpreter setup and usage
 - **examples/shell_path_example.md** - FZ_SHELL_PATH configuration examples

doc/INDEX.md

@@ -95,6 +95,7 @@ Quick reference index for finding specific topics in the FZ context documentatio
 | commentline | model-definition.md | "Model Fields" → "commentline" |
 | interpreter | model-definition.md | "Model Fields" → "interpreter" |
 | output | model-definition.md | "Model Fields" → "output" |
+| Vector / array outputs | model-definition.md | "Model Fields" → "Vector / array outputs" |
 | Complete model examples | model-definition.md | "Complete Examples" |
 | Model aliases | model-definition.md | "Model Aliases" |
 | Output extraction | model-definition.md | "Advanced Output Extraction" |
@@ -260,6 +261,7 @@ Quick keyword search:
 - **Examples**: quick-examples.md
 - **CLI**: quick-examples.md → "CLI Quick Examples"
 - **DataFrame**: core-functions.md → "fzr", "fzo"
+- **Vector / array outputs**: model-definition.md → "Vector / array outputs", examples/vector_outputs_example.md
 - **Interrupt**: parallel-and-caching.md → "Interrupt Handling"
 - **Retry**: parallel-and-caching.md → "Retry Mechanism"
 - **Performance**: parallel-and-caching.md → "Performance Optimization"

doc/model-definition.md

@@ -242,6 +242,65 @@ model = {
 }
 ```
 
+If the expression matches more than one node (e.g.
+`//result/value/text()` against several `<value>` siblings), the result is
+a list of per-node values instead of a single string — see "Vector / array
+outputs" below.
+
+### Vector / array outputs
+
+An `output` entry does not have to resolve to a single value. Any of the
+forms above can return a Python list instead of a scalar, and `fzo`/`fzr`
+store it as-is (one full list per matched case, in one DataFrame cell) —
+this is the natural way to capture a time series, a spatial profile, a
+spectrum, or any other array-shaped simulation result:
+
+```python
+model = {
+    "output": {
+        # Whole JSON array file -> plain Python list
+        "T_series": "python://json_file('series.json')",
+        # Every regex match -> list
+        "T_series_grep": "python://grep(r'T=(\S+)', 'log.txt', all=True)",
+        # A CSV column -> list
+        "T_series_csv": "python://csv_file('data.csv', column='T')",
+        # An HDF5 dataset -> list (requires the optional h5py package)
+        "T_series_h5": "python://hdf5_file('results.h5', 'T')",
+        # jq/yq filter selecting a JSON/YAML array
+        "T_series_jq": "jq://.temperatures results.json",
+        # xpath:// selecting more than one XML node
+        "T_series_xpath": "xpath://'//value/text()' output.xml",
+        # Plain shell command whose stdout is a JSON array
+        "T_series_bash": "cat series.json",
+    }
+}
+```
+
+`fzr` places no constraint on vector length or shape across cases: two
+cases can perfectly well produce vectors of different lengths (e.g. an
+iterative solver that converges after a variable number of steps) — each
+row simply keeps its own list, with no padding or truncation.
+
+**Single-element vectors**: the plain-shell-command form (`bash://` or the
+implicit default) applies `cast_output`'s backward-compatible
+simplification, which unwraps a single-element JSON array to its scalar
+element (`"echo '[42]'"` → `42`, not `[42]`). This can silently turn one
+row of an otherwise vector-valued column into a bare scalar, if that
+particular case happens to produce a length-1 result. The `python://`,
+`jq://`, `yq://` and `xpath://` forms never do this — a length-1 result
+stays a one-element list — so prefer one of those forms whenever a vector
+output's length can legitimately be 1.
+
+**Persisting vector-valued results**: `fzr`/`fzo` results are plain pandas
+DataFrames. `to_dict(orient="records")` / `json.dumps(..., default=str)`
+(and the CLI's `--format json`) round-trip vectors as native JSON arrays.
+`to_csv()` (and `--format csv`) stringifies each list instead (e.g.
+`"[1, 2, 3]"`); reload with `json.loads`/`ast.literal_eval` per cell, or
+prefer `to_pickle`/`to_parquet` for a lossless round trip.
+
+See `examples/vector_outputs_example.md` for a complete, runnable
+walk-through.
+
 ### id (optional)
 
 Unique identifier for the model, useful for documentation and logging.

examples/vector_outputs_example.md

@@ -0,0 +1,149 @@
+# Vector (array) outputs with fzr and fzo
+
+Simulation outputs are not always a single number: a time series, a spatial
+profile, per-node results, a spectrum... fz already lets an `output` entry
+evaluate to a Python **list** instead of a scalar. This example shows the
+supported ways to produce a vector output, and how it flows through `fzr`
+(one full list per case, in one DataFrame cell) and `fzo` (reading the same
+results back).
+
+`fzd` (design of experiments / optimization) still expects a single scalar
+objective per case for now — vector output support there is a separate,
+follow-up piece of work.
+
+## The toy model
+
+A "simulation" that prints an exponential decay time series to `series.json`,
+run for `n_steps` steps starting at `T0`:
+
+```python
+import os
+
+# input.txt and run_case.sh live side by side in the working directory fzr
+# is called from -- the same layout used throughout fz's own examples/tests.
+with open("input.txt", "w") as f:
+    f.write("n_steps=${n_steps}\nT0=${T0}\n")
+
+with open("run_case.sh", "w", newline="\n") as f:
+    f.write("#!/bin/bash\n")
+    f.write("source input.txt\n")
+    f.write(
+        "python3 -c \"\n"
+        "import json\n"
+        "n = int($n_steps)\n"
+        "T0 = float($T0)\n"
+        "series = [round(T0 * (0.9 ** i), 3) for i in range(n)]\n"
+        "print(json.dumps(series))\n"
+        "with open(\'series.txt\', \'w\') as fh:\n"
+        "    fh.write(chr(10).join(str(v) for v in series))\n"
+        "\" > series.json\n"
+    )
+os.chmod("run_case.sh", 0o755)
+```
+
+## Four ways to extract the same vector
+
+```python
+model_python_json = {
+    "varprefix": "$",
+    "delim": "{}",
+    "output": {
+        # Whole JSON array file -> plain Python list, no shell involved
+        "T_series": "python://json_file('series.json')",
+    },
+}
+
+model_python_grep_all = {
+    "varprefix": "$",
+    "delim": "{}",
+    "output": {
+        # One value per line in series.txt -> list, via grep(..., all=True)
+        "T_series": "python://grep(r'(\\S+)', 'series.txt', all=True)",
+    },
+}
+
+model_jq = {
+    "varprefix": "$",
+    "delim": "{}",
+    "output": {
+        # jq filter selecting the whole array (requires the jq executable)
+        "T_series": "jq://. series.json",
+    },
+}
+
+model_bash = {
+    "varprefix": "$",
+    "delim": "{}",
+    "output": {
+        # Legacy shell command: cast_output() parses JSON arrays too
+        "T_series": "cat series.json",
+    },
+}
+```
+
+All four are equivalent for this file, and all four return `T_series` as a
+Python list — pick whichever matches the rest of your output-extraction
+tooling (`csv_file(path, column=...)` and `hdf5_file(path, dataset=...)` work
+the same way for CSV/HDF5 vector results, and `xpath://` returns a list when
+the expression matches more than one XML node).
+
+## Running fzr across several cases
+
+```python
+import fz
+
+result = fz.fzr(
+    "input.txt",
+    {"n_steps": 5, "T0": [50.0, 100.0, 200.0]},   # 3 cases
+    model_python_json,
+    calculators="sh://bash run_case.sh",
+    results_dir="vector_demo_results",
+)
+
+for i in range(len(result)):
+    print(result["T0"].iloc[i], "->", result["T_series"].iloc[i])
+# 50.0  -> [50.0, 45.0, 40.5, 36.45, 32.805]
+# 100.0 -> [100.0, 90.0, 81.0, 72.9, 65.61]
+# 200.0 -> [200.0, 180.0, 162.0, 145.8, 131.22]
+```
+
+Each `T_series` cell holds the *full* list for that case — `fzr` does not
+flatten, truncate or otherwise reshape vector outputs. Cases are free to
+produce vectors of different lengths (e.g. an iterative solver that
+converges after a variable number of steps); each row simply keeps its own
+list, no padding is applied.
+
+## Reading the same results back with fzo
+
+```python
+fzo_result = fz.fzo("vector_demo_results/*", model_python_json)
+
+# fzo() and fzr() may not list cases in the same order (fzo() sorts by
+# matched directory), so compare per case via the T0 column rather than
+# raw row position.
+by_t0 = dict(zip(result["T0"], result["T_series"]))
+fzo_by_t0 = dict(zip(fzo_result["T0"], fzo_result["T_series"]))
+assert by_t0 == fzo_by_t0
+```
+
+## A note on single-element vectors
+
+The legacy plain-shell-command form (no `python://`/`jq://`/`yq://`/`xpath://`
+prefix) applies a backward-compatible simplification: a single-element JSON
+array is unwrapped to its scalar element (`"echo '[42]'"` -> `42`, not
+`[42]`). This can silently turn one row of a vector-output column into a bare
+scalar if that particular case happens to produce a length-1 result. The
+`python://`, `jq://`, `yq://` and `xpath://` forms never do this: a
+single-element result stays a one-element list. Prefer one of those forms
+when a vector output's length can legitimately be 1.
+
+## Persisting results with vectors
+
+`fzr`/`fzo` results are plain pandas DataFrames, so list-valued cells behave
+like any other Python object column:
+
+- `results.to_dict(orient="records")` / `json.dumps(..., default=str)` (or
+  the CLI's `--format json`) round-trip vectors as native JSON arrays.
+- `results.to_csv(...)` (or `--format csv`) stringifies each list
+  (`"[1, 2, 3]"`); reload with `ast.literal_eval` or `json.loads` per cell,
+  or prefer `to_pickle`/`to_parquet` for a lossless round trip.

fz/outparsers.py

@@ -552,7 +552,11 @@ def evaluate_xpath_output(
     Returns:
         The raw text matched by the XPath expression, cast to int/float
         when possible (same casting as :func:`grep`'s default), otherwise
-        returned as a string.
+        returned as a string. If the expression selects a node-set with
+        more than one node (a vector output — e.g. ``//result/value/text()``
+        matching several ``<value>`` elements), a list of per-node values
+        (each cast individually) is returned instead of a single
+        concatenated string.
 
     Raises:
         RuntimeError: If the ``xmllint`` executable is not found on PATH.
@@ -581,10 +585,49 @@ def evaluate_xpath_output(
     if not file_path.is_absolute():
         file_path = out_dir / file_path
 
+    def _run_xpath(one_expr: str) -> _subprocess.CompletedProcess:
+        cmd = ["xmllint", "--xpath", one_expr, str(file_path)]
+        return _subprocess.run(cmd, capture_output=True, text=True)
+
+    # First, find out whether the expression selects a node-set, and if so
+    # how many nodes it matches. xmllint concatenates the serialized text
+    # of every matched node with no separator we can rely on, so a naive
+    # single call cannot distinguish "one node" from "several nodes whose
+    # text happens to contain no digits/whitespace" reliably. Wrapping the
+    # expression in count(...) tells us unambiguously: it succeeds (with an
+    # integer result) only when the expression evaluates to a node-set,
+    # and fails with "Invalid type" when it already returns a scalar
+    # (string/number/boolean, e.g. from count()/sum()/string() themselves).
     log_debug(f"Evaluating xpath output expression: {xpath_expr} on {file_path}")
-    cmd = ["xmllint", "--xpath", xpath_expr, str(file_path)]
-    result = _subprocess.run(cmd, capture_output=True, text=True)
+    count_result = _run_xpath(f"count({xpath_expr})")
+    node_count: Optional[int] = None
+    if count_result.returncode == 0:
+        try:
+            node_count = int(count_result.stdout.strip())
+        except ValueError:
+            node_count = None
+
+    if node_count is not None and node_count > 1:
+        # Vector output: fetch and cast each matched node individually so
+        # embedded whitespace/newlines in node text can never corrupt the
+        # split, unlike naively splitting the concatenated output.
+        values = []
+        for i in range(1, node_count + 1):
+            indexed_result = _run_xpath(f"({xpath_expr})[{i}]")
+            if indexed_result.returncode != 0:
+                cmd = ["xmllint", "--xpath", f"({xpath_expr})[{i}]", str(file_path)]
+                raise _subprocess.CalledProcessError(
+                    indexed_result.returncode, cmd,
+                    output=indexed_result.stdout, stderr=indexed_result.stderr,
+                )
+            values.append(_cast_numeric(indexed_result.stdout.strip()))
+        return values
+
+    # Scalar path (0 or 1 matched node, or an expression that already
+    # evaluates to a scalar) — unchanged, backward-compatible behavior.
+    result = _run_xpath(xpath_expr)
     if result.returncode != 0:
+        cmd = ["xmllint", "--xpath", xpath_expr, str(file_path)]
         raise _subprocess.CalledProcessError(
             result.returncode, cmd, output=result.stdout, stderr=result.stderr,
         )

skills/fz/reference.md

@@ -38,7 +38,14 @@ fz.fzo(output_path: str, model: str | dict) -> pandas.DataFrame
 Runs each `model["output"]` command in every directory matched by `output_path` (plain
 path or glob like `results/*`). Returns one row per directory. Directory names following
 `key=val,key=val` are parsed back into variable columns. Results are auto-cast (int,
-float, list, dict) when possible.
+float, list, dict) when possible. An output entry may resolve to a **list** (vector
+output: time series, per-node profile, ...) via `python://grep(..., all=True)`,
+`csv_file(column=...)`, `hdf5_file(dataset=...)`, `jq://`/`yq://` filters selecting an
+array, `xpath://` matching several nodes, or a plain shell command printing a JSON
+array; `fzr`/`fzo` store the full list per case, unmodified (cases may have
+different-length vectors). Note: the plain-shell-command form still simplifies a
+single-element array to its scalar (`echo '[42]'` → `42`) for backward compatibility;
+the other forms never do.
 
 ### fz.fzr — run a parametric study
 

tests/test_examples_scripts.py

@@ -108,3 +108,6 @@ def test_r_interpreter_example_md_python_blocks_valid_syntax(self):
 
     def test_shell_path_example_md_python_blocks_valid_syntax(self):
         self._check_md_syntax("shell_path_example.md")
+
+    def test_vector_outputs_example_md_python_blocks_valid_syntax(self):
+        self._check_md_syntax("vector_outputs_example.md")