AttackBED: Information-Gain Policies under Strategic Disclosure

English | 简体中文

A fully synthetic, runnable research pilot: what happens when a question changes both the information available now and the willingness to disclose later?

AttackBED is a small Bayesian Experimental Design (BED) environment. It reduces a complex interaction to a hidden target, a disclosure state, and three actions in order to test one intuition: a policy that maximizes immediate information may ignore observation noise or damage the future information channel.

This is not a paper-reproduction repository or a model of real people. It is an executable thought experiment: small enough to inspect, expressive enough to make the mechanism visible.

The idea

Element	Values and role
Hidden target	T ∈ {0, 1, 2}; the policy must identify it
Disclosure state	open, cautious, guarded
Action	direct, indirect, repair
Observation	exact evidence e0/e1/e2, vague v, refusal r
Core dynamic	an action changes both the current emission and the next disclosure state

flowchart LR
    B["Belief over target"] --> P["Policy selects an action"]
    P --> W["Synthetic disclosure world"]
    W --> O["Current observation"]
    W --> S["Next disclosure state"]
    O --> U["Bayesian update"]
    U --> B

Loading

direct can reveal more now but is more likely to push the channel toward guarded. indirect is gentler. repair reveals almost nothing about the target but can restore future disclosure.

The project story in one figure

The figure uses a representative full-grid synthetic pilot. It is not a universal leaderboard; it demonstrates three separable effects:

1. A wrong observation model produces the wrong information gain. As structured decoy noise rises, BED-open falls from 99.4% to 31.8%, while BED-noise-aware remains at 97.8% at the highest-noise point.
2. Modeling noise matters when noise and state dynamics coexist. Across the E5 grid, noise-aware accuracy averages 82.2%, about 19.5 points above the open-assumption policy's 62.8% and close to the two-step oracle's 86.4%.
3. Preserving the channel does not solve inference by itself. Surrogate reduces mean channel loss by about 19%, but accuracy does not improve materially because its observation model still misses the decoy structure.

The central takeaway is simple: a useful policy must reason about both how observations are corrupted and how its actions change future information quality.

The compact data behind the figure is stored in examples/pilot_results.json. It is a showcase snapshot from a synthetic pilot, not a real-world measurement.

Policies

Policy	Core idea
BED-open	Always assumes a clean, open state and maximizes one-step information gain
BED-noise-aware	Knows the synthetic decoy-noise mechanism but not the true state
BED-state	Maintains a belief over the disclosure state
BED-oracle-state	Reads the true simulator state as a diagnostic baseline
BED-full-oracle	Knows both the true state and the noise mechanism
BED-full-oracle-2step	Adds two-step lookahead to the full oracle
Surrogate	Scores information gain - channel damage
Safe heuristic	Uses gentle probing and repairs guarded channels
Random	Uniform random-action baseline

See the policy guide for the assumptions behind each contrast. Oracle variants are diagnostic references, not deployable methods.

Quickstart

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

python run_bench.py \
  --num-episodes 5 \
  --groups E0 \
  --max-workers 2 \
  --log-profile compact

This command was verified with Python 3.9, NumPy 2.0.2, and Matplotlib 3.9.4. It completes 225 synthetic episodes: 9 policies × 5 seeds × 5 episodes.

E0 contains no noise or dynamic channel damage, so most BED variants intentionally coincide. It verifies the execution path; it is not the source of the project results above.

Experiment groups

Group	Mechanism
E0	clean, static baseline
E1	increasing symmetric random noise
E2	increasing strategic decoy noise
E3	actions begin to change future disclosure
E4	dynamic disclosure × symmetric noise
E5	dynamic disclosure × strategic noise

Running without --groups executes the full grid and produces substantially more output than the smoke command.

Outputs

Each run writes to ignored outputs/ and includes the resolved configuration, episode/round JSONL records, seed/config aggregates, and diagnostic figures. compact is intended for quick trials; full retains more counterfactual detail.

Repository map

run_bench.py          # single CLI entry point
sandbox/config.py     # E0–E5 grid
sandbox/world.py      # emissions, noise, and state transitions
sandbox/policies.py   # policies, belief updates, and action scores
sandbox/runner.py     # episodes, parallel execution, aggregation
sandbox/metrics.py    # accuracy, calibration, channel metrics
sandbox/plots.py      # run-level figures
docs/                 # English and Chinese project documentation
examples/             # compact showcase data and smoke summary

Scope

• The environment and data are fully synthetic; they do not represent real users or conversations.
• Parameters are designed to expose a mechanism, not fitted to a population.
• The figure presents one representative pilot, not a paper-level claim.
• Historical large outputs are excluded; the smoke run does not reproduce the full showcase grid.
• This is a research-idea prototype, not a production system or a published Python package.

Read the benchmark design, policy guide, and limitations for details.

License and contact

Licensed under the Apache License 2.0. Use GitHub Issues or the maintainer's GitHub profile for questions.