Skip to content

codex/budget accounting - #200

Merged
merlerm merged 1 commit into
mainfrom
codex/budget-accounting
Sep 19, 2026
Merged

merlerm merged 1 commit into
mainfrom
codex/budget-accounting

Conversation

@merlerm

@merlerm merlerm commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Meter Codex from native per-response token_usage_record usage, deduplicated per
experiment across root/subagent sessions and continuations. Resettable UI totals
and turn-scoped stdout usage are not authoritative for a finite budget.

  • Preserve cumulative accounting; fail closed on missing/invalid/mutated history,
    unexpected models, status-write failures, or no timely initial usage.
  • Stop the target invocation's process tree at budget, including detached children,
    without signaling parallel experiment siblings. Evaluate the last saved policy.
  • Resume unconfirmed exits with only the remainder; do not resume with <= $0.50.
  • Publish atomic experiment-local budget status and advisory $2/$1 warnings.
    Warnings prioritize targeted finishing work, not immediate final answers.
  • Keep confidence honest; unchanged code may reflect useful validation. No
    unchanged-file heuristic or artificial low-confidence retry cap.

Validation

  • First PR layer: 115 focused tests passed, including synthetic strict-Apptainer
    cutoff and real process-tree/parallel isolation. No paid calls in CI tests.
  • Full $20 strict Kinematic2D run: independently verified $19.522296, four useful
    continuations, stopped with $0.477704 left, complete 100-episode evaluation.
  • A prior $5 smoke exposed contradictory wrap-up instructions; corrected smoke
    spent $4.7431024 with two productive continuations rather than empty restarts.
  • Source typing and import-order checks. Repository .pylintrc has legacy options
    rejected by the installed Pylint; focused code lint passed with only those
    configuration diagnostics disabled. GitHub CI remains the authoritative check.

Limits / follow-up

Costs are configured API-equivalent estimates including caching, not subscription
billing. In-flight usage can overshoot before the next completed response record.
The paid validation runs exercised remainder stopping, not the live hard-kill path;
that path has deterministic native-record/container coverage.

Status-file warnings are advisory, not guaranteed live steering. App Server steering
is deferred. Session/status files are not a security boundary against a deliberately
malicious agent. Existing retry telemetry retains some final-attempt fields and
separate prior-attempt totals; use native logs for full-run timing/token analysis.

The next stacked PR adds end-to-end fake-CLI coverage and an opt-in standalone
integration_tests/check_codex_budget.py live diagnostic. No environment changes,
model/pricing changes, credentials, private seeds, or experiment artifacts included.

@merlerm
merlerm marked this pull request as ready for review September 8, 2026 13:54
@merlerm
merlerm requested a review from Jaraxxus-Me September 8, 2026 13:55
@merlerm
merlerm force-pushed the codex/budget-accounting branch from c162a3f to dfd124b Compare September 9, 2026 13:32
Jaraxxus-Me pushed a commit that referenced this pull request Sep 12, 2026
…#216)

Moves the pin from 7e4d422 (v0.2.4-8) to upstream main at 5b2dbac, which
carries the dynamic3D fixes merged since 2026-09-05:

- Dynamo3D: task-owned room so the goal stays reachable with the background
  on (#202), stable chair spawns with real footprints and a 1.5x layout
  (#199), task cameras framing the full layout (#184).
- Tossing3D: colliding room walls and an anchored barrier (#198), receivers
  farther from the barrier (#191), bin-relative goals for the two-cube task
  (#187), the 18-dimensional action space with arm velocity targets declared
  (#200).
- ScoopPour3D: transfer scored into the target bin (#186), 20/30/40/50-cube
  variants (#194).
- BalanceBeam3D balance measured against world horizontal (#189), Rearrange3D
  goal heights (#188), sorting bin goals attached to receptacles (#185),
  Shelf3D goals and intermediate counts (#180, #181), ConstrainedCupboard3D
  one-rod goal (#183), drawer positions restored in state (#190), faster
  shared MuJoCo stepping (#193), limb range-of-motion model (#175, #176,
  #177), DynPushT2D goal poses kept out of walls (#178).

Eval seeds for every dynamic3D family change with this pin; Dynamo3D and
Tossing3D results recorded before it are not comparable to new runs.
@merlerm
merlerm requested a review from tomsilver September 18, 2026 15:29
@merlerm
merlerm force-pushed the codex/budget-accounting branch from dfd124b to f6560bb Compare September 19, 2026 10:40
@merlerm
merlerm merged commit 6af4465 into main Sep 19, 2026
4 checks passed
@merlerm
merlerm deleted the codex/budget-accounting branch September 19, 2026 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants