Skip to content

contrib: MolecularFullerene - schema-bridge-components - #13

Merged
0liveiraaa merged 2 commits into
0liveiraaa:mainfrom
MolecularFullerene:contrib/molecularfullerene-schema-bridge-components
Aug 17, 2026
Merged

0liveiraaa merged 2 commits into
0liveiraaa:mainfrom
MolecularFullerene:contrib/molecularfullerene-schema-bridge-components

Conversation

@MolecularFullerene

Copy link
Copy Markdown
Contributor

Summary

This contribution adds a preregistered component-level follow-up to the already merged molecularfullerene-dsh-schema-ablation study (PR #7).

It runs the real DSH agent loop with the registered official persistent bash executor held fixed, while crossing only the model-visible bash.description × bash.parameters bundles. The Minimal system, str_replace_editor, tool count/order, reasoning effort, execution barriers, and scorer are fixed.

Main result

The main sample is 10 frozen read-only prompts × 4 fixed runtime identities × 4 arms = 160 first-assistant trajectories, after an independent four-cell protocol pilot:

  • PP: 26/40 minimal-like (65.0%)
  • PO: 21/40 (52.5%)
  • OP: 14/40 (35.0%)
  • OO: 9/40 (22.5%)

Preregistered task-level Latin randomization tests with Holm correction across D/Q/I:

  • description D (one-shot − persistent): −0.300; two-sided p = 0.000110; Holm-adjusted p = 0.000330
  • parameters Q: −0.125; p = 0.159868; Holm-adjusted p = 0.319737
  • interaction I: 0; p = 1

D is the only primary contrast that passes the preregistered multiplicity correction. The Q bootstrap interval is reported as descriptive and is not treated as confirmatory evidence.

All 164 protocol observations were accepted, with zero retries, zero executor dispatches, and zero failed-unit replacements. Tool-name and argument validity are protocol diagnostics only.

Relationship to existing contributions

This does not modify the prior MolecularFullerene contribution. It creates a separate follow-up folder because the repository's incremental CI permits new contribution directories but does not permit modifying an existing one.

It also complements xiaobright-v4-tool-surface-dose-response: that study varies catalog size/composition across Pro and Flash, while this study keeps exactly two tools and a fixed executor, then isolates description and parameter bundles under a preregistered Pro-only factorial design. It does not reproduce or make claims about that contribution's Project2 results.

Included material

  • contribution README with environment, provenance, interpretation, and limitations
  • one concise results report
  • the strict scorer's complete publication-allowlisted JSON, copied byte-for-byte from the canonical research repository

The public-summary file SHA-256 is 77e3f411a584c5ef32e2939e740693388d4c5f9fe33090406daa27f8c9644f88.

Source provenance:

Validation and publication boundary

  • python3 scripts/check_pr.py --base origin/main --head HEAD: 16/16 checks passed
  • git diff --check origin/main...HEAD: passed
  • both repository indexes are append-only (zero deleted lines)
  • no API key, Authorization value, raw reasoning/text, raw tool arguments, request body, real identity/session/request/call identifier or hash, absolute local path, or per-request timestamp is included
  • the deidentified matrix ordinals are reconstructible from the public seed and are explicitly not presented as cryptographic anonymization

Important limitations

The endpoint is a lexical trajectory label, not task ability, answer quality, Project2 performance, an internal persona/route, or evidence of RL overfitting. Only 10 prompt clusters are represented; 160 trajectories are not 160 independent tasks. No model-proposed tool call was executed.

deepseek-v4-pro is a mutable service alias. Account, endpoint cohort, server shard, and time were not randomized. The raw-artifact digest is not a provider signature, and the reported Harness commit does not attest ignored generated lib/ or node_modules bytes.

@MolecularFullerene
MolecularFullerene marked this pull request as ready for review August 17, 2026 05:35
@0liveiraaa
0liveiraaa merged commit 5128d7d into 0liveiraaa:main Aug 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants