Skip to content

Bind reader-appropriateness to scoring, revision and delivery (#592) - #609

Merged
Stahl-G merged 3 commits into
mainfrom
codex/issue-592-delivery-consistency
Sep 11, 2026
Merged

Bind reader-appropriateness to scoring, revision and delivery (#592)#609
Stahl-G merged 3 commits into
mainfrom
codex/issue-592-delivery-consistency

Conversation

@Stahl-G

@Stahl-G Stahl-G commented Sep 10, 2026

Copy link
Copy Markdown
Owner

First, contained slice of the #592 plan (private_planning/issue-592-delivery-consistency-plan-2026-09-10.md). It makes the writing-noise problem observable to users without adding roles or a new revision loop.

What changed

  • Gate decoupled from input field. release.decision decides soft/hard from the saved reader-contract clause type (requirement_severity), so the same sentence typed into the objective or into writing preferences cannot flip a formal-delivery gate.
  • Score-to-revision connected. Expression <=2 is must_fix; it triggers the existing single auto-revision even when the overall verdict would look passing. A must-fix body marked 达到要求 is recorded as assessment_inconsistent and shown in the assessment panel ("表达分偏低但总体仍判为达到要求,两者不一致…"), not only left in the events table. Expression anchors 4/3/2/1 are in the evaluator rules.
  • One effective rule set. report_profiles keeps only the industry increment; shared body/record, accuracy and evaluation rules stay in deliverable_spec.
  • Handoff. generation_prompt writes scout-contract.md and hands every Scout its absolute path plus the saved plan.json reader_contract through the real dispatch; the scout contract text points at plan.json instead of implying it is self-contained. build_packet refuses to drop a saved reader contract.
  • Gaps carry impact. Delivery-affecting gaps record related/impact/action/status; execution detail stays out of the body.

Tests

  • Behaviour checks: gate is independent of the input field; must_fix/overall_inconsistent thresholds; generation_prompt writes the scout contract and wires it into the dispatch.
  • pytest: 183 passed, 3 subtests with node_modules present. Without it, test_rich_document cannot import @tiptap/core (environmental, unrelated to this diff).

Not in this PR

Ref #592.

yihong guo added 3 commits September 10, 2026 19:01
- Formal-delivery soft/hard follows the saved reader-contract clause type,
  not which input field a requirement came from (#592).
- Expression <=2 is must-fix: it triggers the existing single revision and
  flags a self-contradicting "达到要求" verdict instead of passing silently.
- report_profiles keeps only the industry increment; shared reader and
  evaluation rules stay in deliverable_spec.
- The scout contract is delivered through the actual dispatch, and building a
  review packet that drops a saved contract now fails loudly.
- Delivery-affecting gaps carry related/impact/action/status; execution detail
  stays out of the body.
- Show the must-fix/overall contradiction in the assessment panel (expression
  <=2 while overall is 达到要求) instead of leaving it only in the events table.
- Scout now gets the saved plan.json reader_contract in the dispatch, and the
  scout contract text points at that file rather than implying it is self-contained.
- Add a behaviour test that generation_prompt writes scout-contract.md and wires
  both its absolute path and plan.json into the dispatch.
- Use SOFT_CONTRACT_KINDS in requirement_severity instead of a dead constant.
…ery-consistency

# Conflicts:
#	src/briefloop/static/app.js
@cursor

cursor Bot commented Sep 10, 2026

Copy link
Copy Markdown

Review

上一轮要的三件事都做了:表达分与总体矛盾进评分面板、Scout 交接有行为测试、requirement_severitySOFT_CONTRACT_KINDS。门不再看输入框、表达 ≤2 会走现有一次修订、行业增量与共用规则拆开,方向对。CI 绿。可以合;下面几处是切片里还虚的地方,不否定 #592 这一刀。

已对齐

  • 正式交付 soft/hard 跟已保存条款类型走;无合同时才回退 kind == 'writing'
  • must_fix(表达 ≤2)触发 auto_revise达到要求 + 表达 ≤2 记 assessment_inconsistent,面板有说明,不替模型改分。
  • generation_prompt 写出 scout-contract.md,payload 带绝对路径,提示里点名 plan.json 的 reader_contract。
  • 缺口四字段写在共用 instructions 里,不改数据形状(和「Not in this PR」一致)。

建议收紧

1. build_packet 的约定丢失检查在生产路径上基本走不到。 _snapshot 已经是 resolve(requirements, reader_contract=detail.get('reader_contract')),再比较 detail.reader_contractrequirements.reader_contract 等于用同一来源和自己比。以后真的漏拷贝,也要先改 _snapshot 才会碰这行。要么在测试里塞一份被改过的 snapshot 证明它会响,要么断言「detail 有约定则 spec 必须带上」,不要只做恒真比较。

2. 门控测试没有覆盖「同一个句子换输入栏」。 test_writing_constraint_gate_ignores_which_input_field_it_came_from 只有一条 objective,把整段(含「说明客户交付变化」)标成 writing_preferencereader_content。soft 用例等于把必答内容和写作约束绑在同一个 id 上再整段变软——这是 Orchestrator 漏拆时的失败态,不是「写进 writing_preferences 栏」的成功态。真正的栏位无关应是:内容句在 objective、写作句在 writing_preferences(或都在 objective 但拆成两条 clause),未覆盖写作只出 notice、未覆盖内容仍 blocker。

3. Scout 交接仍靠提示词。 文件写了、prompt 里有路径,测试也只断言这两点。scout_slots 没有结构化带上 scout_contract_path / plan.json。模型漏贴路径时,Python 不会知道。本切片可以先这样;下一刀最好把路径写进 slot,而不是只写进 Orchestrator 说明。

小点

  • 评分面板用 Number(d.expression)<=2expressionnullNumber(null)===0,会误出「表达分偏低但总体达到要求」。Python 的 must_fix 要求 isinstance(score, int),两边不一致。用 Number.isInteger(d.expression) && d.expression<=2
  • 行业提示里删掉的「不为披露缺口加分」「正式交付资格由独立检查决定」在 deliverable_spec 里只有部分对应;若还靠行业增量承担,确认一下不要只靠共用读者规则兜。

结论: 作为 #592 第一刀可以合。优先补一门真正换栏位的测试,以及别让 Number(null) 在面板上误报。

Comment thread tests/test_release.py
'kind': kind, 'source_quote': quote, 'instruction': quote}]})


def test_writing_constraint_gate_ignores_which_input_field_it_came_from():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

只有一条 objective,把整段标成 writing_preference / reader_content。soft 用例等于内容句和写作句绑在同一个 id 上再整段变软,测的是漏拆,不是「同一句换输入栏」。应覆盖:内容在 objective、写作在 writing_preferences(或同栏但两条 clause),未覆盖写作只 notice、未覆盖内容仍 blocker。

Comment thread src/briefloop/review.py
# A saved reader contract must survive into the packet. Losing it silently would
# let the Reviewer check content without the user's own delivery interpretation.
saved_contract=snapshot['detail'].get('reader_contract')
if saved_contract and snapshot['requirements'].get('reader_contract')!=saved_contract:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_snapshot 已经用 detail.reader_contract 去做 resolve,这里再和自己比,生产路径上几乎不会响。要证明「丢掉就失败」,测试里需要一份被改过的 snapshot,或断言 detail 有约定则 spec 必须带上。

Comment thread frontend/app.js
@@ -126,7 +126,7 @@ for(const id of ['download','download-docx','download-bundle']){
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Number(null)===0,缺 expression 且 overall 为「达到要求」时会误出这段。Python must_fix 要求整数分。用 Number.isInteger(d.expression) && d.expression<=2

@Stahl-G
Stahl-G marked this pull request as ready for review September 11, 2026 00:41
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@Stahl-G
Stahl-G merged commit 923e5e3 into main Sep 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant