first commit

This commit is contained in:
2026-06-11 03:33:14 +08:00
commit 5f555bf342
599 changed files with 142347 additions and 0 deletions

View File

@@ -0,0 +1,49 @@
# Test: conflicting reviewers
## Input
```text
Editor decision: Major revision.
Editor:
Please avoid expanding the manuscript substantially; focus on clarifying the central claim and
addressing the reviewers' concerns with existing data where possible.
Reviewer 1:
1. The abstract should make a stronger causal claim that X drives Y.
Reviewer 2:
1. The causal language is not supported by the current observational design and should be softened.
Author notes:
- The study is observational.
- We can soften the abstract and discussion.
- We can add a sentence explaining that the findings support an association, not causality.
```
## Expected behavior
- Assign editor instruction ID `E.1` and address it before reviewer comments.
- Assign reviewer IDs `R1.1` and `R2.1`.
- Detect a conflict between Reviewer 1 and Reviewer 2.
- Prioritize the editor instruction and the evidentiary limit of the observational design.
- Use `SOFTEN_CLAIM` for `R2.1`.
- Use `PARTIAL` or `DISAGREE` for the stronger causal-claim request in `R1.1`, with respectful reasoning.
- Avoid incompatible promises.
- Mark readiness as `draft_with_placeholders` unless exact revised abstract/discussion wording or locations are supplied.
## Forbidden behavior
- Do not promise both stronger causal language and softened causal language.
- Do not ignore the editor instruction.
- Do not claim causality from an observational design.
- Do not accuse either reviewer of being wrong.
- Do not invent revised abstract or discussion line numbers.
## Pass/fail checklist
- [ ] `E.1` appears in the tracker or strategy summary.
- [ ] The conflict is surfaced explicitly.
- [ ] The chosen response is consistent with the observational design.
- [ ] `R1.1` and `R2.1` are both answered.
- [ ] No incompatible manuscript-change promises appear.

View File

@@ -0,0 +1,47 @@
# Test: defensive draft audit
## Input
```text
Mode requested: audit and revise this draft response.
Reviewer 1:
1. The method description is unclear and does not explain how model calibration was performed.
2. The authors should report the software version.
Author draft:
The reviewer clearly misunderstood our method. We already explained the calibration in the paper.
We have revised accordingly. The software version is now included.
Author notes:
- Calibration is described in Methods, but the exact paragraph may not be clear.
- Software version: v2.3.1.
- No line numbers are available yet.
```
## Expected behavior
- Detect task mode as `audit` or `revise`.
- Assign stable IDs `R1.1` and `R1.2`.
- Flag the author draft as defensive and insufficiently traceable.
- Rewrite the misunderstanding sentence as manuscript-clarity framing.
- Treat `R1.1` as `CLARIFY_EXISTING` plus possible `ACCEPT_TEXT`.
- Treat `R1.2` as `ACCEPT_TEXT` with supplied version `v2.3.1`.
- Use section names rather than invented line numbers.
- Mark package readiness as `draft_with_placeholders` or `needs_author_input` until exact Methods location or revised text is supplied.
## Forbidden behavior
- Do not retain "The reviewer clearly misunderstood our method."
- Do not retain bare "We have revised accordingly."
- Do not invent line numbers or a Methods paragraph.
- Do not claim the calibration explanation was already sufficient without clarifying the manuscript.
- Do not remove the supplied software version.
## Pass/fail checklist
- [ ] Defensive language is removed.
- [ ] Each reviewer comment receives its own ID.
- [ ] Revised response includes manuscript-clarity framing.
- [ ] `v2.3.1` is preserved exactly.
- [ ] Missing location details remain visible.

View File

@@ -0,0 +1,43 @@
# Evaluation summary
`nature-response` is evaluated with synthetic Markdown fixtures. These tests are not executable
unit tests; they are behavior contracts for manual and agent review.
## Status rationale
Recommended status: `Beta`.
Rationale:
- The core rules are defined in `SKILL.md` and modular references.
- The skill has synthetic fixtures covering minor revision, major revision with missing evidence,
impossible experiment, defensive draft audit, and conflicting reviewers.
- Each fixture includes expected behavior, forbidden behavior, and pass/fail criteria.
- The examples show expected output shape without using real confidential reviewer comments.
- The skill has not yet been validated on real anonymized revision packages, so `Stable` would be premature.
## Fixture coverage
| Fixture | Coverage | Key failure prevented |
|---|---|---|
| `minor-revision.md` | stable IDs, minor comments, missing citation metadata | fabricated citation or line numbers |
| `major-revision-missing-evidence.md` | validation request, statistical details, missing evidence | invented results or p-values |
| `impossible-experiment.md` | out-of-scope longitudinal evidence | time/funding excuse or fabricated survival data |
| `defensive-draft-audit.md` | hostile draft language, vague compliance | accusatory reviewer language |
| `conflicting-reviewers.md` | editor priority and incompatible reviewer requests | contradictory manuscript promises |
## Manual evaluation checklist
- [x] Every fixture has input, expected behavior, forbidden behavior, and pass/fail checklist.
- [x] No fixture uses real reviewer comments.
- [x] Examples are synthetic and do not contain confidential review content.
- [x] Status remains below `Stable` until real anonymized cases are reviewed.
## Promotion path to Stable
Promote from `Beta` to `Stable` only after:
- at least two real anonymized revision packages are tested with author permission;
- no fabricated actions, line numbers, statistics, or citations are observed;
- Chinese-note workflows produce usable English response drafts and Chinese confirmation notes;
- edge cases such as conflicting reviewers and impossible experiments remain traceable.

View File

@@ -0,0 +1,42 @@
# Test: impossible experiment
## Input
```text
Editor decision: Major revision.
Reviewer 2:
1. Please add 2-year survival outcomes to support the clinical relevance of the biomarker.
Author notes:
- The study is cross-sectional.
- We do not have longitudinal follow-up.
- We can soften the claim and add a limitation in the Discussion.
- We can point to the existing association analysis in Figure 3.
```
## Expected behavior
- Assign stable ID `R2.1`.
- Classify the request as evidence / interpretation plus scope / feasibility.
- Use `PARTIAL` or `OUT_OF_SCOPE` with a high-risk flag, not simple refusal.
- Acknowledge the scientific value of longitudinal survival data.
- Explain that 2-year survival requires longitudinal follow-up beyond the present cross-sectional design.
- Offer the supplied alternative evidence: existing association analysis in `Figure 3`.
- Add a limitation / softened claim action in the Discussion.
## Forbidden behavior
- Do not cite time, money, convenience, or lack of funding as the primary reason.
- Do not say the experiment is impossible without explaining the study-design boundary.
- Do not imply survival data were collected.
- Do not accuse the reviewer of asking for an unreasonable experiment.
- Do not leave the central claim unchanged if the requested evidence is absent.
## Pass/fail checklist
- [ ] The response acknowledges the value of the requested survival evidence.
- [ ] The scope boundary is scientific and design-based.
- [ ] The response includes alternative evidence from `Figure 3`.
- [ ] The manuscript checklist includes claim softening or limitation text.
- [ ] No fabricated survival results appear.

View File

@@ -0,0 +1,40 @@
# Test: major revision with missing evidence
## Input
```text
Editor decision: Major revision.
Reviewer 1:
1. The manuscript requires validation in an independent cohort.
2. The statistical replicate definition is unclear.
Author notes:
- We added validation using dataset GSEXXXX and placed it in new Fig. 5.
- We fixed the statistics description.
- Please write the reply in Nature style.
```
## Expected behavior
- Assign stable IDs: `R1.1`, `R1.2`.
- Classify `R1.1` as major evidence / validation with `ACCEPT_ANALYSIS` or `ACCEPT_EXPERIMENT`, depending on whether dataset validation is presented as analysis or experiment.
- Mention dataset `GSEXXXX` and `Fig. 5` because the author supplied them.
- Flag missing result details for `R1.1`, such as outcome direction, performance/effect summary, sample count if relevant, and manuscript section or line location.
- Classify `R1.2` as statistical / methodological and flag missing exact details.
- Request the statistical test name, replicate unit, sample size or replicate count, correction method when relevant, and Methods location.
## Forbidden behavior
- Do not invent validation results, performance numbers, p-values, confidence intervals, sample sizes, or effect sizes.
- Do not claim "the revised Methods now states" unless revised text or location is supplied.
- Do not treat "We fixed the statistics description" as enough evidence for a final confident response.
- Do not downgrade a major validation request to minor wording.
## Pass/fail checklist
- [ ] Major risks are surfaced in the strategy summary.
- [ ] `GSEXXXX` and `Fig. 5` are preserved exactly.
- [ ] Missing evidence is marked as `AUTHOR_INPUT_NEEDED`.
- [ ] Statistical details are requested explicitly.
- [ ] No fabricated quantitative results or manuscript locations appear.

View File

@@ -0,0 +1,44 @@
# Test: minor revision
## Input
```text
Editor decision: Minor revision.
Reviewer 1:
1. Please define X in the Introduction.
2. Figure 2 legend is unclear.
Reviewer 2:
1. Please cite recent work on Y.
Author notes:
- X means cross-domain calibration.
- We revised the Introduction definition.
- We clarified the Figure 2 legend.
- We know one relevant citation but have not provided DOI or full bibliographic details yet.
```
## Expected behavior
- Assign stable IDs: `R1.1`, `R1.2`, `R2.1`.
- Classify `R1.1` and `R1.2` as minor editorial / presentation comments.
- Classify `R2.1` as citation / positioning with missing citation metadata.
- Draft concise English responses for `R1.1` and `R1.2`.
- Mark `R2.1` as `ADD_CITATION` with `AUTHOR_INPUT_NEEDED` until the citation is verified.
- Use section names when line numbers are absent.
## Forbidden behavior
- Do not invent a citation, DOI, journal, year, or title for work on Y.
- Do not claim exact line numbers.
- Do not answer any comment only with thanks.
- Do not merge the two Reviewer 1 comments into one untraceable response.
## Pass/fail checklist
- [ ] Every reviewer comment receives an ID.
- [ ] Every ID appears in the tracker and the draft letter.
- [ ] Citation metadata is requested or placeholder-flagged.
- [ ] Responses are concise and non-defensive.
- [ ] No fabricated line numbers or citation details appear.

View File

@@ -0,0 +1,84 @@
# nature-response test rubric
Use this rubric to manually evaluate `nature-response` outputs against the Markdown fixtures.
## Completeness
Pass when:
- Every reviewer comment receives a stable ID.
- Every ID appears in the tracker and response letter.
- Repeated concerns are cross-referenced rather than ignored.
- Ambiguous reviewer boundaries are flagged.
Fail when:
- A comment is skipped.
- Two concerns are merged without traceability.
- A major concern receives only a polite acknowledgement.
## Traceability
Pass when:
- Every claimed manuscript change has a section, page, line, figure, table, supplement, or explicit placeholder.
- New analyses, experiments, figures, citations, and limitations are mapped to action labels.
- Missing locations are flagged rather than invented.
Fail when:
- The response claims a change without location or evidence.
- The response invents line numbers, figure panels, supplementary items, or citation metadata.
## Factuality
Pass when:
- Missing evidence is marked `AUTHOR_INPUT_NEEDED`.
- Quantitative details are used only when supplied by the author.
- Reviewer wording is preserved unless the user asks for anonymization or summarization.
Fail when:
- The response invents data, p-values, confidence intervals, sample sizes, accession details, reviewer identities, or editor instructions.
- The response overstates unsupported causal or clinical claims.
## Tone
Pass when:
- The response is cooperative, concise, and evidence-forward.
- Disagreement is respectful and scientifically justified.
- Reviewer misunderstanding is framed as manuscript clarification when appropriate.
Fail when:
- The response accuses the reviewer of error, incompetence, or misunderstanding.
- The response is excessively apologetic, defensive, or repetitive.
- The response uses time, money, or convenience as the primary reason for not doing requested work.
## Actionability
Pass when:
- The author can see what to change in the manuscript.
- Missing information is listed as concrete author questions.
- Blocking or high-risk issues are visible before the draft letter.
Fail when:
- The output only produces prose and no action checklist.
- The author cannot identify what evidence is still needed.
## Nature-fit
Pass when:
- The output is organized as editor-readable point-by-point response material.
- All referee criticisms are seriously addressed, justified, or flagged.
- The response letter could be audited if it became part of transparent peer review.
Fail when:
- The output reads like generic language polishing.
- The response hides limitations or makes compliance appear stronger than the evidence provided.