Contribute evidence

A registry entry begins with a primary source.

Propose a benchmark, add an official evaluation, correct a result, or report a broken source through structured GitHub forms.

What happens to a submitted paper

  1. DOI, preprint ID, canonical URL, title fingerprint, open candidate issues, and recent paper-intake PRs are checked for duplication.
  2. The source is MIME-checked, capped at 45 MiB / 150 pages, fingerprinted, and read only from an ignored local temporary directory.
  3. After the owner explicitly selects the paper, a fresh read-only Codex session separates evaluation, training/fine-tuning, validation, model selection, external-result summary, and pure background citations.
  4. A second fresh Codex session independently rechecks every claim and locator against the original source. Disagreement never becomes a normal field.
  5. Deterministic code—not either session—assigns IDs and creates Work, BenchmarkUse, Model, and sufficiently complete EvaluationRun records.
  6. A Ready PR runs normal CI and can merge only after wang422003 comments /approve-paper-intake <full-current-head-sha>.

Pull request checklist

Review policy

All production changes use pull requests. Paper-intake PRs additionally require the paper-owner-gate: only an exact-SHA approval comment by wang422003 passes, and any later push invalidates it. Local Codex never auto-merges. Candidate issues, source files, evidence excerpts, session transcripts, and model drafts are excluded from public exports.