official-release · benchmark creator

BioSecBench-Surveillance repository result snapshot

LatchBio · 2026-07-09

Relationship layer

Benchmark usage

This table records what the work did with each benchmark before attempting to normalize a run. Partial claims remain visible without being treated as comparable evaluations.

evaluation

biosecbench-8d53fd8-claude-code-use

Benchmark: BioSecBench-Surveillance · version initial-release

Selection: not applicable · All 100 registered evaluations
Models: Opus 4.6, Opus 4.7, Opus 4.8, Sonnet 4.6
Metrics: endpoint pass rate
Linked runs: biosecbench-8d53fd8-claude-code

Official commit-pinned Claude Code evaluation result snapshot.

Evidence

repository-path: METHODS.md and results/config_results.csv at commit 8d53fd8517cc74202eb18b618e8b39b4ffaf0c87 (Full-scope Claude Code evaluation relation, models, metric, and linked run.)
Supports: /benchmark_version, /relation_type, /status, /model_ids, /scope, /metric_labels, /evaluation_run_ids, /notes

evaluation

biosecbench-8d53fd8-openai-codex-use

Benchmark: BioSecBench-Surveillance · version initial-release

Selection: not applicable · All 100 registered evaluations
Models: GPT-5.4, GPT-5.5
Metrics: endpoint pass rate
Linked runs: biosecbench-8d53fd8-openai-codex

Official commit-pinned OpenAI Codex evaluation result snapshot.

Evidence

repository-path: METHODS.md and results/config_results.csv at commit 8d53fd8517cc74202eb18b618e8b39b4ffaf0c87 (Full-scope OpenAI Codex evaluation relation, models, metric, and linked run.)
Supports: /benchmark_version, /relation_type, /status, /model_ids, /scope, /metric_labels, /evaluation_run_ids, /notes

evaluation

biosecbench-8d53fd8-pi-use

Benchmark: BioSecBench-Surveillance · version initial-release

Selection: not applicable · All 100 registered evaluations
Models: Opus 4.6, Opus 4.7, Opus 4.8, Sonnet 4.6, gemini-3.1-pro-preview, Gemini 3.5 Flash, GPT-5.4, GPT-5.5, grok-4.20-beta-0309-reasoning, Grok 4.3
Metrics: endpoint pass rate
Linked runs: biosecbench-8d53fd8-pi

Official commit-pinned PI evaluation result snapshot.

Evidence

repository-path: METHODS.md and results/config_results.csv at commit 8d53fd8517cc74202eb18b618e8b39b4ffaf0c87 (Full-scope PI evaluation relation, models, metric, and linked run.)
Supports: /benchmark_version, /relation_type, /status, /model_ids, /scope, /metric_labels, /evaluation_run_ids, /notes

Normalized evaluation runs

BioSecBench-Surveillance3 runs

Open benchmark record →

Evaluation run

biosecbench-8d53fd8-claude-code

From BioSecBench-Surveillance repository result snapshot

Evaluated models / systems: Opus 4.6, Opus 4.7, Opus 4.8, Sonnet 4.6

Scopefull · n=100

ShotsNot reported

TurnsNot reported

System prompt publicNot reported

Reasoning / effortNot reported

BrowserNot reported

InternetYes

Databasesresistance databases, virulence databases

Code executionNot reported

Containeridentical containerized sandbox

External toolsClaude Code, assemblers, aligners, taxonomic classifiers, variant callers

Token budgetNot reported

Time / cost budgetNot reported

TemperatureNot reported

SeedNot reported

Repeats3

Graderdeterministic typed-field checks · human review: not reported

Statistics95% Student-t confidence interval over evaluations

ContaminationNot reported

Metrics, results, and full protocol

Metrics

Metric	Kind / baseline	Unit	Aggregation	Threshold / tolerance
endpoint pass rate	absolute	%	mean of per-evaluation pass rates; evaluations with no gradeable run are dropped	Not reported

Results

Model	Metric	Value	n
Opus 4.6	endpoint pass rate	46.7 % Claude Code; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	80
Opus 4.7	endpoint pass rate	44.2 % Claude Code; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	77
Sonnet 4.6	endpoint pass rate	43.6 % Claude Code; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	83
Opus 4.8	endpoint pass rate	39.5 % Claude Code; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	76

Evidence

repository-path: METHODS.md at commit 8d53fd8517cc74202eb18b618e8b39b4ffaf0c87 (Full 100-evaluation scope, Claude Code harness, tools, three repeats, deterministic grading, aggregation, and confidence intervals.) — supports /benchmark_version, /scope, /protocol, /metrics
repository-path: results/config_results.csv; harness=claude-code at commit 8d53fd8517cc74202eb18b618e8b39b4ffaf0c87 (Exact model labels, endpoint pass rates, 95% confidence intervals, and gradable evaluation counts.) — supports /results

Evaluation run

biosecbench-8d53fd8-openai-codex

From BioSecBench-Surveillance repository result snapshot

Evaluated models / systems: GPT-5.4, GPT-5.5

Scopefull · n=100

ShotsNot reported

TurnsNot reported

System prompt publicNot reported

Reasoning / effortNot reported

BrowserNot reported

InternetYes

Databasesresistance databases, virulence databases

Code executionNot reported

Containeridentical containerized sandbox

External toolsOpenAI Codex, assemblers, aligners, taxonomic classifiers, variant callers

Token budgetNot reported

Time / cost budgetNot reported

TemperatureNot reported

SeedNot reported

Repeats3

Graderdeterministic typed-field checks · human review: not reported

Statistics95% Student-t confidence interval over evaluations

ContaminationNot reported

Metrics, results, and full protocol

Metrics

Metric	Kind / baseline	Unit	Aggregation	Threshold / tolerance
endpoint pass rate	absolute	%	mean of per-evaluation pass rates; evaluations with no gradeable run are dropped	Not reported

Results

Model	Metric	Value	n
GPT-5.5	endpoint pass rate	50.2 % OpenAI Codex; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	93
GPT-5.4	endpoint pass rate	41.3 % OpenAI Codex; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	92

Evidence

repository-path: METHODS.md at commit 8d53fd8517cc74202eb18b618e8b39b4ffaf0c87 (Full 100-evaluation scope, OpenAI Codex harness, tools, three repeats, deterministic grading, aggregation, and confidence intervals.) — supports /benchmark_version, /scope, /protocol, /metrics
repository-path: results/config_results.csv; harness=openai-codex at commit 8d53fd8517cc74202eb18b618e8b39b4ffaf0c87 (Exact model labels, endpoint pass rates, 95% confidence intervals, and gradable evaluation counts.) — supports /results

Evaluation run

biosecbench-8d53fd8-pi

From BioSecBench-Surveillance repository result snapshot

Evaluated models / systems: Opus 4.6, Opus 4.7, Opus 4.8, Sonnet 4.6, gemini-3.1-pro-preview, Gemini 3.5 Flash, GPT-5.4, GPT-5.5, grok-4.20-beta-0309-reasoning, Grok 4.3

Scopefull · n=100

ShotsNot reported

TurnsNot reported

System prompt publicNot reported

Reasoning / effortNot reported

BrowserNot reported

InternetYes

Databasesresistance databases, virulence databases

Code executionNot reported

Containeridentical containerized sandbox

External toolsPI, assemblers, aligners, taxonomic classifiers, variant callers

Token budgetNot reported

Time / cost budgetNot reported

TemperatureNot reported

SeedNot reported

Repeats3

Graderdeterministic typed-field checks · human review: not reported

Statistics95% Student-t confidence interval over evaluations

ContaminationNot reported

Metrics, results, and full protocol

Metrics

Metric	Kind / baseline	Unit	Aggregation	Threshold / tolerance
endpoint pass rate	absolute	%	mean of per-evaluation pass rates; evaluations with no gradeable run are dropped	Not reported

Results

Model	Metric	Value	n
Opus 4.8	endpoint pass rate	50.2 % PI; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	83
GPT-5.5	endpoint pass rate	44.8 % PI; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	77
Opus 4.7	endpoint pass rate	49.6 % PI; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	78
Sonnet 4.6	endpoint pass rate	48.6 % PI; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	83
Gemini 3.5 Flash	endpoint pass rate	47.1 % PI; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	92
Opus 4.6	endpoint pass rate	45.7 % PI; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	81
gemini-3.1-pro-preview	endpoint pass rate	45.3 % PI; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	95
GPT-5.4	endpoint pass rate	38.5 % PI; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	81
Grok 4.3	endpoint pass rate	16 % PI; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	100
grok-4.20-beta-0309-reasoning	endpoint pass rate	13.7 % PI; n is the number of gradable evaluations after evaluations with no gradable run were dropped; the configured benchmark scope was all 100 evaluations.	100

Evidence

repository-path: METHODS.md at commit 8d53fd8517cc74202eb18b618e8b39b4ffaf0c87 (Full 100-evaluation scope, PI harness, tools, three repeats, deterministic grading, aggregation, and confidence intervals.) — supports /benchmark_version, /scope, /protocol, /metrics
repository-path: results/config_results.csv; harness=pi at commit 8d53fd8517cc74202eb18b618e8b39b4ffaf0c87 (Exact model labels, endpoint pass rates, 95% confidence intervals, and gradable evaluation counts.) — supports /results

Source record

Status: published
Published: 2026-07-09
Authors: LatchBio
DOI: Not reported
arXiv: Not reported
Current source version: biosecbench-surveillance-repository-result-snapshot-8d53fd8
Source access: open-url
Content SHA256: 27ef42fc30608b4f73085cbcc355f6e631e4cc0d8bdf1195d9c5558a174fa8e6
Retrieved: 2026-08-11T09:29:54Z

Review provenance

AI-assisted owner-reviewed

Method: Local Codex double-pass
Surface: local-codex-cli
Pipeline: 1.4.0
Prompt: paper-evidence-local-v17
Extractor: Not reported
requested gpt-5.6-sol
Verifier: Not reported
requested gpt-5.6-sol
Model resolution: not-reported
Codex CLI: codex-cli 0.147.0-alpha.6.5
Local run: 71a96e8c-edfd-4bb0-a0f7-4197d05b5342
Generated: 2026-08-11T09:29:54Z

“Owner-reviewed” means this production record passed the required exact-head-SHA approval gate; deterministic code—not the model—authored Registry YAML.

Source versions

Repository snapshot 8d53fd8
biosecbench-surveillance-repository-result-snapshot-8d53fd8 · current · 2026-07-09

Open canonical source ↗