BioBench Atlas
Benchmarks/VirBench

agentic-eval · legacy · verified 2026-07-21

VirBench

Retrieval benchmark that tests whether scientific agents can answer verified viral-sequence questions by querying NCBI Virus.

VirologyGenomicsBioinformatics
Legacy record: this entry has not yet completed the v1.1 field-level audit. Existing values remain visible but do not carry the stronger audited claim.

Benchmark definition

What is counted

Version
1.0
Total
120 (manually verified queries)
Task formats
database retrieval query
Capabilities
RetrievalTool useData analysis
Modalities
TextDNA or RNA sequenceDatabaseWeb

Tracks and subsets

IDCountBasisPartition?Notes
Pathogens represented
pathogen-set		40pathogensNoCount is not a task partition.

Scientific Task Atlas

Scientific task classification

unclassified for 1.0. No evidence-backed scientific-task mapping is published for this record yet.

This legacy record is intentionally unclassified in the current Scientific Task Atlas.

Evaluation registry

Works and run settings

A setting change—scope, prompt, tools, budget, grader, or repeats—creates a separate run. Charts never cross a comparability group.

Evaluation run

virbench-official-run

From Paving the way for AI agents in biology

virbench-v1-full-ncbi-virusv1.0
Scopefull · n=120
ShotsNot reported
Turnsmulti-turn
System prompt publicNot reported
Reasoning / effortNot reported
BrowserYes
InternetYes
DatabasesNCBI Virus
Code executionNot reported
ContainerNot reported
External toolsscientific-agent retrieval tools
Token budgetNot reported
Time / cost budgetNot reported
TemperatureNot reported
SeedNot reported
RepeatsNot reported
Gradermanual verified-answer comparison · human review: yes
Statisticsmean accuracy
ContaminationNot reported
Metrics, results, and full protocol

Metrics

MetricKind / baselineUnitAggregationThreshold / tolerance
Mean accuracyabsolutepercentproblem-weighted meanNot reported

No numeric result rows are published yet; the verified protocol remains useful.

Evidence

  • VirBench methods and results — supports scope, protocol.tools, protocol.grader, metrics

Evidence and change history

Source locators remain visible; expand an item to inspect the exact Registry fields it supports.

Paving the way for AI agents in biology · VirBench methodology and results sections · Supports 6 fields

Open source →

  • summary
  • task_counts
  • domains
  • capabilities
  • modalities
  • access

View source-level modification history on GitHub →