Benchmarks/Biology-Instructions

suite · audited-with-caveats · verified 2026-07-21

Biology-Instructions

A multi-omics sequence-instruction suite with 21 formal predictive tasks across DNA, RNA, protein, and multi-molecule inputs; the final paper reports task-specific held-out evaluations rather than a single aggregate score.

+20 more

Audited with caveats: 2 field(s) are marked provisional or conflicted. Warnings are shown next to affected values and these claims are excluded from unqualified comparisons.

Task and release audit: the final suite has 21 formal tasks—6 DNA, 6 RNA, 5 protein, and 4 multi-molecule—not 21 questions. Table 2's task rows sum to 243,227 test examples although its printed total is 244,681. The pinned Stage-3 workbook has 8,002 rows versus the paper's 8,000-example description, and no dataset/code license is reported.

Benchmark definition

What is counted

Version: emnlp-2025
Total: 21 (formal evaluation tasks in the final creator paper; sample counts are tracked on child records and never added to this task count)
Task formats: sequence-conditioned classification and regression instructions
Capabilities: PredictionClassificationRegressionScientific reasoning
Modalities: TextDNA or RNA sequenceProtein sequence

Version history

Version	Status	Release / as-of	Total	Formal tracks
preprint-v1 `bioinstruction-preprint-v1`	superseded	2024-12-26	21 (formal evaluation tasks in the final creator paper; sample counts are tracked on child records and never added to this task count)	bioinstruction-emp, bioinstruction-ea, bioinstruction-pd300, bioinstruction-cpd, bioinstruction-tb-human, bioinstruction-tb-mouse, bioinstruction-apa, bioinstruction-ncrna, bioinstruction-modification, bioinstruction-mrl, bioinstruction-prs, bioinstruction-crispr-on-target, bioinstruction-ec, bioinstruction-stability, bioinstruction-fluorescence, bioinstruction-solubility, bioinstruction-thermostability, bioinstruction-aan, bioinstruction-rpi, bioinstruction-epi, bioinstruction-sirna
emnlp-2025 `bioinstruction-emnlp-2025`	current	2025-11-04	21 (formal evaluation tasks in the final creator paper; sample counts are tracked on child records and never added to this task count)	bioinstruction-emp, bioinstruction-ea, bioinstruction-pd300, bioinstruction-cpd, bioinstruction-tb-human, bioinstruction-tb-mouse, bioinstruction-apa, bioinstruction-ncrna, bioinstruction-modification, bioinstruction-mrl, bioinstruction-prs, bioinstruction-crispr-on-target, bioinstruction-ec, bioinstruction-stability, bioinstruction-fluorescence, bioinstruction-solubility, bioinstruction-thermostability, bioinstruction-aan, bioinstruction-rpi, bioinstruction-epi, bioinstruction-sirna

Tracks and subsets

ID	Count	Basis	Partition?	Notes
DNA tasks `bioinstruction-dna-tasks`	6	formal tasks	Exclusive & exhaustive	EMP, EA, PD300, CPD, TB-H, and TB-M.
RNA tasks `bioinstruction-rna-tasks`	6	formal tasks	Exclusive & exhaustive	APA, ncRNA, Modif, MRL, PRS, and CRI-On.
Protein tasks `bioinstruction-protein-tasks`	5	formal tasks	Exclusive & exhaustive	EC, Stability, Fluorescence, Solubility, and Thermostability.
Multi-molecule tasks `bioinstruction-multi-molecule-tasks`	4	formal tasks	Exclusive & exhaustive	AAN, RPI, EPI, and siRNA.

Registered child tracks

63 normalized runs are registered across these 21 tracks. Open a track for its exact split, prompt group, metrics, results, and comparison charts.

Biology-Instructions Antibody-Antigen Neutralization

Binary neutralization prediction for an antibody-antigen protein-sequence pair, evaluated with Matthews correlation coefficient.

3 evaluation run(s)

Biology-Instructions APA Isoform Prediction

Regression of alternative-polyadenylation isoform usage from an RNA sequence, reported as squared Pearson correlation under the paper's R2 label.

3 evaluation run(s)

Biology-Instructions Core Promoter Detection

Binary detection of a core promoter in a short DNA sequence, evaluated with Matthews correlation coefficient.

3 evaluation run(s)

Biology-Instructions CRISPR On-Target Prediction

Regression of CRISPR guide on-target activity from an RNA sequence, evaluated with Spearman rank correlation.

3 evaluation run(s)

Biology-Instructions Enhancer Activity Prediction

Two-output regression of housekeeping and developmental enhancer activity from a DNA sequence, evaluated with separate Pearson correlations.

3 evaluation run(s)

Biology-Instructions Enzyme Commission Number Prediction

Multi-label Enzyme Commission number prediction from a protein sequence, evaluated with the creator's Fmax implementation.

3 evaluation run(s)

Biology-Instructions Epigenetic Marks Prediction

Binary prediction of whether a DNA sequence carries an epigenetic mark, evaluated with Matthews correlation coefficient.

3 evaluation run(s)

Biology-Instructions Enhancer-Promoter Interaction Prediction

Binary interaction prediction for enhancer and promoter DNA sequences, evaluated with Matthews correlation coefficient.

3 evaluation run(s)

Biology-Instructions Protein Fluorescence Prediction

Regression of protein fluorescence from an amino-acid sequence, evaluated with Spearman rank correlation.

3 evaluation run(s)

Biology-Instructions RNA Modification Prediction

Multi-label prediction of RNA chemical modifications, evaluated with macro area under the ROC curve.

3 evaluation run(s)

Biology-Instructions Mean Ribosome Loading Prediction

Regression of mean ribosome loading from an RNA sequence, reported as squared Pearson correlation under the paper's R2 label.

3 evaluation run(s)

Biology-Instructions Non-coding RNA Function Classification

Thirteen-class functional classification of a non-coding RNA sequence, evaluated with exact extracted-label accuracy.

3 evaluation run(s)

Biology-Instructions Promoter Detection 300

Binary promoter detection in a 300-base-pair DNA context, evaluated with Matthews correlation coefficient.

3 evaluation run(s)

Biology-Instructions Programmable RNA Switches

Three-output regression of ON, OFF, and ON/OFF programmable RNA-switch values, aggregated as their mean squared Pearson correlation.

3 evaluation run(s)

Biology-Instructions RNA-Protein Interaction Prediction

Binary interaction prediction for an RNA and protein sequence pair, evaluated with Matthews correlation coefficient.

3 evaluation run(s)

Biology-Instructions siRNA Efficiency Prediction

Regression of siRNA efficiency from paired sequence context, evaluated with the SAIS-inspired mixed score.

3 evaluation run(s)

Biology-Instructions Protein Solubility Prediction

Binary protein-solubility prediction from an amino-acid sequence, evaluated with accuracy.

3 evaluation run(s)

Biology-Instructions Protein Stability Prediction

Regression of protein stability from an amino-acid sequence, evaluated with Spearman rank correlation.

3 evaluation run(s)

Biology-Instructions Human Transcription Binding Sites Detection

Binary detection of transcription-factor binding sites in human DNA sequences, evaluated with Matthews correlation coefficient.

3 evaluation run(s)

Biology-Instructions Mouse Transcription Binding Sites Detection

Binary detection of transcription-factor binding sites in mouse DNA sequences, evaluated with Matthews correlation coefficient.

3 evaluation run(s)

Biology-Instructions Protein Thermostability Prediction

Regression of protein thermostability from an amino-acid sequence, evaluated with Spearman rank correlation.

3 evaluation run(s)

Scientific Task Atlas

Scientific task classification

complete for emnlp-2025. The classification is exhaustive over the 21 formal evaluation tracks in the final creator paper; example counts remain on child records.

Scientific task	Coverage	Count	Mapping	Evidence
Antibody-antigen interaction	explicitly-in-scope	1 tracks Formal evaluation tracks (Antibody-Antigen Neutralization).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
RNA processing and translation prediction	explicitly-in-scope	2 tracks Formal evaluation tracks (APA Isoform Prediction and Mean Ribosome Loading).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
Promoter detection	explicitly-in-scope	2 tracks Formal evaluation tracks (Core Promoter Detection and Promoter Detection 300).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
CRISPR guide activity prediction	explicitly-in-scope	1 tracks Formal evaluation tracks (CRISPR On-Target Prediction).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
Enhancer activity prediction	explicitly-in-scope	1 tracks Formal evaluation tracks (Enhancer Activity Prediction).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
Protein function annotation	explicitly-in-scope	1 tracks Formal evaluation tracks (Enzyme Commission Number Prediction).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
Epigenetic-mark prediction	explicitly-in-scope	1 tracks Formal evaluation tracks (Epigenetic Marks Prediction).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
Enhancer-promoter interaction prediction	explicitly-in-scope	1 tracks Formal evaluation tracks (Enhancer-Promoter Interaction Prediction).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
Protein fluorescence prediction	explicitly-in-scope	1 tracks Formal evaluation tracks (Protein Fluorescence Prediction).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
RNA modification prediction	explicitly-in-scope	1 tracks Formal evaluation tracks (RNA Modification Prediction).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
RNA function classification	explicitly-in-scope	1 tracks Formal evaluation tracks (Non-coding RNA Function Classification).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
RNA design	explicitly-in-scope	1 tracks Formal evaluation tracks (Programmable RNA Switches).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
RNA-protein interaction prediction	explicitly-in-scope	1 tracks Formal evaluation tracks (RNA-Protein Interaction Prediction).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
siRNA efficacy prediction	explicitly-in-scope	1 tracks Formal evaluation tracks (siRNA Efficiency Prediction).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
Protein solubility prediction	explicitly-in-scope	1 tracks Formal evaluation tracks (Protein Solubility Prediction).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
Protein stability prediction	explicitly-in-scope	2 tracks Formal evaluation tracks (Protein Stability and Protein Thermostability).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
Transcription-factor binding-site prediction	explicitly-in-scope	2 tracks Formal evaluation tracks (Human and mouse Transcription Binding Sites Detection).	official-track high confidence	`bioinstruction-evidence-paper` Track count; not an example count.
Protein sequence design	not-in-scope	0 tracks Formal evaluation tracks in the final creator paper.	official-taxonomy high confidence	`bioinstruction-evidence-paper` The paper explicitly leaves generative sequence design for future work.

Scientific coverage notes

Domain	Coverage	Count	Interpretation
Protein sequence	explicitly-in-scope	5	Five protein-primary formal tasks; protein-containing interaction tasks are tracked separately.
Antibody-antigen	explicitly-in-scope	1	Antibody-Antigen Neutralization is one formal multi-molecule task.
RNA-protein binding	explicitly-in-scope	1	RNA-Protein Interaction Prediction is one formal multi-molecule task.
Protein design	not-in-scope	0	Section 3.2.1 explicitly leaves generative applications such as sequence design for future work; property prediction is not relabeled as design.

Evaluation registry

Works and run settings

A setting change—scope, prompt, tools, budget, grader, or repeats—creates a separate run. Charts never cross a comparability group.

Evaluation settings and results are published on the child-track pages above.

Evidence and change history

Source locators remain visible; expand an item to inspect the exact Registry fields it supports.

Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models · table: pp. 17984, 17987-17989, 17992, 17997-18009; Figure 2; Tables 2 and 4-9 (Canonical name, affiliations, 21-task taxonomy, explicit exclusion of sequence design, split counts, metrics, prompts, model results, and 8,000-example Stage-3 claim.) · Supports 47 fields

Open source →

/name
/aliases
/summary
/organizations
/release_date
/latest_version
/kind
/domains
/capabilities
/modalities
/task_formats
/task_counts/total
/task_counts/basis
/task_counts/subsets
/coverage_notes
/access/level
/access/tasks
/access/artifacts
/access/grader
/access/license
/access/biosafety_notes
/resources
/implementations
/versions/1/release_date
/versions/1/as_of
/versions/1/task_counts/total
/versions/1/task_counts/basis
/versions/1/task_counts/subsets
/versions/1/formal_tracks
/scientific_task_classification/entries/0
/scientific_task_classification/entries/1
/scientific_task_classification/entries/2
/scientific_task_classification/entries/3
/scientific_task_classification/entries/4
/scientific_task_classification/entries/5
/scientific_task_classification/entries/6
/scientific_task_classification/entries/7
/scientific_task_classification/entries/8
/scientific_task_classification/entries/9
/scientific_task_classification/entries/10
/scientific_task_classification/entries/11
/scientific_task_classification/entries/12
/scientific_task_classification/entries/13
/scientific_task_classification/entries/14
/scientific_task_classification/entries/15
/scientific_task_classification/entries/16
/scientific_task_classification/entries/17

bioinstruction-preprint-resource · section: arXiv metadata for 2412.19191 v1 (Initial public date and preprint title.) · Supports 7 fields

Open source →

/release_date
/versions/0/release_date
/versions/0/as_of
/versions/0/task_counts/total
/versions/0/task_counts/basis
/versions/0/task_counts/subsets
/versions/0/formal_tracks

bioinstruction-repository-resource · repository-path: README.md, evaluation/register_tasks.json, evaluation/evaluate.py at 600acaa08c0302e8f5ce86de0fe041f21c13b53e (Partial release, absent repository license, 24 evaluator registration keys, prompt-independent grader implementation, and public code pin.) · Supports 7 fields

Open source →

/access/level
/access/tasks
/access/artifacts
/access/grader
/access/license
/resources
/implementations

bioinstruction-stage3-resource · repository-path: stage3.xlsx at 600acaa08c0302e8f5ce86de0fe041f21c13b53e (One worksheet, used range A1:F8003, and 8,002 non-empty data rows; workbook read without modification.) · Supports 2 fields

Open source →

/access/artifacts
/resources

Unresolved field claims

/access/artifactsConflicted · high — The paper reports 8,000 final AI-polished Stage-3 examples; the pinned official stage3.xlsx contains 8,002 non-empty data rows.
Evidence: bioinstruction-evidence-paper, bioinstruction-evidence-stage3
/implementationsConflicted · high — The final paper defines 21 formal tasks, while the pinned evaluator register_tasks.json contains 24 keys including duplicated or obsolete registrations.
Evidence: bioinstruction-evidence-paper, bioinstruction-evidence-repository

View source-level modification history on GitHub →

Access & reproducibility

Access: Partially open
License: Not reported
Grader: The official evaluator is public, including numeric/label extraction, task-specific metrics, …
Implementations: 1 recorded · official

Access details

Tasks: The final paper publishes task definitions, source datasets, split counts, prompts, and metrics; the creator repository does not include the complete evaluation inputs and labels.
Artifacts: The pinned repository exposes the Stage-3 workbook and evaluator, but not the complete 21-task evaluation splits or published model outputs. Its Stage-3 workbook has 8,002 non-empty data rows, while the paper describes 8,000 final AI-polished examples.
Grader: The official evaluator is public, including numeric/label extraction, task-specific metrics, and a sentiment-model fallback for some textual classifications.
Biosafety: The suite includes antibody-antigen and RNA-protein interaction prediction. BioBench Atlas stores only metadata and aggregate results and does not mirror sequences.

Resources & implementations

Primary resources

bioinstruction-final-paper-resource · CC-BY-4.0
paper · snapshot sha256:3bc92905728242b59bc660372f16c768433d69b64639e801e58e63f4e2b84d4a · checked 2026-07-21
bioinstruction-preprint-resource
paper · checked 2026-07-21
bioinstruction-repository-resource
repository · commit 600acaa08c0302e8f5ce86de0fe041f21c13b53e · checked 2026-07-21
bioinstruction-stage3-resource
dataset · commit 600acaa08c0302e8f5ce86de0fe041f21c13b53e · checked 2026-07-21

Implementations

Biology-Instructions official evaluator
official · 600acaa0