Benchmarks/Biology-Instructions/Biology-Instructions Enhancer Activity Prediction

track · audited · verified 2026-07-21

Biology-Instructions Enhancer Activity Prediction

Two-output regression of housekeeping and developmental enhancer activity from a DNA sequence, evaluated with separate Pearson correlations.

Benchmark definition

What is counted

Version: emnlp-2025
Total: 484052 (distinct examples across the published train, validation, and test splits)
Task formats: DNA-sequence two-output regression
Capabilities: PredictionRegression
Modalities: TextDNA or RNA sequence

Version history

Version	Status	Release / as-of	Total	Formal tracks
emnlp-2025 `bioinstruction-ea-emnlp-2025`	current	2025-11-04	484052 (distinct examples across the published train, validation, and test splits)	None registered

Tracks and subsets

ID	Count	Basis	Partition?	Notes
Training split `bioinstruction-ea-train`	402296	examples	Exclusive & exhaustive	Published training split.
Validation split `bioinstruction-ea-validation`	40570	examples	Exclusive & exhaustive	Published validation split.
Test split `bioinstruction-ea-test`	41186	examples	Exclusive & exhaustive	Held-out split used for creator-paper Tables 4-7.

Scientific Task Atlas

Scientific task classification

complete for emnlp-2025. Single-purpose formal Biology-Instructions evaluation track.

Scientific task	Coverage	Count	Mapping	Evidence
Enhancer activity prediction	explicitly-in-scope	484052 examples distinct examples across the published train, validation, and test splits	official-track high confidence	`bioinstruction-ea-evidence-paper` Enhancer activity prediction.

Evaluation registry

Works and run settings

A setting change—scope, prompt, tools, budget, grader, or repeats—creates a separate run. Charts never cross a comparability group.

Evaluation run

bioinstruction-ea-closed-baselines

From Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models

Evaluated models / systems: GPT-4o (Biology-Instructions snapshot not reported), GPT-4o-mini (Biology-Instructions snapshot not reported)

Scopesubset · n=41186

Shots0

Turnssingle-turn

System prompt publicYes

Reasoning / effortThe prompt requests a direct JSON answer and no chain-of-thought.

BrowserNo

InternetNo

DatabasesNo

Code executionNo

ContainerNot reported

External toolsNo

Token budgetNot reported

Time / cost budgetNot reported

TemperatureNot reported

SeedNot reported

RepeatsNot reported

Gradertwo-number extraction and separate Pearson correlations · human review: no

StatisticsPoint metric over the complete published test split, scaled by 100 and rounded to two decimals; no confidence interval is reported.

ContaminationNot reported

Metrics, results, and full protocol

Metrics

Metric	Kind / baseline	Unit	Aggregation	Threshold / tolerance
PCC (housekeeping enhancer activity)	absolute	percent	Pearson correlation across held-out test examples, scaled by 100	Not reported
PCC (developmental enhancer activity)	absolute	percent	Pearson correlation across held-out test examples, scaled by 100	Not reported

Results

Model	Metric	Value	n
GPT-4o-mini (Biology-Instructions snapshot not reported)	PCC (housekeeping enhancer activity)	-0.76 percent EA creator-paper result; metric scaled by 100.	41186
GPT-4o-mini (Biology-Instructions snapshot not reported)	PCC (developmental enhancer activity)	0.09 percent EA creator-paper result; metric scaled by 100.	41186
GPT-4o (Biology-Instructions snapshot not reported)	PCC (housekeeping enhancer activity)	-1.17 percent EA creator-paper result; metric scaled by 100.	41186
GPT-4o (Biology-Instructions snapshot not reported)	PCC (developmental enhancer activity)	-1.49 percent EA creator-paper result; metric scaled by 100.	41186

Evidence

table: Table 2 (EA test split); Appendix A.3; Table 8; Table 9 closed-source prompt (Scope, prompt, output parser, grader, scaling, and aggregation.) — supports /scope, /benchmark_version, /model_ids, /protocol, /metrics
table: Table 4 (EA column) (All registered model values; literature-SOTA row omitted.) — supports /results

Evaluation run

bioinstruction-ea-creator-systems

From Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models

Evaluated models / systems: ChatMultiOmics stage 1 + balanced stage 2, ChatMultiOmics stage 1 + stage 2, ChatMultiOmics, ChatMultiOmics stage 2 only

Scopesubset · n=41186

Shots0

Turnssingle-turn

System prompt publicYes

Reasoning / effortPsc requests clear, concise task answers and numeric output for regression tasks.

BrowserNo

InternetNo

DatabasesNo

Code executionNo

ContainerNot reported

External toolsNo

Token budgetNot reported

Time / cost budgetNot reported

TemperatureNot reported

SeedNot reported

RepeatsNot reported

Gradertwo-number extraction and separate Pearson correlations · human review: no

StatisticsPoint metric over the complete published test split, scaled by 100 and rounded to two decimals; no confidence interval is reported.

ContaminationNot reported

Metrics, results, and full protocol

Metrics

Metric	Kind / baseline	Unit	Aggregation	Threshold / tolerance
PCC (housekeeping enhancer activity)	absolute	percent	Pearson correlation across held-out test examples, scaled by 100	Not reported
PCC (developmental enhancer activity)	absolute	percent	Pearson correlation across held-out test examples, scaled by 100	Not reported

Results

Model	Metric	Value	n
ChatMultiOmics stage 1 + balanced stage 2	PCC (housekeeping enhancer activity)	0.92 percent EA creator-paper result; metric scaled by 100.	41186
ChatMultiOmics stage 1 + balanced stage 2	PCC (developmental enhancer activity)	0.06 percent EA creator-paper result; metric scaled by 100.	41186
ChatMultiOmics stage 2 only	PCC (housekeeping enhancer activity)	-0.16 percent EA creator-paper result; metric scaled by 100.	41186
ChatMultiOmics stage 2 only	PCC (developmental enhancer activity)	0.08 percent EA creator-paper result; metric scaled by 100.	41186
ChatMultiOmics stage 1 + stage 2	PCC (housekeeping enhancer activity)	59.74 percent EA creator-paper result; metric scaled by 100.	41186
ChatMultiOmics stage 1 + stage 2	PCC (developmental enhancer activity)	46.82 percent EA creator-paper result; metric scaled by 100.	41186
ChatMultiOmics	PCC (housekeeping enhancer activity)	57.24 percent EA creator-paper result; metric scaled by 100.	41186
ChatMultiOmics	PCC (developmental enhancer activity)	45.92 percent EA creator-paper result; metric scaled by 100.	41186

Evidence

table: Table 2 (EA test split); Appendix A.3; Table 8; Section 4.2 Psc prompt (Scope, prompt, output parser, grader, scaling, and aggregation.) — supports /scope, /benchmark_version, /model_ids, /protocol, /metrics
table: Table 4 (EA column) (All registered model values; literature-SOTA row omitted.) — supports /results

Evaluation run

bioinstruction-ea-open-baselines

From Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models

Evaluated models / systems: Alpaca-7B (Biology-Instructions label), Galactica-1.3B (Biology-Instructions label), GLM-4-9B-Chat (Biology-Instructions label), InstructProtein-1.3B (Biology-Instructions label), Llama-molinst-protein-7B (Mol-Ins), Llama2-7B-Chat (Biology-Instructions label), LLaMA3.1-8B-Instruct (Biology-Instructions label), Qwen2-7B (Biology-Instructions label), Vicuna-v1.5-7B (Biology-Instructions label)

Scopesubset · n=41186

Shots0

Turnssingle-turn

System prompt publicNo

Reasoning / effortThe prompt requests the task-formatted answer and says not to explain or repeat.

BrowserNo

InternetNo

DatabasesNo

Code executionNo

ContainerNot reported

External toolsNo

Token budgetNot reported

Time / cost budgetNot reported

TemperatureNot reported

SeedNot reported

RepeatsNot reported

Gradertwo-number extraction and separate Pearson correlations · human review: no

StatisticsPoint metric over the complete published test split, scaled by 100 and rounded to two decimals; no confidence interval is reported.

ContaminationNot reported

Metrics, results, and full protocol

Metrics

Metric	Kind / baseline	Unit	Aggregation	Threshold / tolerance
PCC (housekeeping enhancer activity)	absolute	percent	Pearson correlation across held-out test examples, scaled by 100	Not reported
PCC (developmental enhancer activity)	absolute	percent	Pearson correlation across held-out test examples, scaled by 100	Not reported

Results

Model	Metric	Value	n
LLaMA3.1-8B-Instruct (Biology-Instructions label)	PCC (housekeeping enhancer activity)	0.61 percent EA creator-paper result; metric scaled by 100.	41186
LLaMA3.1-8B-Instruct (Biology-Instructions label)	PCC (developmental enhancer activity)	0.27 percent EA creator-paper result; metric scaled by 100.	41186
Qwen2-7B (Biology-Instructions label)	PCC (housekeeping enhancer activity)	0.4 percent EA creator-paper result; metric scaled by 100.	41186
Qwen2-7B (Biology-Instructions label)	PCC (developmental enhancer activity)	0.35 percent EA creator-paper result; metric scaled by 100.	41186
Llama2-7B-Chat (Biology-Instructions label)	PCC (housekeeping enhancer activity)	0.55 percent EA creator-paper result; metric scaled by 100.	41186
Llama2-7B-Chat (Biology-Instructions label)	PCC (developmental enhancer activity)	0.13 percent EA creator-paper result; metric scaled by 100.	41186
Alpaca-7B (Biology-Instructions label)	PCC (housekeeping enhancer activity)	-0.11 percent EA creator-paper result; metric scaled by 100.	41186
Alpaca-7B (Biology-Instructions label)	PCC (developmental enhancer activity)	0.31 percent EA creator-paper result; metric scaled by 100.	41186
GLM-4-9B-Chat (Biology-Instructions label)	PCC (housekeeping enhancer activity)	0.87 percent EA creator-paper result; metric scaled by 100.	41186
GLM-4-9B-Chat (Biology-Instructions label)	PCC (developmental enhancer activity)	0.17 percent EA creator-paper result; metric scaled by 100.	41186
Vicuna-v1.5-7B (Biology-Instructions label)	PCC (housekeeping enhancer activity)	0.18 percent EA creator-paper result; metric scaled by 100.	41186
Vicuna-v1.5-7B (Biology-Instructions label)	PCC (developmental enhancer activity)	0.69 percent EA creator-paper result; metric scaled by 100.	41186
Galactica-1.3B (Biology-Instructions label)	PCC (housekeeping enhancer activity)	0.13 percent EA creator-paper result; metric scaled by 100.	41186
Galactica-1.3B (Biology-Instructions label)	PCC (developmental enhancer activity)	0.09 percent EA creator-paper result; metric scaled by 100.	41186
InstructProtein-1.3B (Biology-Instructions label)	PCC (housekeeping enhancer activity)	0 percent EA creator-paper result; metric scaled by 100.	41186
InstructProtein-1.3B (Biology-Instructions label)	PCC (developmental enhancer activity)	0.39 percent EA creator-paper result; metric scaled by 100.	41186
Llama-molinst-protein-7B (Mol-Ins)	PCC (housekeeping enhancer activity)	0.02 percent EA creator-paper result; metric scaled by 100.	41186
Llama-molinst-protein-7B (Mol-Ins)	PCC (developmental enhancer activity)	0.1 percent EA creator-paper result; metric scaled by 100.	41186

Evidence

table: Table 2 (EA test split); Appendix A.3; Table 8; Table 9 open-source prompt (Scope, prompt, output parser, grader, scaling, and aggregation.) — supports /scope, /benchmark_version, /model_ids, /protocol, /metrics
table: Table 4 (EA column) (All registered model values; literature-SOTA row omitted.) — supports /results

Comparable result views

PCC (housekeeping enhancer activity)

bioinstruction-ea-closed-baselines · bioinstruction-ea-closed-baselines-emnlp-2025

CSV ↓

Accessible data table

Model	Value	Comparability group
GPT-4o-mini (Biology-Instructions snapshot not reported)	-0.76	bioinstruction-ea-closed-baselines-emnlp-2025
GPT-4o (Biology-Instructions snapshot not reported)	-1.17	bioinstruction-ea-closed-baselines-emnlp-2025

PCC (developmental enhancer activity)

bioinstruction-ea-closed-baselines · bioinstruction-ea-closed-baselines-emnlp-2025

CSV ↓

Accessible data table

Model	Value	Comparability group
GPT-4o-mini (Biology-Instructions snapshot not reported)	0.09	bioinstruction-ea-closed-baselines-emnlp-2025
GPT-4o (Biology-Instructions snapshot not reported)	-1.49	bioinstruction-ea-closed-baselines-emnlp-2025

PCC (housekeeping enhancer activity)

bioinstruction-ea-creator-systems · bioinstruction-ea-creator-systems-emnlp-2025

CSV ↓

Accessible data table

Model	Value	Comparability group
ChatMultiOmics stage 1 + balanced stage 2	0.92	bioinstruction-ea-creator-systems-emnlp-2025
ChatMultiOmics stage 2 only	-0.16	bioinstruction-ea-creator-systems-emnlp-2025
ChatMultiOmics stage 1 + stage 2	59.74	bioinstruction-ea-creator-systems-emnlp-2025
ChatMultiOmics	57.24	bioinstruction-ea-creator-systems-emnlp-2025

PCC (developmental enhancer activity)

bioinstruction-ea-creator-systems · bioinstruction-ea-creator-systems-emnlp-2025

CSV ↓

Accessible data table

Model	Value	Comparability group
ChatMultiOmics stage 1 + balanced stage 2	0.06	bioinstruction-ea-creator-systems-emnlp-2025
ChatMultiOmics stage 2 only	0.08	bioinstruction-ea-creator-systems-emnlp-2025
ChatMultiOmics stage 1 + stage 2	46.82	bioinstruction-ea-creator-systems-emnlp-2025
ChatMultiOmics	45.92	bioinstruction-ea-creator-systems-emnlp-2025

PCC (housekeeping enhancer activity)

bioinstruction-ea-open-baselines · bioinstruction-ea-open-baselines-emnlp-2025

CSV ↓

Accessible data table

Model	Value	Comparability group
LLaMA3.1-8B-Instruct (Biology-Instructions label)	0.61	bioinstruction-ea-open-baselines-emnlp-2025
Qwen2-7B (Biology-Instructions label)	0.4	bioinstruction-ea-open-baselines-emnlp-2025
Llama2-7B-Chat (Biology-Instructions label)	0.55	bioinstruction-ea-open-baselines-emnlp-2025
Alpaca-7B (Biology-Instructions label)	-0.11	bioinstruction-ea-open-baselines-emnlp-2025
GLM-4-9B-Chat (Biology-Instructions label)	0.87	bioinstruction-ea-open-baselines-emnlp-2025
Vicuna-v1.5-7B (Biology-Instructions label)	0.18	bioinstruction-ea-open-baselines-emnlp-2025
Galactica-1.3B (Biology-Instructions label)	0.13	bioinstruction-ea-open-baselines-emnlp-2025
InstructProtein-1.3B (Biology-Instructions label)	0	bioinstruction-ea-open-baselines-emnlp-2025
Llama-molinst-protein-7B (Mol-Ins)	0.02	bioinstruction-ea-open-baselines-emnlp-2025

PCC (developmental enhancer activity)

bioinstruction-ea-open-baselines · bioinstruction-ea-open-baselines-emnlp-2025

CSV ↓

Accessible data table

Model	Value	Comparability group
LLaMA3.1-8B-Instruct (Biology-Instructions label)	0.27	bioinstruction-ea-open-baselines-emnlp-2025
Qwen2-7B (Biology-Instructions label)	0.35	bioinstruction-ea-open-baselines-emnlp-2025
Llama2-7B-Chat (Biology-Instructions label)	0.13	bioinstruction-ea-open-baselines-emnlp-2025
Alpaca-7B (Biology-Instructions label)	0.31	bioinstruction-ea-open-baselines-emnlp-2025
GLM-4-9B-Chat (Biology-Instructions label)	0.17	bioinstruction-ea-open-baselines-emnlp-2025
Vicuna-v1.5-7B (Biology-Instructions label)	0.69	bioinstruction-ea-open-baselines-emnlp-2025
Galactica-1.3B (Biology-Instructions label)	0.09	bioinstruction-ea-open-baselines-emnlp-2025
InstructProtein-1.3B (Biology-Instructions label)	0.39	bioinstruction-ea-open-baselines-emnlp-2025
Llama-molinst-protein-7B (Mol-Ins)	0.1	bioinstruction-ea-open-baselines-emnlp-2025

Evidence and change history

Source locators remain visible; expand an item to inspect the exact Registry fields it supports.

Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models · table: Table 2 (EA row); Appendix A.2-A.3; Table 8; Tables 4-7 (Task definition, split counts, input/output format, metric, and creator evaluation.) · Supports 29 fields

Open source →

/name
/aliases
/summary
/kind
/parent_id
/organizations
/release_date
/latest_version
/domains
/capabilities
/modalities
/task_formats
/task_counts/total
/task_counts/basis
/task_counts/subsets
/access/level
/access/tasks
/access/artifacts
/access/grader
/access/license
/access/biosafety_notes
/resources
/implementations
/versions/0/release_date
/versions/0/as_of
/versions/0/task_counts/total
/versions/0/task_counts/basis
/versions/0/task_counts/subsets
/scientific_task_classification/entries/0

bioinstruction-ea-repository-resource · repository-path: evaluation/evaluate.py and evaluation/register_tasks.json at 600acaa08c0302e8f5ce86de0fe041f21c13b53e (Public grader implementation, partial artifact release, and absent repository license.) · Supports 7 fields

Open source →

/access/level
/access/tasks
/access/artifacts
/access/grader
/access/license
/resources
/implementations

View source-level modification history on GitHub →

Access & reproducibility

Access: Partially open
License: Not reported
Grader: two-number parser followed by separate housekeeping and developmental Pearson correlations.
Implementations: 1 recorded · official

Access details

Tasks: The final paper publishes this task's source, definition, and split counts; the creator repository does not include the complete evaluation inputs and labels.
Artifacts: The official post-processing evaluator is public, but the published test examples, labels, and model outputs are not included in the official repository.
Grader: two-number parser followed by separate housekeeping and developmental Pearson correlations.
Biosafety: BioBench Atlas stores task metadata and aggregate results only and does not mirror the upstream biological sequences.