BioBench Atlas
Benchmarks/scIB

suite · audited · verified 2026-07-22

scIB

A 13-task benchmark of single-cell data integration across simulated, scRNA-seq, and scATAC-seq settings, evaluated with 14 batch-removal and biological-conservation metrics.

Single-cellTranscriptomicsEpigenomicsSingle-cell data integration

Benchmark definition

What is counted

Version
paper-2021
Total
13 (atlas-level integration tasks)
Task formats
single-cell batch integration; representation evaluation
Capabilities
Data analysis
Modalities
Raw omicsTable

Version history

VersionStatusRelease / as-ofTotalFormal tracks
paper-2021
scib-paper-2021		current2021-12-2313 (atlas-level integration tasks)None registered

Tracks and subsets

IDCountBasisPartition?Notes
Simulation tasks
scib-simulation		2atlas-level integration tasksExclusive & exhaustiveTwo simulations with known ground truth.
scRNA-seq tasks
scib-scrna		5atlas-level integration tasksExclusive & exhaustiveFive RNA integration settings.
scATAC-seq tasks
scib-scatac		6atlas-level integration tasksExclusive & exhaustiveSix ATAC integration settings across feature spaces and scales.

Scientific Task Atlas

Scientific task classification

complete for paper-2021. The creator study evaluates integration methods across thirteen atlas-level tasks and does not equate cells or batches with task count.

Scientific taskCoverageCountMappingEvidence
Single-cell data integrationexplicitly-in-scope13 tasks
Atlas-level integration tasks.		official-taxonomy
high confidence		scib-paper-definition-evidence
Two simulation, five scRNA-seq, and six scATAC-seq integration tasks.

Scientific coverage notes

DomainCoverageCountInterpretation
Single-cellexplicitly-in-scope13All thirteen tasks assess integration of single-cell datasets.
Transcriptomicsexplicitly-in-scope5Five real scRNA-seq integration tasks; simulation is counted separately.
Epigenomicsexplicitly-in-scope6Six scATAC-seq integration tasks.

Evaluation registry

Works and run settings

A setting change—scope, prompt, tools, budget, grader, or repeats—creates a separate run. Charts never cross a comparability group.

scib-paper-2021-nativevpaper-2021
Scopefull · n=13
ShotsNot applicable
TurnsNot applicable
System prompt publicNot applicable
Reasoning / effortNot applicable
BrowserNot applicable
InternetNot applicable
DatabasesNot applicable
Code executionNot applicable
Containertask and method-specific environments in the Snakemake pipeline
External tools16 integration methods with four preprocessing combinations
Token budgetNot applicable
Time / cost budgetNot reported
TemperatureNot applicable
SeedNot reported
RepeatsNot reported
Graderdeterministic single-cell integration metric suite · human review: no
StatisticsRaw metrics are rescaled to higher-is-better; overall = 0.6 bio-conservation + 0.4 batch-removal.
ContaminationTask-specific preprocessing and pre-annotation establish biological labels; simulations provide known ground truth.
Metrics, results, and full protocol

Metrics

MetricKind / baselineUnitAggregationThreshold / tolerance
kBETabsolutenormalized scorebatch removal within labelsNot reported
Graph connectivityabsolutenormalized scorebatch removal within labelsNot reported
Batch ASWabsolutenormalized scorebatch removal within labelsNot reported
Graph iLISIabsolutenormalized scorelabel-independent batch removalNot reported
PCA regressionabsolutenormalized scorelabel-independent batch removalNot reported
Graph cLISIabsolutenormalized scorelabel conservationNot reported
ARIabsolutenormalized scorecluster-label agreementNot reported
NMIabsolutenormalized scorecluster-label agreementNot reported
Cell-type ASWabsolutenormalized scorelabel conservationNot reported
Isolated-label F1absolutenormalized scorerare labelsNot reported
Isolated-label ASWabsolutenormalized scorerare labelsNot reported
Cell-cycle conservationabsolutenormalized scorelabel-free biological conservationNot reported
HVG conservationabsolutenormalized scorelabel-free biological conservationNot reported
Trajectory conservationabsolutenormalized scorelabel-free biological conservationNot reported
Batch-removal scoreabsolutenormalized scoremean of applicable batch metricsNot reported
Bio-conservation scoreabsolutenormalized scoremean of applicable biological metricsNot reported
Overall scoreabsolutenormalized score0.6 bio-conservation plus 0.4 batch-removalNot reported

No numeric result rows are published yet; the verified protocol remains useful.

Evidence

  • section: Results: scIB; Figure 1; Table 1; Methods: Evaluation metrics and Metric aggregation (Defines tasks, methods, preprocessing, fourteen raw metrics, rescaling, category means, and 60/40 overall score.) — supports /scope, /protocol, /metrics

Evidence and change history

Source locators remain visible; expand an item to inspect the exact Registry fields it supports.

Benchmarking atlas-level data integration in single-cell genomics · section: Abstract; Results pp. 41-50, Figure 1, Table 1, Methods: Metric aggregation (Reports 13 tasks as 2 simulation, 5 scRNA-seq, and 6 scATAC-seq; 16 methods, 4 preprocessing combinations, 14 metrics, and 0.6 bio plus 0.4 batch aggregation.) · Supports 23 fields

Open source →

  • /name
  • /aliases
  • /summary
  • /kind
  • /organizations
  • /release_date
  • /latest_version
  • /domains
  • /capabilities
  • /modalities
  • /task_formats
  • /task_counts/total
  • /task_counts/basis
  • /task_counts/subsets
  • /coverage_notes
  • /access/level
  • /access/license
  • /resources
  • /versions/0/release_date
  • /versions/0/task_counts/total
  • /versions/0/task_counts/basis
  • /versions/0/task_counts/subsets
  • /scientific_task_classification/entries/0
scib-reproducibility-resource · repository-path: README.md and workflow/config sources at 3afbffd3674726e5146797be21cf6bd7470a2c5f; scib package at cd67913396b4c0430710b3d90f1d1841f5fa4468 (Pins public pipeline, task artifacts, metrics implementation, and code licenses.) · Supports 7 fields

Open source →

  • /access/tasks
  • /access/artifacts
  • /access/grader
  • /access/license
  • /resources
  • /implementations
  • /versions/0/formal_tracks

View source-level modification history on GitHub →