Evaluation run
scib-creator-full
From Benchmarking atlas-level data integration in single-cell genomics
Scopefull · n=13
ShotsNot applicable
TurnsNot applicable
System prompt publicNot applicable
Reasoning / effortNot applicable
BrowserNot applicable
InternetNot applicable
DatabasesNot applicable
Code executionNot applicable
Containertask and method-specific environments in the Snakemake pipeline
External tools16 integration methods with four preprocessing combinations
Token budgetNot applicable
Time / cost budgetNot reported
TemperatureNot applicable
SeedNot reported
RepeatsNot reported
Graderdeterministic single-cell integration metric suite · human review: no
StatisticsRaw metrics are rescaled to higher-is-better; overall = 0.6 bio-conservation + 0.4 batch-removal.
ContaminationTask-specific preprocessing and pre-annotation establish biological labels; simulations provide known ground truth.
Metrics, results, and full protocol
Metrics
| Metric | Kind / baseline | Unit | Aggregation | Threshold / tolerance |
|---|---|---|---|---|
| kBET | absolute | normalized score | batch removal within labels | Not reported |
| Graph connectivity | absolute | normalized score | batch removal within labels | Not reported |
| Batch ASW | absolute | normalized score | batch removal within labels | Not reported |
| Graph iLISI | absolute | normalized score | label-independent batch removal | Not reported |
| PCA regression | absolute | normalized score | label-independent batch removal | Not reported |
| Graph cLISI | absolute | normalized score | label conservation | Not reported |
| ARI | absolute | normalized score | cluster-label agreement | Not reported |
| NMI | absolute | normalized score | cluster-label agreement | Not reported |
| Cell-type ASW | absolute | normalized score | label conservation | Not reported |
| Isolated-label F1 | absolute | normalized score | rare labels | Not reported |
| Isolated-label ASW | absolute | normalized score | rare labels | Not reported |
| Cell-cycle conservation | absolute | normalized score | label-free biological conservation | Not reported |
| HVG conservation | absolute | normalized score | label-free biological conservation | Not reported |
| Trajectory conservation | absolute | normalized score | label-free biological conservation | Not reported |
| Batch-removal score | absolute | normalized score | mean of applicable batch metrics | Not reported |
| Bio-conservation score | absolute | normalized score | mean of applicable biological metrics | Not reported |
| Overall score | absolute | normalized score | 0.6 bio-conservation plus 0.4 batch-removal | Not reported |
No numeric result rows are published yet; the verified protocol remains useful.
Evidence
- section: Results: scIB; Figure 1; Table 1; Methods: Evaluation metrics and Metric aggregation (Defines tasks, methods, preprocessing, fourteen raw metrics, rescaling, category means, and 60/40 overall score.) — supports /scope, /protocol, /metrics