BioBench Atlas
Benchmarks/FLIP/FLIP AAV

track · audited · verified 2026-07-21

FLIP AAV

Seven supervised splits over sampled and machine-designed AAV2 VP-1 capsid variants, measuring generalization across mutation depth, fitness, and sampled-versus-designed pools.

Protein sequenceProtein designProtein fitness prediction

Benchmark definition

What is counted

Version
original-2021
Total
284009 (distinct sequence-fitness examples across the sampled and designed AAV pools)
Task formats
sequence-to-capsid-viability regression
Capabilities
PredictionRegression
Modalities
Protein sequenceTable

Version history

VersionStatusRelease / as-ofTotalFormal tracks
original-2021
flip-aav-original-2021		current2021-10-11284009 (distinct sequence-fitness examples across the sampled and designed AAV pools)None registered

Tracks and subsets

IDCountBasisPartition?Notes
Mut-Des test set
aav-mut-des-test		201426test examplesNoTrain on 82,583 sampled variants; test on 201,426 designed variants.
Des-Mut test set
aav-des-mut-test		82583test examplesNoTrain on 201,426 designed variants; test on 82,583 sampled variants.
1-vs-rest test set
aav-one-vs-rest-test		81413test examplesNo1,170 training examples; sampled pool only.
2-vs-rest test set
aav-two-vs-rest-test		50776test examplesNo31,807 training examples; sampled pool only.
7-vs-rest test set
aav-seven-vs-rest-test		12581test examplesNo70,002 training examples; sampled pool only.
low-vs-high test set
aav-low-vs-high-test		35037test examplesNo47,546 training examples at or below wild-type fitness; test examples are above wild-type fitness.
Sampled test set
aav-sampled-test		16517test examplesNo66,066 random training examples; discourse-only split not recommended for performance comparison.

Scientific Task Atlas

Scientific task classification

complete for original-2021. AAV sequence-to-fitness landscape.

Scientific taskCoverageCountMappingEvidence
Protein fitness predictionexplicitly-in-scope284009 examples
Distinct sequence-fitness examples across the sampled and designed AAV pools.		official-track
high confidence		flip-aav-evidence-paper
Example count, not a count of formal split tasks.

Scientific coverage notes

DomainCoverageCountInterpretation
Protein designexplicitly-in-scopeNot reportedThe dataset includes model-designed variants and is motivated by capsid engineering, but the evaluated task is fitness prediction rather than sequence generation.

Evaluation registry

Works and run settings

A setting change—scope, prompt, tools, budget, grader, or repeats—creates a separate run. Charts never cross a comparability group.

flip-aav-des-mut-spearmanvoriginal-2021

Evaluated models / systems: Convolutional network (FLIP), ESM-untrained (mean), ESM-untrained (mut mean), ESM-1b (mean; FLIP head), ESM-1b (mut mean; FLIP head), ESM-1v (mean; FLIP head), ESM-1v (mut mean; FLIP head), Levenshtein distance baseline (FLIP), Ridge regression (FLIP)

Scopesubset · n=82583
ShotsNot applicable
TurnsNot applicable
System prompt publicNot applicable
Reasoning / effortNot applicable
BrowserNot applicable
InternetNot applicable
DatabasesNot applicable
Code executionNot applicable
ContainerNot reported
External toolsmodel-specific frozen embeddings or one-hot sequence encoding
Token budgetNot applicable
Time / cost budgetNot reported
TemperatureNot applicable
SeedNot reported
RepeatsNot reported
Graderdeterministic regression scorer · human review: no
StatisticsSpearman correlation over held-out test examples; the main table reports point values without confidence intervals.
ContaminationPool shift: model-designed variants are training data and sampled variants are held out.
Metrics, results, and full protocol

Metrics

MetricKind / baselineUnitAggregationThreshold / tolerance
Spearman correlationabsolutecorrelationcomputed across all examples in this held-out test splitNot reported
Mean squared errorabsolutesquared fitness unitsmean across all examples in this held-out test splitNot reported

Results

ModelMetricValuen
ESM-1b (mean; FLIP head)Spearman correlation0.59 correlation
Held-out test-set Spearman from the creator paper.		82583
ESM-1b (mut mean; FLIP head)Spearman correlation0.7 correlation
Held-out test-set Spearman from the creator paper.		82583
ESM-1v (mean; FLIP head)Spearman correlation0.44 correlation
Held-out test-set Spearman from the creator paper.		82583
ESM-1v (mut mean; FLIP head)Spearman correlation0.71 correlation
Held-out test-set Spearman from the creator paper.		82583
ESM-untrained (mean)Spearman correlation0.34 correlation
Held-out test-set Spearman from the creator paper.		82583
ESM-untrained (mut mean)Spearman correlation0.64 correlation
Held-out test-set Spearman from the creator paper.		82583
Ridge regression (FLIP)Spearman correlation0.53 correlation
Held-out test-set Spearman from the creator paper.		82583
Convolutional network (FLIP)Spearman correlation0.75 correlation
Held-out test-set Spearman from the creator paper.		82583
Levenshtein distance baseline (FLIP)Spearman correlation-0.07 correlation
Held-out test-set Spearman from the creator paper.		82583

Evidence

  • table: pp. 4–7, Table 2 and Sections 3–4 (Exact train/test counts, split rule, baseline identities, pooling, optimizer, batch sizes, early stopping, and hardware.) — supports /scope, /protocol, /model_ids
  • table: p. 9, Table 5 (Des-Mut column) (Published held-out Spearman values; em dashes and NA entries are omitted rather than encoded as zeros.) — supports /metrics, /results
flip-aav-low-vs-high-spearmanvoriginal-2021

Evaluated models / systems: Convolutional network (FLIP), ESM-untrained (mean), ESM-untrained (mut mean), ESM-untrained (per AA), ESM-1b (mean; FLIP head), ESM-1b (mut mean; FLIP head), ESM-1b (per AA; FLIP head), ESM-1v (mean; FLIP head), ESM-1v (mut mean; FLIP head), ESM-1v (per AA; FLIP head), Levenshtein distance baseline (FLIP), Ridge regression (FLIP)

Scopesubset · n=35037
ShotsNot applicable
TurnsNot applicable
System prompt publicNot applicable
Reasoning / effortNot applicable
BrowserNot applicable
InternetNot applicable
DatabasesNot applicable
Code executionNot applicable
ContainerNot reported
External toolsmodel-specific frozen embeddings or one-hot sequence encoding
Token budgetNot applicable
Time / cost budgetNot reported
TemperatureNot applicable
SeedNot reported
RepeatsNot reported
Graderdeterministic regression scorer · human review: no
StatisticsSpearman correlation over held-out test examples; the main table reports point values without confidence intervals.
ContaminationFitness extrapolation above wild-type fitness.
Metrics, results, and full protocol

Metrics

MetricKind / baselineUnitAggregationThreshold / tolerance
Spearman correlationabsolutecorrelationcomputed across all examples in this held-out test splitNot reported
Mean squared errorabsolutesquared fitness unitsmean across all examples in this held-out test splitNot reported

Results

ModelMetricValuen
ESM-1b (per AA; FLIP head)Spearman correlation0.39 correlation
Held-out test-set Spearman from the creator paper.		35037
ESM-1b (mean; FLIP head)Spearman correlation0.18 correlation
Held-out test-set Spearman from the creator paper.		35037
ESM-1b (mut mean; FLIP head)Spearman correlation0.33 correlation
Held-out test-set Spearman from the creator paper.		35037
ESM-1v (per AA; FLIP head)Spearman correlation0.34 correlation
Held-out test-set Spearman from the creator paper.		35037
ESM-1v (mean; FLIP head)Spearman correlation0.2 correlation
Held-out test-set Spearman from the creator paper.		35037
ESM-1v (mut mean; FLIP head)Spearman correlation0.31 correlation
Held-out test-set Spearman from the creator paper.		35037
ESM-untrained (per AA)Spearman correlation0.08 correlation
Held-out test-set Spearman from the creator paper.		35037
ESM-untrained (mean)Spearman correlation0.22 correlation
Held-out test-set Spearman from the creator paper.		35037
ESM-untrained (mut mean)Spearman correlation0.24 correlation
Held-out test-set Spearman from the creator paper.		35037
Ridge regression (FLIP)Spearman correlation0.12 correlation
Held-out test-set Spearman from the creator paper.		35037
Convolutional network (FLIP)Spearman correlation0.34 correlation
Held-out test-set Spearman from the creator paper.		35037
Levenshtein distance baseline (FLIP)Spearman correlation0.25 correlation
Held-out test-set Spearman from the creator paper.		35037

Evidence

  • table: pp. 4–7, Table 2 and Sections 3–4 (Exact train/test counts, split rule, baseline identities, pooling, optimizer, batch sizes, early stopping, and hardware.) — supports /scope, /protocol, /model_ids
  • table: p. 9, Table 5 (low-vs-high column) (Published held-out Spearman values; em dashes and NA entries are omitted rather than encoded as zeros.) — supports /metrics, /results
flip-aav-mut-des-spearmanvoriginal-2021

Evaluated models / systems: Convolutional network (FLIP), ESM-untrained (mean), ESM-untrained (mut mean), ESM-untrained (per AA), ESM-1b (mean; FLIP head), ESM-1b (mut mean; FLIP head), ESM-1b (per AA; FLIP head), ESM-1v (mean; FLIP head), ESM-1v (mut mean; FLIP head), ESM-1v (per AA; FLIP head), Levenshtein distance baseline (FLIP), Ridge regression (FLIP)

Scopesubset · n=201426
ShotsNot applicable
TurnsNot applicable
System prompt publicNot applicable
Reasoning / effortNot applicable
BrowserNot applicable
InternetNot applicable
DatabasesNot applicable
Code executionNot applicable
ContainerNot reported
External toolsmodel-specific frozen embeddings or one-hot sequence encoding
Token budgetNot applicable
Time / cost budgetNot reported
TemperatureNot applicable
SeedNot reported
RepeatsNot reported
Graderdeterministic regression scorer · human review: no
StatisticsSpearman correlation over held-out test examples; the main table reports point values without confidence intervals.
ContaminationPool shift: sampled variants are training data and model-designed variants are held out.
Metrics, results, and full protocol

Metrics

MetricKind / baselineUnitAggregationThreshold / tolerance
Spearman correlationabsolutecorrelationcomputed across all examples in this held-out test splitNot reported
Mean squared errorabsolutesquared fitness unitsmean across all examples in this held-out test splitNot reported

Results

ModelMetricValuen
ESM-1b (per AA; FLIP head)Spearman correlation0.76 correlation
Held-out test-set Spearman from the creator paper.		201426
ESM-1b (mean; FLIP head)Spearman correlation0.63 correlation
Held-out test-set Spearman from the creator paper.		201426
ESM-1b (mut mean; FLIP head)Spearman correlation0.7 correlation
Held-out test-set Spearman from the creator paper.		201426
ESM-1v (per AA; FLIP head)Spearman correlation0.79 correlation
Held-out test-set Spearman from the creator paper.		201426
ESM-1v (mean; FLIP head)Spearman correlation0.55 correlation
Held-out test-set Spearman from the creator paper.		201426
ESM-1v (mut mean; FLIP head)Spearman correlation0.7 correlation
Held-out test-set Spearman from the creator paper.		201426
ESM-untrained (per AA)Spearman correlation0.56 correlation
Held-out test-set Spearman from the creator paper.		201426
ESM-untrained (mean)Spearman correlation0.27 correlation
Held-out test-set Spearman from the creator paper.		201426
ESM-untrained (mut mean)Spearman correlation0.62 correlation
Held-out test-set Spearman from the creator paper.		201426
Ridge regression (FLIP)Spearman correlation0.64 correlation
Held-out test-set Spearman from the creator paper.		201426
Convolutional network (FLIP)Spearman correlation0.71 correlation
Held-out test-set Spearman from the creator paper.		201426
Levenshtein distance baseline (FLIP)Spearman correlation0.6 correlation
Held-out test-set Spearman from the creator paper.		201426

Evidence

  • table: pp. 4–7, Table 2 and Sections 3–4 (Exact train/test counts, split rule, baseline identities, pooling, optimizer, batch sizes, early stopping, and hardware.) — supports /scope, /protocol, /model_ids
  • table: p. 9, Table 5 (Mut-Des column) (Published held-out Spearman values; em dashes and NA entries are omitted rather than encoded as zeros.) — supports /metrics, /results
flip-aav-one-vs-rest-spearmanvoriginal-2021

Evaluated models / systems: Convolutional network (FLIP), ESM-untrained (mean), ESM-untrained (mut mean), ESM-untrained (per AA), ESM-1b (mean; FLIP head), ESM-1b (mut mean; FLIP head), ESM-1b (per AA; FLIP head), ESM-1v (mean; FLIP head), ESM-1v (mut mean; FLIP head), ESM-1v (per AA; FLIP head), Levenshtein distance baseline (FLIP), Ridge regression (FLIP)

Scopesubset · n=81413
ShotsNot applicable
TurnsNot applicable
System prompt publicNot applicable
Reasoning / effortNot applicable
BrowserNot applicable
InternetNot applicable
DatabasesNot applicable
Code executionNot applicable
ContainerNot reported
External toolsmodel-specific frozen embeddings or one-hot sequence encoding
Token budgetNot applicable
Time / cost budgetNot reported
TemperatureNot applicable
SeedNot reported
RepeatsNot reported
Graderdeterministic regression scorer · human review: no
StatisticsSpearman correlation over held-out test examples; the main table reports point values without confidence intervals.
ContaminationMutation-depth extrapolation; the held-out set excludes wild type and single-mutant training examples.
Metrics, results, and full protocol

Metrics

MetricKind / baselineUnitAggregationThreshold / tolerance
Spearman correlationabsolutecorrelationcomputed across all examples in this held-out test splitNot reported
Mean squared errorabsolutesquared fitness unitsmean across all examples in this held-out test splitNot reported

Results

ModelMetricValuen
ESM-1b (per AA; FLIP head)Spearman correlation0.03 correlation
Held-out test-set Spearman from the creator paper.		81413
ESM-1b (mean; FLIP head)Spearman correlation0.04 correlation
Held-out test-set Spearman from the creator paper.		81413
ESM-1b (mut mean; FLIP head)Spearman correlation0.31 correlation
Held-out test-set Spearman from the creator paper.		81413
ESM-1v (per AA; FLIP head)Spearman correlation0.1 correlation
Held-out test-set Spearman from the creator paper.		81413
ESM-1v (mean; FLIP head)Spearman correlation0.18 correlation
Held-out test-set Spearman from the creator paper.		81413
ESM-1v (mut mean; FLIP head)Spearman correlation0.44 correlation
Held-out test-set Spearman from the creator paper.		81413
ESM-untrained (per AA)Spearman correlation0.18 correlation
Held-out test-set Spearman from the creator paper.		81413
ESM-untrained (mean)Spearman correlation0.01 correlation
Held-out test-set Spearman from the creator paper.		81413
ESM-untrained (mut mean)Spearman correlation0.26 correlation
Held-out test-set Spearman from the creator paper.		81413
Ridge regression (FLIP)Spearman correlation0.22 correlation
Held-out test-set Spearman from the creator paper.		81413
Convolutional network (FLIP)Spearman correlation0.48 correlation
Held-out test-set Spearman from the creator paper.		81413
Levenshtein distance baseline (FLIP)Spearman correlation-0.11 correlation
Held-out test-set Spearman from the creator paper.		81413

Evidence

  • table: pp. 4–7, Table 2 and Sections 3–4 (Exact train/test counts, split rule, baseline identities, pooling, optimizer, batch sizes, early stopping, and hardware.) — supports /scope, /protocol, /model_ids
  • table: p. 9, Table 5 (1-vs-rest column) (Published held-out Spearman values; em dashes and NA entries are omitted rather than encoded as zeros.) — supports /metrics, /results
flip-aav-sampled-spearmanvoriginal-2021

Evaluated models / systems: Convolutional network (FLIP), ESM-untrained (per AA), ESM-1b (per AA; FLIP head), ESM-1v (per AA; FLIP head), Ridge regression (FLIP)

Scopesubset · n=16517
ShotsNot applicable
TurnsNot applicable
System prompt publicNot applicable
Reasoning / effortNot applicable
BrowserNot applicable
InternetNot applicable
DatabasesNot applicable
Code executionNot applicable
ContainerNot reported
External toolsmodel-specific frozen embeddings or one-hot sequence encoding
Token budgetNot applicable
Time / cost budgetNot reported
TemperatureNot applicable
SeedNot reported
RepeatsNot reported
Graderdeterministic regression scorer · human review: no
StatisticsSpearman correlation over held-out test examples; the main table reports point values without confidence intervals.
ContaminationRandom split can place closely related variants across train/test and is explicitly described as optimistic.
Metrics, results, and full protocol

Metrics

MetricKind / baselineUnitAggregationThreshold / tolerance
Spearman correlationabsolutecorrelationcomputed across all examples in this held-out test splitNot reported
Mean squared errorabsolutesquared fitness unitsmean across all examples in this held-out test splitNot reported

Results

ModelMetricValuen
ESM-1b (per AA; FLIP head)Spearman correlation0.9 correlation
Held-out test-set Spearman from the creator paper.		16517
ESM-1v (per AA; FLIP head)Spearman correlation0.92 correlation
Held-out test-set Spearman from the creator paper.		16517
ESM-untrained (per AA)Spearman correlation0.78 correlation
Held-out test-set Spearman from the creator paper.		16517
Ridge regression (FLIP)Spearman correlation0.83 correlation
Held-out test-set Spearman from the creator paper.		16517
Convolutional network (FLIP)Spearman correlation0.92 correlation
Held-out test-set Spearman from the creator paper.		16517

Evidence

  • table: pp. 4–7, Table 2 and Sections 3–4 (Exact train/test counts, split rule, baseline identities, pooling, optimizer, batch sizes, early stopping, and hardware.) — supports /scope, /protocol, /model_ids
  • table: p. 10, Table 7 (AAV column) (Published held-out Spearman values; em dashes and NA entries are omitted rather than encoded as zeros.) — supports /metrics, /results
flip-aav-seven-vs-rest-spearmanvoriginal-2021

Evaluated models / systems: Convolutional network (FLIP), ESM-untrained (mean), ESM-untrained (mut mean), ESM-untrained (per AA), ESM-1b (mean; FLIP head), ESM-1b (mut mean; FLIP head), ESM-1b (per AA; FLIP head), ESM-1v (mean; FLIP head), ESM-1v (mut mean; FLIP head), ESM-1v (per AA; FLIP head), Levenshtein distance baseline (FLIP), Ridge regression (FLIP)

Scopesubset · n=12581
ShotsNot applicable
TurnsNot applicable
System prompt publicNot applicable
Reasoning / effortNot applicable
BrowserNot applicable
InternetNot applicable
DatabasesNot applicable
Code executionNot applicable
ContainerNot reported
External toolsmodel-specific frozen embeddings or one-hot sequence encoding
Token budgetNot applicable
Time / cost budgetNot reported
TemperatureNot applicable
SeedNot reported
RepeatsNot reported
Graderdeterministic regression scorer · human review: no
StatisticsSpearman correlation over held-out test examples; the main table reports point values without confidence intervals.
ContaminationMutation-depth extrapolation above seven changes.
Metrics, results, and full protocol

Metrics

MetricKind / baselineUnitAggregationThreshold / tolerance
Spearman correlationabsolutecorrelationcomputed across all examples in this held-out test splitNot reported
Mean squared errorabsolutesquared fitness unitsmean across all examples in this held-out test splitNot reported

Results

ModelMetricValuen
ESM-1b (per AA; FLIP head)Spearman correlation0.65 correlation
Held-out test-set Spearman from the creator paper.		12581
ESM-1b (mean; FLIP head)Spearman correlation0.46 correlation
Held-out test-set Spearman from the creator paper.		12581
ESM-1b (mut mean; FLIP head)Spearman correlation0.61 correlation
Held-out test-set Spearman from the creator paper.		12581
ESM-1v (per AA; FLIP head)Spearman correlation0.7 correlation
Held-out test-set Spearman from the creator paper.		12581
ESM-1v (mean; FLIP head)Spearman correlation0.45 correlation
Held-out test-set Spearman from the creator paper.		12581
ESM-1v (mut mean; FLIP head)Spearman correlation0.64 correlation
Held-out test-set Spearman from the creator paper.		12581
ESM-untrained (per AA)Spearman correlation0.42 correlation
Held-out test-set Spearman from the creator paper.		12581
ESM-untrained (mean)Spearman correlation0.22 correlation
Held-out test-set Spearman from the creator paper.		12581
ESM-untrained (mut mean)Spearman correlation0.56 correlation
Held-out test-set Spearman from the creator paper.		12581
Ridge regression (FLIP)Spearman correlation0.65 correlation
Held-out test-set Spearman from the creator paper.		12581
Convolutional network (FLIP)Spearman correlation0.74 correlation
Held-out test-set Spearman from the creator paper.		12581
Levenshtein distance baseline (FLIP)Spearman correlation0.53 correlation
Held-out test-set Spearman from the creator paper.		12581

Evidence

  • table: pp. 4–7, Table 2 and Sections 3–4 (Exact train/test counts, split rule, baseline identities, pooling, optimizer, batch sizes, early stopping, and hardware.) — supports /scope, /protocol, /model_ids
  • table: p. 9, Table 5 (7-vs-rest column) (Published held-out Spearman values; em dashes and NA entries are omitted rather than encoded as zeros.) — supports /metrics, /results
flip-aav-two-vs-rest-spearmanvoriginal-2021

Evaluated models / systems: Convolutional network (FLIP), ESM-untrained (mean), ESM-untrained (mut mean), ESM-untrained (per AA), ESM-1b (mean; FLIP head), ESM-1b (mut mean; FLIP head), ESM-1b (per AA; FLIP head), ESM-1v (mean; FLIP head), ESM-1v (mut mean; FLIP head), ESM-1v (per AA; FLIP head), Levenshtein distance baseline (FLIP), Ridge regression (FLIP)

Scopesubset · n=50776
ShotsNot applicable
TurnsNot applicable
System prompt publicNot applicable
Reasoning / effortNot applicable
BrowserNot applicable
InternetNot applicable
DatabasesNot applicable
Code executionNot applicable
ContainerNot reported
External toolsmodel-specific frozen embeddings or one-hot sequence encoding
Token budgetNot applicable
Time / cost budgetNot reported
TemperatureNot applicable
SeedNot reported
RepeatsNot reported
Graderdeterministic regression scorer · human review: no
StatisticsSpearman correlation over held-out test examples; the main table reports point values without confidence intervals.
ContaminationMutation-depth extrapolation above two changes.
Metrics, results, and full protocol

Metrics

MetricKind / baselineUnitAggregationThreshold / tolerance
Spearman correlationabsolutecorrelationcomputed across all examples in this held-out test splitNot reported
Mean squared errorabsolutesquared fitness unitsmean across all examples in this held-out test splitNot reported

Results

ModelMetricValuen
ESM-1b (per AA; FLIP head)Spearman correlation0.65 correlation
Held-out test-set Spearman from the creator paper.		50776
ESM-1b (mean; FLIP head)Spearman correlation0.26 correlation
Held-out test-set Spearman from the creator paper.		50776
ESM-1b (mut mean; FLIP head)Spearman correlation0.65 correlation
Held-out test-set Spearman from the creator paper.		50776
ESM-1v (per AA; FLIP head)Spearman correlation0.7 correlation
Held-out test-set Spearman from the creator paper.		50776
ESM-1v (mean; FLIP head)Spearman correlation0.16 correlation
Held-out test-set Spearman from the creator paper.		50776
ESM-1v (mut mean; FLIP head)Spearman correlation0.64 correlation
Held-out test-set Spearman from the creator paper.		50776
ESM-untrained (per AA)Spearman correlation0.22 correlation
Held-out test-set Spearman from the creator paper.		50776
ESM-untrained (mean)Spearman correlation0.14 correlation
Held-out test-set Spearman from the creator paper.		50776
ESM-untrained (mut mean)Spearman correlation0.16 correlation
Held-out test-set Spearman from the creator paper.		50776
Ridge regression (FLIP)Spearman correlation0.03 correlation
Held-out test-set Spearman from the creator paper.		50776
Convolutional network (FLIP)Spearman correlation0.74 correlation
Held-out test-set Spearman from the creator paper.		50776
Levenshtein distance baseline (FLIP)Spearman correlation0.57 correlation
Held-out test-set Spearman from the creator paper.		50776

Evidence

  • table: pp. 4–7, Table 2 and Sections 3–4 (Exact train/test counts, split rule, baseline identities, pooling, optimizer, batch sizes, early stopping, and hardware.) — supports /scope, /protocol, /model_ids
  • table: p. 9, Table 5 (2-vs-rest column) (Published held-out Spearman values; em dashes and NA entries are omitted rather than encoded as zeros.) — supports /metrics, /results

Comparable result views

Spearman correlation

flip-aav-des-mut · flip-aav-des-mut-spearman

CSV ↓
Accessible data table
ModelValueComparability group
ESM-1b (mean; FLIP head)0.59flip-aav-des-mut-spearman
ESM-1b (mut mean; FLIP head)0.7flip-aav-des-mut-spearman
ESM-1v (mean; FLIP head)0.44flip-aav-des-mut-spearman
ESM-1v (mut mean; FLIP head)0.71flip-aav-des-mut-spearman
ESM-untrained (mean)0.34flip-aav-des-mut-spearman
ESM-untrained (mut mean)0.64flip-aav-des-mut-spearman
Ridge regression (FLIP)0.53flip-aav-des-mut-spearman
Convolutional network (FLIP)0.75flip-aav-des-mut-spearman
Levenshtein distance baseline (FLIP)-0.07flip-aav-des-mut-spearman

Spearman correlation

flip-aav-low-vs-high · flip-aav-low-vs-high-spearman

CSV ↓
Accessible data table
ModelValueComparability group
ESM-1b (per AA; FLIP head)0.39flip-aav-low-vs-high-spearman
ESM-1b (mean; FLIP head)0.18flip-aav-low-vs-high-spearman
ESM-1b (mut mean; FLIP head)0.33flip-aav-low-vs-high-spearman
ESM-1v (per AA; FLIP head)0.34flip-aav-low-vs-high-spearman
ESM-1v (mean; FLIP head)0.2flip-aav-low-vs-high-spearman
ESM-1v (mut mean; FLIP head)0.31flip-aav-low-vs-high-spearman
ESM-untrained (per AA)0.08flip-aav-low-vs-high-spearman
ESM-untrained (mean)0.22flip-aav-low-vs-high-spearman
ESM-untrained (mut mean)0.24flip-aav-low-vs-high-spearman
Ridge regression (FLIP)0.12flip-aav-low-vs-high-spearman
Convolutional network (FLIP)0.34flip-aav-low-vs-high-spearman
Levenshtein distance baseline (FLIP)0.25flip-aav-low-vs-high-spearman

Spearman correlation

flip-aav-mut-des · flip-aav-mut-des-spearman

CSV ↓
Accessible data table
ModelValueComparability group
ESM-1b (per AA; FLIP head)0.76flip-aav-mut-des-spearman
ESM-1b (mean; FLIP head)0.63flip-aav-mut-des-spearman
ESM-1b (mut mean; FLIP head)0.7flip-aav-mut-des-spearman
ESM-1v (per AA; FLIP head)0.79flip-aav-mut-des-spearman
ESM-1v (mean; FLIP head)0.55flip-aav-mut-des-spearman
ESM-1v (mut mean; FLIP head)0.7flip-aav-mut-des-spearman
ESM-untrained (per AA)0.56flip-aav-mut-des-spearman
ESM-untrained (mean)0.27flip-aav-mut-des-spearman
ESM-untrained (mut mean)0.62flip-aav-mut-des-spearman
Ridge regression (FLIP)0.64flip-aav-mut-des-spearman
Convolutional network (FLIP)0.71flip-aav-mut-des-spearman
Levenshtein distance baseline (FLIP)0.6flip-aav-mut-des-spearman

Spearman correlation

flip-aav-one-vs-rest · flip-aav-one-vs-rest-spearman

CSV ↓
Accessible data table
ModelValueComparability group
ESM-1b (per AA; FLIP head)0.03flip-aav-one-vs-rest-spearman
ESM-1b (mean; FLIP head)0.04flip-aav-one-vs-rest-spearman
ESM-1b (mut mean; FLIP head)0.31flip-aav-one-vs-rest-spearman
ESM-1v (per AA; FLIP head)0.1flip-aav-one-vs-rest-spearman
ESM-1v (mean; FLIP head)0.18flip-aav-one-vs-rest-spearman
ESM-1v (mut mean; FLIP head)0.44flip-aav-one-vs-rest-spearman
ESM-untrained (per AA)0.18flip-aav-one-vs-rest-spearman
ESM-untrained (mean)0.01flip-aav-one-vs-rest-spearman
ESM-untrained (mut mean)0.26flip-aav-one-vs-rest-spearman
Ridge regression (FLIP)0.22flip-aav-one-vs-rest-spearman
Convolutional network (FLIP)0.48flip-aav-one-vs-rest-spearman
Levenshtein distance baseline (FLIP)-0.11flip-aav-one-vs-rest-spearman

Spearman correlation

flip-aav-sampled · flip-aav-sampled-spearman

CSV ↓
Accessible data table
ModelValueComparability group
ESM-1b (per AA; FLIP head)0.9flip-aav-sampled-spearman
ESM-1v (per AA; FLIP head)0.92flip-aav-sampled-spearman
ESM-untrained (per AA)0.78flip-aav-sampled-spearman
Ridge regression (FLIP)0.83flip-aav-sampled-spearman
Convolutional network (FLIP)0.92flip-aav-sampled-spearman

Spearman correlation

flip-aav-seven-vs-rest · flip-aav-seven-vs-rest-spearman

CSV ↓
Accessible data table
ModelValueComparability group
ESM-1b (per AA; FLIP head)0.65flip-aav-seven-vs-rest-spearman
ESM-1b (mean; FLIP head)0.46flip-aav-seven-vs-rest-spearman
ESM-1b (mut mean; FLIP head)0.61flip-aav-seven-vs-rest-spearman
ESM-1v (per AA; FLIP head)0.7flip-aav-seven-vs-rest-spearman
ESM-1v (mean; FLIP head)0.45flip-aav-seven-vs-rest-spearman
ESM-1v (mut mean; FLIP head)0.64flip-aav-seven-vs-rest-spearman
ESM-untrained (per AA)0.42flip-aav-seven-vs-rest-spearman
ESM-untrained (mean)0.22flip-aav-seven-vs-rest-spearman
ESM-untrained (mut mean)0.56flip-aav-seven-vs-rest-spearman
Ridge regression (FLIP)0.65flip-aav-seven-vs-rest-spearman
Convolutional network (FLIP)0.74flip-aav-seven-vs-rest-spearman
Levenshtein distance baseline (FLIP)0.53flip-aav-seven-vs-rest-spearman

Spearman correlation

flip-aav-two-vs-rest · flip-aav-two-vs-rest-spearman

CSV ↓
Accessible data table
ModelValueComparability group
ESM-1b (per AA; FLIP head)0.65flip-aav-two-vs-rest-spearman
ESM-1b (mean; FLIP head)0.26flip-aav-two-vs-rest-spearman
ESM-1b (mut mean; FLIP head)0.65flip-aav-two-vs-rest-spearman
ESM-1v (per AA; FLIP head)0.7flip-aav-two-vs-rest-spearman
ESM-1v (mean; FLIP head)0.16flip-aav-two-vs-rest-spearman
ESM-1v (mut mean; FLIP head)0.64flip-aav-two-vs-rest-spearman
ESM-untrained (per AA)0.22flip-aav-two-vs-rest-spearman
ESM-untrained (mean)0.14flip-aav-two-vs-rest-spearman
ESM-untrained (mut mean)0.16flip-aav-two-vs-rest-spearman
Ridge regression (FLIP)0.03flip-aav-two-vs-rest-spearman
Convolutional network (FLIP)0.74flip-aav-two-vs-rest-spearman
Levenshtein distance baseline (FLIP)0.57flip-aav-two-vs-rest-spearman

Evidence and change history

Source locators remain visible; expand an item to inspect the exact Registry fields it supports.

FLIP: Benchmark tasks in fitness landscape inference for proteins · table: pp. 4, 6, 9; Table 2, Section 3.2, Tables 5 and 7 (AAV definition, seven splits, exact train/test counts, regression setup, and baseline results.) · Supports 26 fields

Open source →

  • /name
  • /summary
  • /kind
  • /organizations
  • /release_date
  • /latest_version
  • /domains
  • /capabilities
  • /modalities
  • /task_formats
  • /task_counts/total
  • /task_counts/basis
  • /task_counts/subsets
  • /coverage_notes
  • /access/level
  • /access/tasks
  • /access/artifacts
  • /access/grader
  • /access/license
  • /access/biosafety_notes
  • /versions/0/release_date
  • /versions/0/as_of
  • /versions/0/task_counts/total
  • /versions/0/task_counts/basis
  • /versions/0/task_counts/subsets
  • /scientific_task_classification/entries/0
flip-aav-repository-resource · repository-path: splits/aav/README.md and splits/aav/splits.zip at commit 62cace8735f5610e2743cf06ce0f944b37fffaa6 (Confirms regression/binary files, active versus sampled status, access, and AFL-3.0 derivative license.) · Supports 5 fields

Open source →

  • /access/tasks
  • /access/artifacts
  • /access/license
  • /resources
  • /implementations

View source-level modification history on GitHub →