paper · benchmark creator

An integrative approach to protein sequence design through multiobjective optimization

University of California, San Francisco · Quantitative Biosciences Institute · Chan Zuckerberg Biohub · 2024-07-11

Relationship layer

Benchmark usage

This table records what the work did with each benchmark before attempting to normalize a run. Partial claims remain visible without being treated as comparable evaluations.

benchmark creation

an-integrative-approach-to-protein-sequence-design-thr-cam-benchmark-5-use

Benchmark: CaM benchmark · version Not reported

Selection: not applicable
Models: Not reported / not applicable
Metrics: Not reported / not applicable
Linked runs: None

Not reported / unresolved: No formal benchmark version or release identifier is reported.; The repository and benchmark-data license is not reported.

AI-assisted double-pass extraction; values are limited to independently supported claims.

Evidence

section: Results — Genetic algorithms can be applied to higher-dimensional design problems
Supports: /relation_type
section: Methods — Structure preparation
Supports: /benchmark_id

evaluation

an-integrative-approach-to-protein-sequence-design-thr-cam-benchmark-6-use

Benchmark: CaM benchmark · version Not reported

Selection: not reported · Table 1's fourteen preprocessed CaM structures; NSGA-III with mutation rates 0.1, 0.3, and 0.5 and reference-direction count equal to population size.
Models: AlphaFold2, ESM-1v, ProteinMPNN
Metrics: native sequence recovery, ESM-1v log likelihood score, pMPNN-SD negative log likelihood score, AF2Rank composite score, pMPNN-SD log likelihood score hypervolume, AF2Rank composite score hypervolume, per-position sequence entropy
Linked runs: None

Not reported / unresolved: realized n/scope; Exact primary metric values are not printed in body text or tables; plotted values are unlabeled.; The starting random seed is not reported.; Confidence intervals and statistical uncertainty are not reported.; Exact release dates for the evaluated models are not reported.; benchmark version; numeric result

AI-assisted double-pass extraction; values are limited to independently supported claims.

Evidence

section: Results — GA[AF2Rank] recapitulates the CaM charge profile
Supports: /relation_type
section: Methods — Structure preparation
Supports: /benchmark_id
section: Methods — Structure preparation
Supports: /scope
section: Methods — Structure preparation
Supports: /scope
section: Methods — Designable positions
Supports: /scope
section: Methods — Structure preparation; Genetic algorithm; Results — Genetic algorithms can be applied to higher-dimensional design problems
Supports: /scope
section: Methods — ProteinMPNN; reference 20
Supports: /model_ids
section: Methods — ESM-1v; reference 27
Supports: /model_ids
section: Methods — AF2Rank; reference 14
Supports: /model_ids
section: Methods — Sequence analysis
Supports: /metric_labels
figure: Fig. 5
Supports: /metric_labels
section: Methods — ProteinMPNN; Genetic algorithm
Supports: /metric_labels
section: Methods — AF2Rank
Supports: /metric_labels
section: Fig. 5; Methods — Hypervolume
Supports: /metric_labels
section: Fig. 5; Methods — Hypervolume
Supports: /metric_labels
section: Results — GA[AF2Rank] recapitulates the CaM charge profile; Figs. 5 and 3
Supports: /metric_labels

benchmark creation

an-integrative-approach-to-protein-sequence-design-thr-papd-benchmark-3-use

Benchmark: PapD benchmark · version Not reported

Selection: not applicable
Models: Not reported / not applicable
Metrics: Not reported / not applicable
Linked runs: None

Not reported / unresolved: No formal benchmark version or release identifier is reported.; The repository and benchmark-data license is not reported.

AI-assisted double-pass extraction; values are limited to independently supported claims.

Evidence

section: Results — Genetic algorithms can be applied to higher-dimensional design problems
Supports: /relation_type
section: Methods — Structure preparation
Supports: /benchmark_id

evaluation

an-integrative-approach-to-protein-sequence-design-thr-papd-benchmark-4-use

Benchmark: PapD benchmark · version Not reported

Selection: not reported · Preprocessed 1N0L, 1PDK, and 1QPP structures; NSGA-II with mutation rates 0.1, 0.3, and 0.5 and otherwise unchanged RfaH hyperparameters.
Models: AlphaFold2, ESM-1v, ProteinMPNN
Metrics: native sequence recovery, ESM-1v log likelihood score, pMPNN-SD negative log likelihood score, AF2Rank composite score, pMPNN-SD log likelihood score hypervolume, AF2Rank composite score hypervolume, per-position sequence entropy
Linked runs: None

AI-assisted double-pass extraction; values are limited to independently supported claims.

Evidence

section: Results — Genetic algorithms can be applied to higher-dimensional design problems
Supports: /relation_type
section: Methods — Structure preparation
Supports: /benchmark_id
section: Results — Genetic algorithms can be applied to higher-dimensional design problems
Supports: /scope
figure: Fig. 1
Supports: /scope
section: Methods — Designable positions
Supports: /scope
section: Methods — Structure preparation; Results — Genetic algorithms can be applied to higher-dimensional design problems
Supports: /scope
section: Methods — ProteinMPNN; reference 20
Supports: /model_ids
section: Methods — ESM-1v; reference 27
Supports: /model_ids
section: Methods — AF2Rank; reference 14
Supports: /model_ids
section: Methods — Sequence analysis
Supports: /metric_labels
figure: Fig. 5
Supports: /metric_labels
section: Methods — ProteinMPNN; Genetic algorithm
Supports: /metric_labels
section: Methods — AF2Rank
Supports: /metric_labels
section: Fig. 5; Methods — Hypervolume
Supports: /metric_labels
section: Fig. 5; Methods — Hypervolume
Supports: /metric_labels
section: Results — Genetic algorithms can be applied to higher-dimensional design problems; Figs. 5 and 3
Supports: /metric_labels

benchmark creation

an-integrative-approach-to-protein-sequence-design-thr-rfah-benchmark-1-use

Benchmark: RfaH benchmark · version Not reported

Selection: not applicable
Models: Not reported / not applicable
Metrics: Not reported / not applicable
Linked runs: None

Not reported / unresolved: No formal benchmark version or release identifier is reported.; The repository and benchmark-data license is not reported.

AI-assisted double-pass extraction; values are limited to independently supported claims.

Evidence

section: Introduction; Results — The random resetting mutation operator results in slow convergence
Supports: /relation_type
section: Methods — Hypervolume
Supports: /benchmark_id

evaluation

an-integrative-approach-to-protein-sequence-design-thr-rfah-benchmark-2-use

Benchmark: RfaH benchmark · version Not reported

Selection: not reported · Preprocessed 5OND and the first 2LCL model; NSGA-II with two-point crossover, random/ESM position selection, and pMPNN-AD or uniform mutation.
Models: AlphaFold2, ESM-1v, ProteinMPNN
Metrics: native sequence recovery, ESM-1v log likelihood score, pMPNN-SD negative log likelihood score, AF2Rank composite score, HV[pMPNN], HV[AF2Rank], per-position sequence entropy, normalized BLOSUM62 sequence similarity
Linked runs: None

Not reported / unresolved: realized n/scope; Exact primary metric values are not printed in body text or tables; plotted values are unlabeled.; Confidence intervals and statistical uncertainty are not reported.; Exact release dates for the evaluated models are not reported.; benchmark version; numeric result

AI-assisted double-pass extraction; values are limited to independently supported claims.

Evidence

figure: Fig. 2
Supports: /relation_type
figure: S2 Fig.
Supports: /benchmark_id
section: Results — The random resetting mutation operator results in slow convergence
Supports: /scope
figure: Fig. 1
Supports: /scope
section: Methods — Designable positions
Supports: /scope
section: Methods — Structure preparation; Genetic algorithm
Supports: /scope
section: Methods — ProteinMPNN; reference 20
Supports: /model_ids
section: Methods — ESM-1v; reference 27
Supports: /model_ids
section: Methods — AF2Rank; reference 14
Supports: /model_ids
section: Methods — Sequence analysis
Supports: /metric_labels
figure: Fig. 2
Supports: /metric_labels
section: Methods — ProteinMPNN; Genetic algorithm
Supports: /metric_labels
section: Methods — AF2Rank
Supports: /metric_labels
figure: Fig. 2
Supports: /metric_labels
figure: Fig. 2
Supports: /metric_labels
figure: Fig. 3; Results — The random resetting mutation operator results in slow convergence
Supports: /metric_labels
section: Methods — Sequence analysis
Supports: /metric_labels

Normalized evaluation runs

This source has no normalized model run. It may be a creator-only source or a partial/non-evaluation benchmark use.

Source record

Status: published
Published: 2024-07-11
Authors: Lu Hong, Tanja Kortemme
DOI: 10.1371/journal.pcbi.1011953
arXiv: Not reported
Current source version: an-integrative-approach-to-protein-sequence-design-thr-pmc-version-1
Source access: open-url
Content SHA256: 8a6d1008e88beef51751dafeafed80b607afcacb081a94f18de3f8d3d9d39814
Retrieved: 2026-07-31T05:54:17+00:00

Review provenance

AI-assisted owner-reviewed

Method: Local Codex double-pass
Surface: local-codex-cli
Pipeline: 1.4.0
Prompt: paper-evidence-local-v15
Extractor: Not reported
requested gpt-5.6-sol
Verifier: Not reported
requested gpt-5.6-sol
Model resolution: not-reported
Codex CLI: codex-cli 0.146.0-alpha.9.2
Local run: 1d7378b3-42fd-42ba-a2c3-9d90f8335349
Generated: 2026-07-31T05:54:17+00:00

“Owner-reviewed” means this production record passed the required exact-head-SHA approval gate; deterministic code—not the model—authored Registry YAML.

Source versions

PMC version 1
an-integrative-approach-to-protein-sequence-design-thr-pmc-version-1 · version-of-record · 2024-07-11

Open canonical source ↗