dataset · audited-with-caveats · verified 2026-07-31
CaM benchmark
A multistate protein sequence-design benchmark spanning CaM conformations and binding modes.
+1 more
Audited with caveats: 2 field(s) are marked provisional or conflicted. Warnings are shown next to affected values and these claims are excluded from unqualified comparisons.
Benchmark definition
What is counted
- Version
- initial-release
- Total
- 14 (Fig. 1B explicitly defines the complete CaM design problem as fourteen-state.)
- Task formats
- unclassified
- Capabilities
DesignOptimization
- Modalities
Protein sequence3D structure
Version history
| Version | Status | Release / as-of | Total | Formal tracks |
|---|
initial-release
cam-benchmark-initial-release-version | current | 2024-07-11 | 14 (Fig. 1B explicitly defines the complete CaM design problem as fourteen-state.) | None registered |
Scientific Task Atlas
Scientific task classification
partial for initial-release. The creator paper explicitly frames this benchmark system as multistate protein sequence design; more specific downstream design objectives are not exhaustively classified.
| Scientific task | Coverage | Count | Mapping | Evidence |
|---|
| Protein sequence design | explicitly-in-scope | Not reported
Creator-defined multistate protein sequence-design benchmark system. | official-taxonomy
high confidence | cam-benchmark-automated-metadata-2-evidence
The fourteen modeled structural states are objective dimensions, not fourteen independent sequence-design tasks. |
Relationship registry
How works use this benchmark
Partial claims, non-evaluation uses, and third-party summaries stay visible without entering model comparisons.
Partial evaluation claims
evaluation
an-integrative-approach-to-protein-sequence-design-thr-cam-benchmark-6-use
Partialunknown
Work: An integrative approach to protein sequence design through multiobjective optimization · source version an-integrative-approach-to-protein-sequence-design-thr-pmc-version-1
- Selection
- not reported · Table 1's fourteen preprocessed CaM structures; NSGA-III with mutation rates 0.1, 0.3, and 0.5 and reference-direction count equal to population size.
- Metrics
- native sequence recovery, ESM-1v log likelihood score, pMPNN-SD negative log likelihood score, AF2Rank composite score, pMPNN-SD log likelihood score hypervolume, AF2Rank composite score hypervolume, per-position sequence entropy
- Linked runs
- None
Not reported / unresolved: realized n/scope; Exact primary metric values are not printed in body text or tables; plotted values are unlabeled.; The starting random seed is not reported.; Confidence intervals and statistical uncertainty are not reported.; Exact release dates for the evaluated models are not reported.; benchmark version; numeric result
AI-assisted double-pass extraction; values are limited to independently supported claims.
Evidence
- section: Results — GA[AF2Rank] recapitulates the CaM charge profile
Supports: /relation_type - section: Methods — Structure preparation
Supports: /benchmark_id - section: Methods — Structure preparation
Supports: /scope - section: Methods — Structure preparation
Supports: /scope - section: Methods — Designable positions
Supports: /scope - section: Methods — Structure preparation; Genetic algorithm; Results — Genetic algorithms can be applied to higher-dimensional design problems
Supports: /scope - section: Methods — ProteinMPNN; reference 20
Supports: /model_ids - section: Methods — ESM-1v; reference 27
Supports: /model_ids - section: Methods — AF2Rank; reference 14
Supports: /model_ids - section: Methods — Sequence analysis
Supports: /metric_labels - figure: Fig. 5
Supports: /metric_labels - section: Methods — ProteinMPNN; Genetic algorithm
Supports: /metric_labels - section: Methods — AF2Rank
Supports: /metric_labels - section: Fig. 5; Methods — Hypervolume
Supports: /metric_labels - section: Fig. 5; Methods — Hypervolume
Supports: /metric_labels - section: Results — GA[AF2Rank] recapitulates the CaM charge profile; Figs. 5 and 3
Supports: /metric_labels
Creation, training, validation, or model-selection uses
benchmark creation
an-integrative-approach-to-protein-sequence-design-thr-cam-benchmark-5-use
Non-evaluationunknown
Work: An integrative approach to protein sequence design through multiobjective optimization · source version an-integrative-approach-to-protein-sequence-design-thr-pmc-version-1
- Selection
- not applicable
- Models
- Not reported / not applicable
- Metrics
- Not reported / not applicable
- Linked runs
- None
Not reported / unresolved: No formal benchmark version or release identifier is reported.; The repository and benchmark-data license is not reported.
AI-assisted double-pass extraction; values are limited to independently supported claims.
Evidence
- section: Results — Genetic algorithms can be applied to higher-dimensional design problems
Supports: /relation_type - section: Methods — Structure preparation
Supports: /benchmark_id
Evaluation registry
Works and run settings
A setting change—scope, prompt, tools, budget, grader, or repeats—creates a separate run. Charts never cross a comparability group.
No normalized evaluation run is published yet. Creator evidence is still attached below.
Evidence and change history
Source locators remain visible; expand an item to inspect the exact Registry fields it supports.
An integrative approach to protein sequence design through multiobjective optimization · section: Data Availability · Supports 1 field
Open source →
An integrative approach to protein sequence design through multiobjective optimization · section: Results — Genetic algorithms can be applied to higher-dimensional design problems · Supports 2 fields
Open source →
/capabilities/scientific_task_classification/entries/0
An integrative approach to protein sequence design through multiobjective optimization · section: Results — The random resetting mutation operator results in slow convergence; Genetic algorithms can be applied to higher-dimensional design problems · Supports 1 field
Open source →
An integrative approach to protein sequence design through multiobjective optimization · section: Data Availability · Supports 1 field
Open source →
An integrative approach to protein sequence design through multiobjective optimization · figure: Fig. 1 · Supports 1 field
Open source →
An integrative approach to protein sequence design through multiobjective optimization · section: Methods — Structure preparation · Supports 1 field
Open source →
An integrative approach to protein sequence design through multiobjective optimization · other: Front matter — author-affiliation mapping · Supports 1 field
Open source →
An integrative approach to protein sequence design through multiobjective optimization · section: Results — Genetic algorithms can be applied to higher-dimensional design problems · Supports 1 field
Open source →
An integrative approach to protein sequence design through multiobjective optimization · other: Crossref bibliographic metadata (Resolved from the canonical paper identifier during intake.) · Supports 1 field
Open source →
An integrative approach to protein sequence design through multiobjective optimization · figure: Fig. 1 · Supports 4 fields
Open source →
/task_counts/total/task_counts/basis/versions/0/task_counts/total/versions/0/task_counts/basis
An integrative approach to protein sequence design through multiobjective optimization · figure: Fig. 1 · Supports 2 fields
Open source →
/task_counts/subsets/versions/0/task_counts/subsets
An integrative approach to protein sequence design through multiobjective optimization · section: Data Availability · Supports 3 fields
Open source →
/resources/implementations/access/license
An integrative approach to protein sequence design through multiobjective optimization · other: Front matter — DOI · Supports 1 field
Open source →
An integrative approach to protein sequence design through multiobjective optimization · other: Front matter — DOI · Supports 2 fields
Open source →
/latest_version/versions/0
Unresolved field claims
/task_counts/subsetsProvisional · high — The double-pass review established the root item total but did not establish an exhaustive formal-subset inventory; the empty list must not be interpreted as evidence that the benchmark has no subsets.
Evidence: cam-benchmark-automated-subset-coverage-evidence/access/licenseProvisional · high — The double-pass review verified the official resource identity but did not establish a redistributable benchmark license; the value remains null pending source-level license verification.
Evidence: cam-benchmark-automated-resource-evidence
View source-level modification history on GitHub →