Benchmarks/CASP/CASP Protein-Ligand Prediction

track · audited · verified 2026-07-21

CASP Protein-Ligand Prediction

Formal CASP track for blind prediction of protein-ligand binding poses, binding affinity or rank, binding pockets, and pose confidence.

Benchmark definition

What is counted

Version: CASP17
Total: Not reported (CASP17 evaluated protein-ligand targets)
Task formats: blind protein-ligand pose prediction; blind binding affinity prediction
Capabilities: PredictionRegression
Modalities: Protein sequence3D structure

Version history

Version	Status	Release / as-of	Total	Formal tracks
CASP16 `casp-ligand-casp16`	superseded	2024-05-01	Not reported (evaluated protein-ligand cases across overlapping pose and affinity tasks)	None registered
CASP17 `casp-ligand-casp17`	rolling	2026-04-27	Not reported (evaluated protein-ligand targets)	None registered

Scientific Task Atlas

Scientific task classification

complete for CASP17 · as of 2026-07-21. Dedicated protein-ligand track; current target counts are not final.

Scientific task	Coverage	Count	Mapping	Evidence
Protein-ligand pose prediction	explicitly-in-scope	Not reported CASP17 protein-ligand targets.	official-track high confidence	`casp-ligand-evidence-casp17` Pose and pocket prediction are explicit objectives.
Protein-ligand binding affinity	explicitly-in-scope	Not reported CASP17 protein-ligand targets.	official-track high confidence	`casp-ligand-evidence-casp17` Affinity or ranking is explicit, without a current standalone count.

Scientific coverage notes

Domain	Coverage	Count	Interpretation
Protein-ligand binding	explicitly-in-scope	Not reported	The dedicated CASP17 category is active, but the official in-numbers page does not yet publish a ligand-only target count.

Evaluation registry

Works and run settings

A setting change—scope, prompt, tools, budget, grader, or repeats—creates a separate run. Charts never cross a comparability group.

Evaluation run

casp16-ligand-affinity-stage1

From Assessment of Pharmaceutical Protein-Ligand Pose and Affinity Predictions in CASP16

Scopesubset · n=122

ShotsNot applicable

TurnsNot applicable

System prompt publicNot applicable

Reasoning / effortNot applicable

Browsermethod-specific

Internetmethod-specific

Databasesmethod-specific

Code executionallowed

ContainerNot reported

External toolsmethod-specific

Token budgetNot applicable

Time / cost budgettarget-specific CASP submission windows before experimental poses were released

TemperatureNot applicable

SeedNot reported

RepeatsModel 1 affinity submission per group and case

Graderofficial affinity ranking analysis plus independent ligand assessor team · human review: yes

StatisticsN-weighted Kendall tau across the two protein systems after disclosure filtering

Contaminationexperimental poses and affinities withheld, with 18 previously disclosed affinities removed from analysis

Metrics, results, and full protocol

Metrics

Metric	Kind / baseline	Unit	Aggregation	Threshold / tolerance
N-weighted Kendall's tau	absolute	rank correlation	case-count-weighted mean of per-protein-system Kendall tau after disclosure filtering	Not reported

No numeric result rows are published yet; the verified protocol remains useful.

Evidence

section: Sections 2.2–2.5 and 3.3; Figure 8A (Original 140 cases, 18 disclosure exclusions, Stage-1 setting, Model 1, and N-weighted Kendall tau.) — supports /benchmark_version, /scope, /protocol, /metrics

Evaluation run

casp16-ligand-affinity-stage2

From Assessment of Pharmaceutical Protein-Ligand Pose and Affinity Predictions in CASP16

Scopesubset · n=103

ShotsNot applicable

TurnsNot applicable

System prompt publicNot applicable

Reasoning / effortNot applicable

Browsermethod-specific

Internetmethod-specific

Databasesmethod-specific

Code executionallowed

ContainerNot reported

External toolsmethod-specific

Token budgetNot applicable

Time / cost budget2024-08-07 through 2024-08-21 for the main Stage-2 challenge

TemperatureNot applicable

SeedNot reported

RepeatsModel 1 affinity submission per group and case

Graderofficial affinity ranking analysis plus independent ligand assessor team · human review: yes

Statisticsper-system Kendall tau and N-weighted cross-system summary after disclosure filtering

Contaminationexperimental co-crystal structures provided while experimental affinities remained withheld; disclosed affinities removed

Metrics, results, and full protocol

Metrics

Metric	Kind / baseline	Unit	Aggregation	Threshold / tolerance
N-weighted Kendall's tau	absolute	rank correlation	case-count-weighted mean of per-protein-system Kendall tau after disclosure filtering	Not reported

No numeric result rows are published yet; the verified protocol remains useful.

Evidence

section: Sections 2.2–2.5 and 3.3; Figures 8B-C and 9 (110 releases, 103 retained cases, co-crystal structure access, Stage-2 dates, and Kendall tau analysis.) — supports /benchmark_version, /scope, /protocol, /metrics

Evaluation run

casp16-ligand-pose-regular

From Assessment of Pharmaceutical Protein-Ligand Pose and Affinity Predictions in CASP16

Scopesubset · n=229

ShotsNot applicable

TurnsNot applicable

System prompt publicNot applicable

Reasoning / effortNot applicable

Browsermethod-specific

Internetmethod-specific

Databasesmethod-specific

Code executionallowed

ContainerNot reported

External toolsmethod-specific

Token budgetNot applicable

Time / cost budgettarget-specific CASP submission windows

TemperatureNot applicable

SeedNot reported

Repeatsup to five pose models per target

Graderofficial OpenStructure ligand and binding-pocket scoring plus independent ligand assessor team · human review: yes

Statisticsmean per-group pose accuracy with explicit handling of skipped targets; Model 1 is the primary ranking view

Contaminationexperimental complex poses withheld during Stage 1 prediction; structural template similarity analyzed post hoc

Metrics, results, and full protocol

Metrics

Metric	Kind / baseline	Unit	Aggregation	Threshold / tolerance
LDDT-PLI	absolute	proportion	mean across eligible pose targets with missing-prediction handling	CASP16 modified the CASP15 definition to penalize erroneous non-native contacts.
BiSyRMSD	absolute	angstrom	per pose then group summary	symmetry-corrected after binding-site superposition
LDDT-lp	absolute	proportion	per binding pocket then group summary	Not reported
BB-RMSD	absolute	angstrom	binding-site C-alpha RMSD then group summary	Not reported

No numeric result rows are published yet; the verified protocol remains useful.

Evidence

section: Abstract; Sections 2.1–2.5 and 3.1–3.2 (229 assessed pose cases, participant protocol, Model 1 handling, pose metrics, missing submissions, and separate baselines.) — supports /benchmark_version, /scope, /protocol, /metrics

Evidence and change history

Source locators remain visible; expand an item to inspect the exact Registry fields it supports.

Assessment of Pharmaceutical Protein-Ligand Pose and Affinity Predictions in CASP16 · section: Abstract; Sections 2.2–2.5 and 3.3; Figures 8–10 (Reports 229 pose targets, 140 original affinity targets, disclosure filtering to 122 Stage-1 and 103 Stage-2 analysis cases, five systems, and pose/affinity evaluation.) · Supports 15 fields

Open source →

/name
/aliases
/summary
/kind
/organizations
/domains
/capabilities
/modalities
/task_formats
/versions/0/task_counts/total
/versions/0/task_counts/basis
/versions/0/task_counts/subsets
/access/grader
/resources
/implementations

CASP16 official competition and results archive · section: CASP16 Timetable and Organic Ligands (Establishes the 2024 round start and official ligand category.) · Supports 2 fields

Open source →

/release_date
/versions/0/release_date

CASP17 official competition · section: Organic Ligand-Protein Complexes; Timetable; Assessment (Defines the active category and confirms that final evaluation is not yet available.) · Supports 17 fields

Open source →

/latest_version
/task_counts/total
/task_counts/basis
/task_counts/subsets
/coverage_notes
/versions/1/release_date
/versions/1/as_of
/versions/1/task_counts/total
/versions/1/task_counts/basis
/versions/1/task_counts/subsets
/access/level
/access/tasks
/access/artifacts
/access/license
/access/biosafety_notes
/scientific_task_classification/entries/0
/scientific_task_classification/entries/1

View source-level modification history on GitHub →

Access & reproducibility

Access: Partially open
License: CC BY 4.0 for Prediction Center website content; pharmaceutical datasets and linked structures may have separate terms
Grader: OpenStructure ligand and pocket metrics plus independent ligand-assessor analysis.
Implementations: 1 recorded · official