suite · audited · verified 2026-07-22

GuacaMol

A reproducible benchmark for de novo molecular design with five distribution-learning tests and twenty goal-directed generation problems in the current v2 suite.

Benchmark definition

What is counted

Version: suite-v2
Total: 25 (formal benchmark problems)
Task formats: unconditional molecular generation; goal-directed molecular optimization
Capabilities: GenerationOptimization
Modalities: Small-molecule structure

Version history

Version	Status	Release / as-of	Total	Formal tracks
suite-v2 `guacamol-suite-v2`	current	2018-11-23	25 (formal benchmark problems)	None registered

Tracks and subsets

ID	Count	Basis	Partition?	Notes
Distribution-learning benchmarks `guacamol-distribution-learning`	5	formal benchmark problems	Exclusive & exhaustive	Validity, uniqueness, novelty, KL divergence, and Fréchet ChemNet Distance.
Goal-directed v2 benchmarks `guacamol-goal-directed-v2`	20	formal benchmark problems	Exclusive & exhaustive	Rediscovery, similarity, isomer, median-molecule, and multi-property optimization problems.

Scientific Task Atlas

Scientific task classification

complete for suite-v2. Version 2 of the official benchmark suite defines five distribution-learning and twenty goal-directed problems.

Scientific task	Coverage	Count	Mapping	Evidence
Small-molecule generation	explicitly-in-scope	25 problems Formal benchmark problems.	official-taxonomy high confidence	`guacamol-paper-definition-evidence` `guacamol-repository-evidence` Includes both distribution-learning assessment and goal-directed molecular optimization problems.

Scientific coverage notes

Domain	Coverage	Count	Interpretation
Medicinal chemistry	explicitly-in-scope	25	Counts formal evaluation problems, not generated molecules or scoring-function calls.

Evaluation registry

Works and run settings

A setting change—scope, prompt, tools, budget, grader, or repeats—creates a separate run. Charts never cross a comparability group.

Evaluation run

guacamol-creator-full

From GuacaMol: Benchmarking Models for de Novo Molecular Design

Scopefull · n=25

ShotsNot applicable

TurnsNot applicable

System prompt publicNot applicable

Reasoning / effortNot applicable

BrowserNot applicable

InternetNot applicable

DatabasesNot applicable

Code executionNot applicable

Containerofficial Dockerfile

External toolsRDKit 2018.09.1 or newer and FCD 1.1

Token budgetNot applicable

Time / cost budgetNot reported

TemperatureNot reported

SeedNot reported

RepeatsNot reported

Graderdeterministic chemistry scoring functions · human review: no

StatisticsEach formal benchmark returns a normalized score; no registry-wide sum is created.

ContaminationStandardized ChEMBL training data exclude a designated holdout set and highly similar molecules.

Metrics, results, and full protocol

Metrics

Metric	Kind / baseline	Unit	Aggregation	Threshold / tolerance
Validity	absolute	proportion	generated sample	Not reported
Uniqueness	absolute	proportion	generated sample	Not reported
Novelty	absolute	proportion	generated sample	Not reported
KL divergence score	absolute	normalized score	physicochemical descriptor distributions	Not reported
Fréchet ChemNet Distance score	absolute	normalized score	generated versus reference distribution	Not reported
Goal-directed benchmark score	absolute	normalized score	benchmark-specific top generated molecules	Not reported

No numeric result rows are published yet; the verified protocol remains useful.

Evidence

section: Sections 2-4 and Tables 1-2; official benchmark_suites.py v2 (Defines both modes, data standardization, baseline generators, and scoring. The pinned implementation establishes the current twenty-problem v2 list.) — supports /scope, /protocol, /metrics

Evidence and change history

Source locators remain visible; expand an item to inspect the exact Registry fields it supports.

GuacaMol: Benchmarking Models for de Novo Molecular Design · section: Sections 2-4 and Tables 1-2 (Defines distribution learning, goal-directed generation, standardized ChEMBL data, baseline evaluation, and score semantics.) · Supports 23 fields

Open source →

/name
/aliases
/summary
/kind
/organizations
/release_date
/latest_version
/domains
/capabilities
/modalities
/task_formats
/task_counts/total
/task_counts/basis
/task_counts/subsets
/coverage_notes
/access/level
/access/license
/resources
/versions/0/release_date
/versions/0/task_counts/total
/versions/0/task_counts/basis
/versions/0/task_counts/subsets
/scientific_task_classification/entries/0

guacamol-repository-resource · repository-path: README.md; guacamol/__init__.py; guacamol/benchmark_suites.py at 60ebe1f6a396f16e08b834dce448e9343d259feb (Confirms package version, five distribution benchmarks, twenty goal-directed v2 problems, public data hashes, and MIT license.) · Supports 13 fields

Open source →

/latest_version
/task_counts/total
/task_counts/subsets
/access/tasks
/access/artifacts
/access/grader
/access/license
/resources
/implementations
/versions/0/task_counts/total
/versions/0/task_counts/subsets
/versions/0/formal_tracks
/scientific_task_classification/entries/0

View source-level modification history on GitHub →

Access & reproducibility

Access: Fully open
License: MIT for GuacaMol code; the standardized molecular data derive from ChEMBL and retain applicable ChEMBL terms.
Grader: Deterministic validity, uniqueness, novelty, KL, FCD, and goal-directed scoring functions wit…
Implementations: 2 recorded · official

Access details

Tasks: Both benchmark suites, standardized ChEMBL-derived train/validation/test files, and the holdout set are public.
Artifacts: The official package provides scorers; a companion creator repository provides baselines and a Docker environment.
Grader: Deterministic validity, uniqueness, novelty, KL, FCD, and goal-directed scoring functions with scores normalized to higher-is-better.
Biosafety: The benchmark optimizes drug-like small molecules; this registry mirrors no molecular datasets or generated compounds.