ADMET Benchmark Group

ADMET is a cornerstone of small molecule drug discovery, defining drug efficacy and toxicity profile. An ML model that could accurately predict all ADMET properties using structural information of compounds would be greatly valuable.

We formulate the ADMET Benchmark Group using 22 ADMET datasets in TDC. The ADMET Group contains the following datasets:

from tdc import utils
names = utils.retrieve_benchmark_names('ADMET_Group')
# ['caco2_wang', 'hia_hou', ....]

Type the following to access any benchmark in the group, for example, Caco2_Wang:

from tdc.benchmark_group import admet_group
group = admet_group(path = 'data/')
benchmark = group.get('Caco2_Wang')

predictions = {}
name = benchmark['name']
train_val, test = benchmark['train_val'], benchmark['test']

## --- train your model --- ##

predictions[name] = y_pred
group.evaluate(predictions)
# {'caco2_wang': {'mae': 0.234}}

Follow the instructions on how to use the BenchmarkGroup class and obtain training, validation, and test sets, and how to submit your model to the leaderboard.

For every dataset in the benchmark group, we use the scaffold split to partition the dataset into training, validation, and test sets. We hold out 20% data samples for the test set. The performance metrics are:

• For binary classification:
• AUROC is used when the number of positive and negative samples are similar.
• AUPRC is used when the number of positive samples are much smaller than negative samples.
• For regression:
• MAE is used for majority of benchmarks.
• Spearman's correlation coefficient is used for benchmarks that depend on factors beyond the chemical structure.

We encourage submissions that reports results for the entire benchmark group. Still, we welcome and accept submissions that report partial results, for example, submissions with results for just one out of five ADMET categories.

Benchmark Data Summary

Absorption

Absorption measures how a drug travels from the site of administration to site of action.

Distribution

Drug distribution refers to how drug moves to and from the various tissues of the body and the amount of drugs in the tissues.

Metabolism

Drug metabolism measures how specialized enzymatic systems breakdown the drugs and it determines the duration and intensity of a drug's action.

Excretion

Drug excretion is the removal of drugs from the body using various different routes of excretion, including urine, bile, sweat, saliva, tears, milk, and stool.

Toxicity

Toxicity measures how much damage a drug could cause to organisms.