ADMET Benchmark Group

ADMET is one of the cornerstones for small molecule drug discovery, where it defines the drug's efficacy and toxicity's profile. Thus, a machine learning model that can accurately predict all of the ADMET properties given the drug's structural information is highly valuable. We select 22 ADMET datasets from TDC's collection and formulate it as a benchmark group. To access, type:

from tdc import BenchmarkGroup
group = BenchmarkGroup(name = 'ADMET_Group', path = 'data/')
predictions = {}

for benchmark in group:
    name = benchmark['name']
    train_val, test = benchmark['train_val'], benchmark['test']
    ## --- train your model --- ##
    predictions[name] = y_pred_test

group.evaluate(predictions)
# {'caco2_wang': {'mae': 4.328}, 'hia_hou': {'roc-auc': 0.802}, ...}

Follow the instruction on how to use the BenchmarkGroup class and for other useful functions to facilitate your model building.

For every dataset, we use scaffold split and hold out 20% for test set. The evaluation metrics are selected given the following criteria:

  • For binary classification:
    • AUROC is used when the number of positive and negative samples are close.
    • AUPRC is used when the number of positive samples are much smaller than negative samples.
  • For regression:
    • MAE is used for majority of benchmarks.
    • Spearman's correlation coefficient is used for benchmarks that depend on factors beyond the chemical structure.

We encourage submissions that reports results for the entire benchmark group. Still, we welcome and accept submissions that report partial results, for example, for just one of the five ADMET categories.


Absorption

Absorption measures how a drug travels from the site of administration to site of action.

Summary

Dataset Unit Number Task Metric Split
Caco2 cm/s 906 Regression MAE Scaffold
HIA % 578 Binary AUROC Scaffold
Pgp % 1,212 Binary AUROC Scaffold
Bioav % 640 Binary AUROC Scaffold
Lipo log-ratio 4,200 Regression MAE Scaffold
AqSol log mol/L 9,982 Regression MAE Scaffold

Leaderboard

Rank Model Contact Link #Params Caco2 HIA Pgp Bioav Lipo AqSol
1 RDKit2D + MLP (DeepPurpose) Kexin Huang GitHub, Paper 633,409 0.393 ± 0.024 0.972 ± 0.008 0.918 ± 0.007 0.672 ± 0.021 0.574 ± 0.017 0.827 ± 0.047
2 CNN (DeepPurpose) Kexin Huang GitHub, Paper 226,625 0.446 ± 0.036 0.869 ± 0.026 0.908 ± 0.012 0.613 ± 0.013 0.743 ± 0.020 1.023 ± 0.023
3 Morgan + MLP (DeepPurpose) Kexin Huang GitHub, Paper 1,477,185 0.908 ± 0.060 0.807 ± 0.072 0.880 ± 0.006 0.581 ± 0.086 0.701 ± 0.009 1.203 ± 0.019

: The higher the better. : The lower the better. : Click to sort models for specific benchmark column.


Distribution

Drug distribution refers to how drug moves to and from the various tissues of the body and the amount of drugs in the tissues.

Summary

Dataset Unit Number Task Metric Split
BBB % 1,975 Binary AUROC Scaffold
PPBR % 1,797 Regression MAE Scaffold
VDss L/kg 1,130 Regression Spearman Scaffold

Leaderboard

Rank Model Contact Link #Params BBB PPBR VD
1 RDKit2D + MLP (DeepPurpose) Kexin Huang GitHub, Paper 633,409 0.889 ± 0.016 9.994 ± 0.319 0.561 ± 0.025
2 Morgan + MLP (DeepPurpose) Kexin Huang GitHub, Paper 1,477,185 0.823 ± 0.015 12.848 ± 0.362 0.493 ± 0.011
3 CNN (DeepPurpose) Kexin Huang GitHub, Paper 226,625 0.781 ± 0.030 11.106 ± 0.358 0.226 ± 0.114

: The higher the better. : The lower the better. : Click to sort models for specific benchmark column.


Metabolism

Drug metabolism measures how specialized enzymatic systems breakdown the drugs and it determines the duration and intensity of a drug's action.

Summary

Dataset Unit Number Task Metric Split
CYP2C9 Inhibition % 12,092 Binary AUPRC Scaffold
CYP2D6 Inhibition % 13,130 Binary AUPRC Scaffold
CYP3A4 Inhibition % 12,328 Binary AUPRC Scaffold
CYP2C9 Substrate % 666 Binary AUPRC Scaffold
CYP2D6 Substrate % 664 Binary AUPRC Scaffold
CYP3A4 Substrate % 667 Binary AUROC Scaffold

Leaderboard

Rank Model Contact Link #Params CYP2C9 Inhibition CYP2D6 Inhibition CYP3A4 Inhibition CYP2C9 Substrate CYP2D6 Substrate CYP3A4 Substrate
1 RDKit2D + MLP (DeepPurpose) Kexin Huang GitHub, Paper 633,409 0.742 ± 0.006 0.616 ± 0.007 0.829 ± 0.007 0.360 ± 0.040 0.677 ± 0.047 0.639 ± 0.012
2 Morgan + MLP (DeepPurpose) Kexin Huang GitHub, Paper 1,477,185 0.715 ± 0.004 0.587 ± 0.011 0.827 ± 0.009 0.380 ± 0.015 0.671 ± 0.066 0.633 ± 0.013
3 CNN (DeepPurpose) Kexin Huang GitHub, Paper 226,625 0.713 ± 0.006 0.544 ± 0.053 0.821 ± 0.003 0.367 ± 0.059 0.485 ± 0.037 0.662 ± 0.031

: The higher the better. : The lower the better. : Click to sort models for specific benchmark column.


Excretion

Drug excretion is the removal of drugs from the body using various different routes of excretion, including urine, bile, sweat, saliva, tears, milk, and stool.

Summary

Dataset Unit Number Task Metric Split
Half Life hr 667 Regression Spearman Scaffold
CL-Hepa uL.min-1.(10^6 cells)-1 1,020 Regression Spearman Scaffold
CL-Micro mL.min-1.g-1 1,102 Regression Spearman Scaffold

Leaderboard

Rank Model Contact Link #Params Half Life CL-Hepa CL-Micro
1 RDKit2D + MLP (DeepPurpose) Kexin Huang GitHub, Paper 633,409 0.184 ± 0.111 0.382 ± 0.007 0.586 ± 0.014
2 Morgan + MLP (DeepPurpose) Kexin Huang GitHub, Paper 1,477,185 0.329 ± 0.083 0.272 ± 0.068 0.492 ± 0.020
3 CNN (DeepPurpose) Kexin Huang GitHub, Paper 226,625 0.038 ± 0.138 0.235 ± 0.021 0.252 ± 0.116

: The higher the better. : The lower the better. : Click to sort models for specific benchmark column.


Toxicity

Toxicity measures how much damage a drug could cause to organisms.

Summary

Dataset Unit Number Task Metric Split
LD50 log(1/(mol/kg)) 7,385 Regression MAE Scaffold
hERG % 648 Binary AUROC Scaffold
Ames % 7,255 Binary AUROC Scaffold
DILI % 475 Binary AUROC Scaffold

Leaderboard

Rank Model Contact Link #Params LD50 hERG Ames DILI
1 RDKit2D + MLP (DeepPurpose) Kexin Huang GitHub, Paper 633,409 0.678 ± 0.003 0.841 ± 0.020 0.823 ± 0.011 0.875 ± 0.019
2 Morgan + MLP (DeepPurpose) Kexin Huang GitHub, Paper 1,477,185 0.649 ± 0.019 0.736 ± 0.023 0.794 ± 0.008 0.832 ± 0.021
3 CNN (DeepPurpose) Kexin Huang GitHub, Paper 226,625 0.675 ± 0.011 0.754 ± 0.037 0.776 ± 0.015 0.792 ± 0.016

: The higher the better. : The lower the better. : Click to sort models for specific benchmark column.