Drug Combination Benchmark Group

Drug combination screening offers significant potential for expanding the use of existing drugs and in improving their efficacy. For instance, the simultaneous modulation of multiple targets can address the common mechanisms of drug resistance seen in the treatment of cancers. However, experimentally exploring the entire space of possible drug combinations is not a feasible task. Thus, computational models that can predict synergistic combinations can prove very valuable.

This benchmark group currently contains the TDC.DrugComb dataset. In this dataset, we have five measurements of combination effects. The main endpoint is called drug combination sensitivity (TDC.DrugComb_CSS) score, which is derived using relative IC50 values of compounds and the area under their dose-response curves. For CSS, we also provide information on the tissue types of the cell lines.

In addition to the CSS, we also provide four additional synergy measurements. Synergy is a measure of deviation of an observed drug combination response from the expected effect of non-interaction. We include four labels where each is a different way of calculating the deviation: Bliss model (TDC.DrugComb_Bliss), Highest Single Agent (TDC.DrugComb_HSA), Loewe additivity model (TDC.DrugComb_Loewe) and Zero Interaction Potency (TDC.DrugComb_ZIP). To retrieve the benchmark names, type:

from tdc import utils
names = utils.retrieve_benchmark_names('DrugCombo_Group')
# ['drugcomb_css', 'drugcomb_hsa', ...]

To access each benchmark, use the following code:

from tdc import BenchmarkGroup
group = BenchmarkGroup(name = 'DrugCombo_Group', path = 'data/', file_format='pkl')

benchmark = group.get('Drugcomb_CSS')

predictions = {}
name = benchmark['name']
train_val, test = benchmark['train_val'], benchmark['test']

## --- train your model --- ##

predictions[name] = pred_test
out = group.evaluate(predictions)
Note that the output includes the automatic evaluations across tissues:

{'drugcomb_css': {'mae': 23.082},
 'drugcomb_css_kidney': {'mae': 21.906},
 'drugcomb_css_lung': {'mae': 21.341},
 'drugcomb_css_breast': {'mae': 18.542},
 'drugcomb_css_hematopoietic_lymphoid': {'mae': 40.55},
 'drugcomb_css_colon': {'mae': 25.224},
 'drugcomb_css_prostate': {'mae': 22.19},
 'drugcomb_css_ovary': {'mae': 19.638},
 'drugcomb_css_skin': {'mae': 18.777},
 'drugcomb_css_brain': {'mae': 21.855}}

Follow the instruction on how to use the BenchmarkGroup class and training validation split, and also submission instructions.

For every dataset, we use drug combination split and hold out a 20% test set. The evaluation metrics is MAE.

Note that the tissue types are automatically calculated based on the test set prediction on TDC.DrugComb_CSS endpoint.

Benchmark Data Summary

Label Number Task Metric Split
TDC.DrugComb_CSS 297,098 Regression MAE Combination
TDC.DrugComb_HSA 297,098 Regression MAE Combination
TDC.DrugComb_Loewe 297,098 Regression MAE Combination
TDC.DrugComb_Bliss 297,098 Regression MAE Combination
TDC.DrugComb_Zip 297,098 Regression MAE Combination