Evaluation of the global performance of eight in silico skin sensitization models using human data

Main Article Content

Emily Golden
Donna S. Macmillan
Greg Dameron
Petra Kern
Thomas Hartung
Alexandra Maertens


Allergic contact dermatitis, or the clinical manifestation of skin sensitization, is a leading occupational hazard. Several testing approaches exist to assess skin sensitization, but in silico models are perhaps the most advantageous due to their high speed and low-cost results. Many in silico skin sensitization models exist, though many have only been tested against results from animal studies (e.g., LLNA); this creates uncertainty in human skin sensitization assessments in both a screening and regulatory context. This project’s aim was to evaluate the accuracy of eight in silico skin sensitization models against two human data sets: one highly curated (Basketter et al., 2014) and one screening level (HSDB). The binary skin sen­sitization status of each chemical in each of the two data sets was compared to the prediction from eight in silico skin sensitization tools (Toxtree, PredSkin, OECD’s QSAR Toolbox, UL’s REACHAcross™, Danish QSAR Database, TIMES-SS, and Lhasa Limited’s Derek Nexus). Models were assessed for coverage, accuracy, sensitivity, and specificity, as well as optimization features (e.g., probability of accuracy, applicability domain, etc.), if available. While there was a wide range of sensitivity and specificity, the models generally performed comparably to the LLNA in predicting human skin sensitization status (i.e., approximately 70-80% accuracy). Additionally, the models did not mispredict the same com­pounds, suggesting there might be an advantage in combining models. In silico skin sensitization models offer accurate and useful insights in a screening context; however, further improvements are necessary so these models may be con­sidered fully reliable for regulatory applications.

Article Details

How to Cite
Golden, E., Macmillan, D. S., Dameron, G., Kern, P., Hartung, T. and Maertens, A. (2021) “Evaluation of the global performance of eight in silico skin sensitization models using human data”, ALTEX - Alternatives to animal experimentation, 38(1), pp. 33–48. doi: 10.14573/altex.1911261.

Akhtar, A. (2015). The flaws and human harms of animal experimentation. Camb Q Healthc Ethics 24, 407-419. doi:10.1017/s0963180115000079

Alves, V. M., Capuzzi, S. J., Muratov, E. et al. (2016a). QSAR models of human data can enrich or replace LLNA testing for human skin sensitization. Green Chem 18, 6501-6515. doi:10.1039/c6gc01836j

Alves, V., Muratov, E., Capuzzi, S. et al. (2016b). Alarms about structural alerts. Green Chem 18, 4348-4360. doi:10.1039/c6gc01492e

Alves, V. M., Capuzzi, S. J., Braga, R. C. et al. (2018). A perspective and a new integrated computational strategy for skin sensitization assessment. ACS Sustainable Chem Eng 6, 2845-2859. doi:10.1021/acssuschemeng.7b04220

Anderson, S. E., Siegel, P. D. and Meade, B. J. (2011). The LLNA: A brief review of recent advances and limitations. J Allergy (Cairo) 2011, 424203. doi:10.1155/2011/424203

Bailey, J., Thew, M. and Balls, M. (2014). An analysis of the use of animal models in predicting human toxicology and drug safety. Altern Lab Anim 42, 181-199. doi:10.1177/026119291404200306

Bailey, J., Thew, M. and Balls, M. (2015). Predicting human drug toxicity and safety via animal tests: Can any one species predict drug toxicity in any other, and do monkeys help? Altern Lab Anim 43, 393-403. doi:10.1177/026119291504300607

Basketter, D. A. (2009). The human repeated insult patch test in the 21st century: A commentary. Cutan Ocul Toxicol 28, 49-53. doi:10.1080/15569520902938032

Basketter, D. A., Clewell, H., Kimber, I. et al. (2012). A roadmap for the development of alternative (non-animal) methods for systemic toxicity testing. ALTEX 29, 3-91. doi:10.14573/altex.2012.1.003

Basketter, D. A., Alépée, N., Ashikaga, T. et al. (2014). Categorization of chemicals according to their relative human skin sensitizing potency. Dermatitis 25, 11-21. doi:10.1097/der.0000000000000003

Bauch, C., Kolle, S. N., Ramirez, T. et al. (2012). Putting the parts together: Combining in vitro methods to test for skin sensitizing potentials. Regul Toxicol Pharmacol 63, 489-504. doi:10.1016/j.yrtph.2012.08.014

Canipa, S. J., Chilton, M. L., Hemingway, R. et al. (2017). A quantitative in silico model for predicting skin sensitization using a nearest neighbours approach within expert-derived structure-activity alert spaces. J Appl Toxicol 37, 985-995. doi:10.1002/jat.3448

Cashman, M. W., Reutemann, P. A. and Ehrlich, A. (2012). Contact dermatitis in the united states: Epidemiology, economic impact, and workplace prevention. Dermatol Clin 30, 87-98. doi:10.1016/j.det.2011.08.004

Chaudhry, Q., Piclin, N., Cotterill, J. et al. (2010). Global QSAR models of skin sensitisers for regulatory purposes. Chem Cent J 4, Suppl 1, S5. doi:10.1186/1752-153x-4-s1-s5

Cronin, M. T. D. (2010). Quantitative structure-activity relationships (QSARs) – Applications and methodology. In T. Puzyn, J. Leszczynski and M. T. Cronin (eds.), Recent Advances in QSAR Studies: Methods and Applications (3-11). Dordrecht, The Netherlands: Springer. doi:10.1007/978-1-4020-9783-6_1

Cronin, M. T. D. and Madden, J. C. (2010). In silico toxicology – An introduction. In M. T. D. Cronin and J. C. Madden (eds.), In Silico Toxicology (1-10). Cambridge, UK: RSC Publishing. doi:10.1039/9781849732093

Dreisbach, R. H. (ed.) (1977). Handbook of Poisoning: Diagnosis & Treatment. Lange Medical Publications.

EC – European Commission (2016). Ban on Animal Testing – Internal Market, Industry, Entrepreneurship and SMEs. https://ec.europa.eu/growth/sectors/cosmetics/animal-testing_en (accessed 30.04.2018).

ECHA – European Chemicals Agency (2018a). Table of harmonised entries in Annex VI to CLP. https://echa.europa.eu/information-on-chemicals/annex-vi-to-clp

ECHA (2018b). Harmonised classification and labelling (CLH). https://echa.europa.eu/regulations/clp/harmonised-classification-and-labelling

Enoch, S. J., Ellison, C. M., Schultz, T. W. et al. (2011). A review of the electrophilic reaction chemistry involved in covalent protein binding relevant to toxicity. Crit Rev Toxicol 41, 783-802. doi:10.3109/10408444.2011.598141

EU – European Union (2009). Regulation (EC) No 1223/2009 of the European Parliament and of the Council of 30 November 2009 on cosmetic products. https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32009R1223

Fitzpatrick, J. M., Roberts, D. W. and Patlewicz, G. (2016). What determines skin sensitization potency: Myths, maybes and realities. The 500 molecular weight cut-off: An updated analysis. J Appl Toxicol 37, 105-116. doi:10.1002/jat.3348

Fitzpatrick, J. M., Roberts, D. W. and Patlewicz, G. (2018). An evaluation of selected (Q)SARs/expert systems for predicting skin sensitisation potential. SAR QSAR Environ Res 29, 439-468. doi:10.1080/1062936x.2018.1455223

Gerberick, G. F., Ryan, C. A., Kern, P. S. et al. (2005). Compilation of historical local lymph node data for evaluation of skin sensitization alternative methods. Dermatitis 16, 157-202. doi:10.1097/01206501-200512000-00002

Hartung, T. (2008). Food for thought... On animal tests. ALTEX 25, 3-16. doi:10.14573/altex.2008.1.3

Hartung, T. (2009). Toxicology for the twenty-first century. Nature 460, 208-212. doi:10.1038/460208a

Hartung, T., Luechtefeld, T., Maertens, A. et al. (2013). Integrated testing strategies for safety assessments. ALTEX 30, 3-18. doi:10.14573/altex.2013.1.003

Hartung, T. (2016). Making big sense from big data in toxicology by read-across. ALTEX 33, 83-93. doi:10.14573/altex.1603091

Hayes, W. J. and Laws, E. R. (eds.) (1991). Handbook of Pesticide Toxicology: Classes of pesticides. New York, NY, USA: Academic Press.

Hoffmann, S. (2015). LLNA variability: An essential ingredient for a comprehensive assessment of non-animal skin sensitization test methods and strategies. ALTEX 32, 379-383. doi:10.14573/altex.1505051

Hoffmann, S., Kleinstreuer, N., Alepee, N. et al. (2018). Non-animal methods to predict skin sensitization (I): The Cosmetics Europe database. Crit Rev Toxicol 48, 344-358. doi:10.1080/10408444.2018.1429385

Jaworska, J. (2016). Integrated testing strategies for skin sensitization hazard and potency assessment – State of the art and challenges. Cosmet Toiletries 3, 16. doi:10.3390/cosmetics3020016

Johansson, H. and Lindstedt, M. (2014). Prediction of skin sensitizers using alternative methods to animal experimentation. Basic Clin Pharmacol Toxicol 115, 110-117. doi:10.1111/bcpt.12199

Kadivar, S. and Belsito, D. V. (2015). Occupational dermatitis in health care workers evaluated for suspected allergic contact dermatitis. Dermatitis 26, 177-183. doi:10.1097/der.0000000000000124

Kimber, I., Basketter, D. A., Gerberick, G. F. et al. (2002). Allergic contact dermatitis. Int Immunopharmacol 2, 201-211. doi:10.1016/s1567-5769(01)00173-4

Kleinstreuer, N. C., Hoffmann, S., Alépée, N. et al. (2018). Non-animal methods to predict skin sensitization (II): An assessment of defined approaches*. Crit Rev Toxicol 48, 359-374. doi:10.1080/10408444.2018.1429386

Kostal, J. and Voutchkova-Kostal, A. (2016). CADRE-SS, an in silico tool for predicting skin sensitization potential based on modeling of molecular interactions. Chem Res Toxicol 29, 58-64. doi:10.1021/acs.chemrestox.5b00392

Leist, M. and Hartung, T. (2013). Inflammatory findings on species extrapolations: Humans are definitely no 70-kg mice. Arch Toxicol 87, 563-567. doi:10.1007/s00204-013-1038-0

Luechtefeld, T., Maertens, A., McKim, J. M. et al. (2015). Probabilistic hazard assessment for skin sensitization potency by dose-response modeling using feature elimination instead of quantitative structure-activity relationships. J Appl Toxicol 35, 1361-1371. doi:10.1002/jat.3172

Luechtefeld, T., Maertens, A., Russo, D. P. et al. (2016). Analysis of publically available skin sensitization data from REACH registrations 2008-2014. ALTEX 33, 135-148. doi:10.14573/altex.1510055

Luechtefeld, T. and Hartung, T. (2017). Computational approaches to chemical hazard assessment. ALTEX 34, 459-478. doi:10.14573/altex.1710141

Luechtefeld, T., Marsh, D., Rowlands, C. et al. (2018a). Machine learning of toxicological big data enables read-across structure activity relationships (RASAR) outperforming animal test reproducibility. Toxicol Sci 165, 198-212. doi:10.1093/toxsci/kfy152

Luechtefeld, T., Rowlands, C. and Hartung, T. (2018b). Big-data and machine learning to revamp computational toxicology and its use in risk assessment. Toxicol Res 7, 732-744. doi:10.1039/c8tx00051d

Macmillan, D. S. and Chilton, M. L. (2019). A defined approach for predicting skin sensitisation hazard and potency based on the guided integration of in silico, in chemico and in vitro data using exclusion criteria. Regul Toxicol Pharmacol 101, 35-47. doi:10.1016/j.yrtph.2018.11.001

Maertens, A., Anastas, N., Spencer, P. J. et al. (2014). Food for thought… Green toxicology. ALTEX 31, 243-249. doi:10.14573/altex.1406181

Maertens, A. and Hartung, T. (2018). Green toxicology-know early about and avoid toxic product liabilities. Toxicol Sci 161, 285-289. doi:10.1093/toxsci/kfx243

NRC – National Research Council, Division on Earth and Life Studies, Institute for Laboratory Animal Research, Board on Environmental Studies and Toxicology, Committee on Toxicity Testing and Assessment of Environmental Agents (2007). Toxicity Testing in the 21st Century: A Vision and a Strategy. National Academies Press. doi:10.17226/25135

OECD – Organisation for Economic Co-operation and Development (2012). The Adverse Outcome Pathway for Skin Sensitisation Initiated by Covalent Binding to Proteins. Part 1: Scientific Evidence. OECD Series on Testing and Assessment, No. 168. OECD Publishing, Paris. doi:10.1787/9789264221444-en

OECD (2016). Guidance Document on the Reporting of Defined Approaches and Individual Information Sources to be Used within Integrated Approaches to Testing and Assessment (IATA) for Skin Sensitisation. OECD Series on Testing and Assessment, No. 256. OECD Publishing, Paris. doi:10.1787/9789264279285-en

OECD (2018a). Test No. 442D: In Vitro Skin Sensitisation ARE-Nrf2 Luciferase Test Method: ARE-Nrf2 Luciferase Test Method. OECD Guidelines for the Testing of Chemicals, Section 4. OECD Publishing, Paris. Adopted 25 June 2018. doi:10.1787/9789264229822-en

OECD (2018b). Test No. 442E: In Vitro Skin Sensitisation: In Vitro Skin Sensitisation assays addressing the Key Event on activation of dendritic cells on the Adverse Outcome Pathway for Skin Sensitisation. OECD Guidelines for the Testing of Chemicals, Section 4. OECD Publishing, Paris. doi:10.1787/9789264264359-en.

OECD (2019). Test No. 442C: In Chemico Skin Sensitisation. Assays addressing the Adverse Outcome Pathway key event on covalent binding to proteins. OECD Guidelines for the Testing of Chemicals, Section 4. OECD Publishing, Paris. doi:10.1787/9789264229709-en

Pohanish, R. P. (ed.) (2012). Sittig’s Handbook of Toxic and Hazardous Chemicals and Carcinogens (Sixth Edition). Oxford, UK: William Andrew Publishing. doi:10.1016/b978-1-4377-7869-4.00010-2

Pound, P. and Bracken, M. B. (2014). Is animal research sufficiently evidence based to be a cornerstone of biomedical research? BMJ 348, g3387. doi:10.1136/bmj.g3387

Raunio, H. (2011). In silico toxicology – Non-testing methods. Front Pharmacol 2, 33. doi:10.3389/fphar.2011.00033

Roberts, D. W. and Patlewicz, G. (2009). Chemistry based nonanimal predictive modeling for skin sensitization. In J. Devillers (ed.), Ecotoxicology Modeling (61-83). Boston, MA, USA: Springer. doi:10.1007/978-1-4419-0197-2_3

Sailstad, D. M., Hattan, D., Hill, R. N. et al. (2001). ICCVAM evaluation of the murine local lymph node assay. Reg Toxicol Pharmacol 34, 249-257. doi:10.1006/rtph.2001.1496

Schmidt, M., Raghavan, B., Muller, V. et al. (2010). Crucial role for human toll-like receptor 4 in the development of contact allergy to nickel. Nat Immunol 11, 814-819. doi:10.1038/ni.1919

Smith Pease, C. K., Basketter, D. A. and Patlewicz, G. Y. (2003). Contact allergy: The role of skin chemistry and metabolism. Clin Exp Dermatol 28, 177-183. doi:10.1046/j.1365-2230.2003.01239.x

Teubner, W., Mehling, A., Schuster, P. X. et al. (2013). Computer models versus reality: How well do in silico models currently predict the sensitization potential of a substance. Regul Toxicol Pharmacol 67, 468-485. doi:10.1016/j.yrtph.2013.09.007

Thyssen, J. P., Linneberg, A., Menne, T. et al. (2007). The epidemiology of contact allergy in the general population – Prevalence and main findings. Contact Dermatitis 57, 287-299. doi:10.1111/j.1600-0536.2007.01220.x

UN – United Nations. (2017). Globally Harmonized System of Classification and Labelling of Chemicals (GHS) (Rev.7). doi:10.18356/e18d11a0-en

Urbisch, D., Mehling, A., Guth, K. et al. (2015). Assessing skin sensitization hazard in mice and men using non-animal test methods. Regul Toxicol Pharmacol 71, 337-351. doi:10.1016/j.yrtph.2014.12.008

Verheyen, G. R., Braeken, E., Van Deun, K. et al. (2017). Evaluation of in silico tools to predict the skin sensitization potential of chemicals. SAR QSAR Environ Res 28, 59-73. doi:10.1080/1062936x.2017.1278617

Warshaw, E. M., Hagen, S. L., Sasseville, D. et al. (2017). Occupational contact dermatitis in mechanics and repairers referred for patch testing: Retrospective analysis from the North American contact dermatitis group 1998-2014. Dermatitis 28, 47-57. doi:10.1097/der.0000000000000231

Wilm, A., Kuhnl, J. and Kirchmair, J. (2018). Computational approaches for skin sensitization prediction. Crit Rev Toxicol 48, 738-760. doi:10.1080/10408444.2018.1528207

Most read articles by the same author(s)

1 2 3 4 5 6 7 8 9 10 > >>