Evaluation of the global performance of eight in silico skin sensitization models using human data
Article Sidebar
Main Article Content
Abstract
Allergic contact dermatitis, or the clinical manifestation of skin sensitization, is a leading occupational hazard. Several testing approaches exist to assess skin sensitization, but in silico models are perhaps the most advantageous due to their high speed and low-cost results. Many in silico skin sensitization models exist, though many have only been tested against results from animal studies (e.g., LLNA); this creates uncertainty in human skin sensitization assessments in both a screening and regulatory context. This project’s aim was to evaluate the accuracy of eight in silico skin sensitization models against two human data sets: one highly curated (Basketter et al., 2014) and one screening level (HSDB). The binary skin sensitization status of each chemical in each of the two data sets was compared to the prediction from eight in silico skin sensitization tools (Toxtree, PredSkin, OECD’s QSAR Toolbox, UL’s REACHAcross™, Danish QSAR Database, TIMES-SS, and Lhasa Limited’s Derek Nexus). Models were assessed for coverage, accuracy, sensitivity, and specificity, as well as optimization features (e.g., probability of accuracy, applicability domain, etc.), if available. While there was a wide range of sensitivity and specificity, the models generally performed comparably to the LLNA in predicting human skin sensitization status (i.e., approximately 70-80% accuracy). Additionally, the models did not mispredict the same compounds, suggesting there might be an advantage in combining models. In silico skin sensitization models offer accurate and useful insights in a screening context; however, further improvements are necessary so these models may be considered fully reliable for regulatory applications.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Articles are distributed under the terms of the Creative Commons Attribution 4.0 International license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is appropriately cited (CC-BY). Copyright on any article in ALTEX is retained by the author(s).
Akhtar, A. (2015). The flaws and human harms of animal experimentation. Camb Q Healthc Ethics 24, 407-419. doi:10.1017/s0963180115000079
Alves, V. M., Capuzzi, S. J., Muratov, E. et al. (2016a). QSAR models of human data can enrich or replace LLNA testing for human skin sensitization. Green Chem 18, 6501-6515. doi:10.1039/c6gc01836j
Alves, V., Muratov, E., Capuzzi, S. et al. (2016b). Alarms about structural alerts. Green Chem 18, 4348-4360. doi:10.1039/c6gc01492e
Alves, V. M., Capuzzi, S. J., Braga, R. C. et al. (2018). A perspective and a new integrated computational strategy for skin sensitization assessment. ACS Sustainable Chem Eng 6, 2845-2859. doi:10.1021/acssuschemeng.7b04220
Anderson, S. E., Siegel, P. D. and Meade, B. J. (2011). The LLNA: A brief review of recent advances and limitations. J Allergy (Cairo) 2011, 424203. doi:10.1155/2011/424203
Bailey, J., Thew, M. and Balls, M. (2014). An analysis of the use of animal models in predicting human toxicology and drug safety. Altern Lab Anim 42, 181-199. doi:10.1177/026119291404200306
Bailey, J., Thew, M. and Balls, M. (2015). Predicting human drug toxicity and safety via animal tests: Can any one species predict drug toxicity in any other, and do monkeys help? Altern Lab Anim 43, 393-403. doi:10.1177/026119291504300607
Basketter, D. A. (2009). The human repeated insult patch test in the 21st century: A commentary. Cutan Ocul Toxicol 28, 49-53. doi:10.1080/15569520902938032
Basketter, D. A., Clewell, H., Kimber, I. et al. (2012). A roadmap for the development of alternative (non-animal) methods for systemic toxicity testing. ALTEX 29, 3-91. doi:10.14573/altex.2012.1.003
Basketter, D. A., Alépée, N., Ashikaga, T. et al. (2014). Categorization of chemicals according to their relative human skin sensitizing potency. Dermatitis 25, 11-21. doi:10.1097/der.0000000000000003
Bauch, C., Kolle, S. N., Ramirez, T. et al. (2012). Putting the parts together: Combining in vitro methods to test for skin sensitizing potentials. Regul Toxicol Pharmacol 63, 489-504. doi:10.1016/j.yrtph.2012.08.014
Canipa, S. J., Chilton, M. L., Hemingway, R. et al. (2017). A quantitative in silico model for predicting skin sensitization using a nearest neighbours approach within expert-derived structure-activity alert spaces. J Appl Toxicol 37, 985-995. doi:10.1002/jat.3448
Cashman, M. W., Reutemann, P. A. and Ehrlich, A. (2012). Contact dermatitis in the united states: Epidemiology, economic impact, and workplace prevention. Dermatol Clin 30, 87-98. doi:10.1016/j.det.2011.08.004
Chaudhry, Q., Piclin, N., Cotterill, J. et al. (2010). Global QSAR models of skin sensitisers for regulatory purposes. Chem Cent J 4, Suppl 1, S5. doi:10.1186/1752-153x-4-s1-s5
Cronin, M. T. D. (2010). Quantitative structure-activity relationships (QSARs) – Applications and methodology. In T. Puzyn, J. Leszczynski and M. T. Cronin (eds.), Recent Advances in QSAR Studies: Methods and Applications (3-11). Dordrecht, The Netherlands: Springer. doi:10.1007/978-1-4020-9783-6_1
Cronin, M. T. D. and Madden, J. C. (2010). In silico toxicology – An introduction. In M. T. D. Cronin and J. C. Madden (eds.), In Silico Toxicology (1-10). Cambridge, UK: RSC Publishing. doi:10.1039/9781849732093
Dreisbach, R. H. (ed.) (1977). Handbook of Poisoning: Diagnosis & Treatment. Lange Medical Publications.
EC – European Commission (2016). Ban on Animal Testing – Internal Market, Industry, Entrepreneurship and SMEs. https://ec.europa.eu/growth/sectors/cosmetics/animal-testing_en (accessed 30.04.2018).
ECHA – European Chemicals Agency (2018a). Table of harmonised entries in Annex VI to CLP. https://echa.europa.eu/information-on-chemicals/annex-vi-to-clp
ECHA (2018b). Harmonised classification and labelling (CLH). https://echa.europa.eu/regulations/clp/harmonised-classification-and-labelling
Enoch, S. J., Ellison, C. M., Schultz, T. W. et al. (2011). A review of the electrophilic reaction chemistry involved in covalent protein binding relevant to toxicity. Crit Rev Toxicol 41, 783-802. doi:10.3109/10408444.2011.598141
EU – European Union (2009). Regulation (EC) No 1223/2009 of the European Parliament and of the Council of 30 November 2009 on cosmetic products. https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32009R1223
Fitzpatrick, J. M., Roberts, D. W. and Patlewicz, G. (2016). What determines skin sensitization potency: Myths, maybes and realities. The 500 molecular weight cut-off: An updated analysis. J Appl Toxicol 37, 105-116. doi:10.1002/jat.3348
Fitzpatrick, J. M., Roberts, D. W. and Patlewicz, G. (2018). An evaluation of selected (Q)SARs/expert systems for predicting skin sensitisation potential. SAR QSAR Environ Res 29, 439-468. doi:10.1080/1062936x.2018.1455223
Gerberick, G. F., Ryan, C. A., Kern, P. S. et al. (2005). Compilation of historical local lymph node data for evaluation of skin sensitization alternative methods. Dermatitis 16, 157-202. doi:10.1097/01206501-200512000-00002
Hartung, T. (2008). Food for thought... On animal tests. ALTEX 25, 3-16. doi:10.14573/altex.2008.1.3
Hartung, T. (2009). Toxicology for the twenty-first century. Nature 460, 208-212. doi:10.1038/460208a
Hartung, T., Luechtefeld, T., Maertens, A. et al. (2013). Integrated testing strategies for safety assessments. ALTEX 30, 3-18. doi:10.14573/altex.2013.1.003
Hartung, T. (2016). Making big sense from big data in toxicology by read-across. ALTEX 33, 83-93. doi:10.14573/altex.1603091
Hayes, W. J. and Laws, E. R. (eds.) (1991). Handbook of Pesticide Toxicology: Classes of pesticides. New York, NY, USA: Academic Press.
Hoffmann, S. (2015). LLNA variability: An essential ingredient for a comprehensive assessment of non-animal skin sensitization test methods and strategies. ALTEX 32, 379-383. doi:10.14573/altex.1505051
Hoffmann, S., Kleinstreuer, N., Alepee, N. et al. (2018). Non-animal methods to predict skin sensitization (I): The Cosmetics Europe database. Crit Rev Toxicol 48, 344-358. doi:10.1080/10408444.2018.1429385
Jaworska, J. (2016). Integrated testing strategies for skin sensitization hazard and potency assessment – State of the art and challenges. Cosmet Toiletries 3, 16. doi:10.3390/cosmetics3020016
Johansson, H. and Lindstedt, M. (2014). Prediction of skin sensitizers using alternative methods to animal experimentation. Basic Clin Pharmacol Toxicol 115, 110-117. doi:10.1111/bcpt.12199
Kadivar, S. and Belsito, D. V. (2015). Occupational dermatitis in health care workers evaluated for suspected allergic contact dermatitis. Dermatitis 26, 177-183. doi:10.1097/der.0000000000000124
Kimber, I., Basketter, D. A., Gerberick, G. F. et al. (2002). Allergic contact dermatitis. Int Immunopharmacol 2, 201-211. doi:10.1016/s1567-5769(01)00173-4
Kleinstreuer, N. C., Hoffmann, S., Alépée, N. et al. (2018). Non-animal methods to predict skin sensitization (II): An assessment of defined approaches*. Crit Rev Toxicol 48, 359-374. doi:10.1080/10408444.2018.1429386
Kostal, J. and Voutchkova-Kostal, A. (2016). CADRE-SS, an in silico tool for predicting skin sensitization potential based on modeling of molecular interactions. Chem Res Toxicol 29, 58-64. doi:10.1021/acs.chemrestox.5b00392
Leist, M. and Hartung, T. (2013). Inflammatory findings on species extrapolations: Humans are definitely no 70-kg mice. Arch Toxicol 87, 563-567. doi:10.1007/s00204-013-1038-0
Luechtefeld, T., Maertens, A., McKim, J. M. et al. (2015). Probabilistic hazard assessment for skin sensitization potency by dose-response modeling using feature elimination instead of quantitative structure-activity relationships. J Appl Toxicol 35, 1361-1371. doi:10.1002/jat.3172
Luechtefeld, T., Maertens, A., Russo, D. P. et al. (2016). Analysis of publically available skin sensitization data from REACH registrations 2008-2014. ALTEX 33, 135-148. doi:10.14573/altex.1510055
Luechtefeld, T. and Hartung, T. (2017). Computational approaches to chemical hazard assessment. ALTEX 34, 459-478. doi:10.14573/altex.1710141
Luechtefeld, T., Marsh, D., Rowlands, C. et al. (2018a). Machine learning of toxicological big data enables read-across structure activity relationships (RASAR) outperforming animal test reproducibility. Toxicol Sci 165, 198-212. doi:10.1093/toxsci/kfy152
Luechtefeld, T., Rowlands, C. and Hartung, T. (2018b). Big-data and machine learning to revamp computational toxicology and its use in risk assessment. Toxicol Res 7, 732-744. doi:10.1039/c8tx00051d
Macmillan, D. S. and Chilton, M. L. (2019). A defined approach for predicting skin sensitisation hazard and potency based on the guided integration of in silico, in chemico and in vitro data using exclusion criteria. Regul Toxicol Pharmacol 101, 35-47. doi:10.1016/j.yrtph.2018.11.001
Maertens, A., Anastas, N., Spencer, P. J. et al. (2014). Food for thought… Green toxicology. ALTEX 31, 243-249. doi:10.14573/altex.1406181
Maertens, A. and Hartung, T. (2018). Green toxicology-know early about and avoid toxic product liabilities. Toxicol Sci 161, 285-289. doi:10.1093/toxsci/kfx243
NRC – National Research Council, Division on Earth and Life Studies, Institute for Laboratory Animal Research, Board on Environmental Studies and Toxicology, Committee on Toxicity Testing and Assessment of Environmental Agents (2007). Toxicity Testing in the 21st Century: A Vision and a Strategy. National Academies Press. doi:10.17226/25135
OECD – Organisation for Economic Co-operation and Development (2012). The Adverse Outcome Pathway for Skin Sensitisation Initiated by Covalent Binding to Proteins. Part 1: Scientific Evidence. OECD Series on Testing and Assessment, No. 168. OECD Publishing, Paris. doi:10.1787/9789264221444-en
OECD (2016). Guidance Document on the Reporting of Defined Approaches and Individual Information Sources to be Used within Integrated Approaches to Testing and Assessment (IATA) for Skin Sensitisation. OECD Series on Testing and Assessment, No. 256. OECD Publishing, Paris. doi:10.1787/9789264279285-en
OECD (2018a). Test No. 442D: In Vitro Skin Sensitisation ARE-Nrf2 Luciferase Test Method: ARE-Nrf2 Luciferase Test Method. OECD Guidelines for the Testing of Chemicals, Section 4. OECD Publishing, Paris. Adopted 25 June 2018. doi:10.1787/9789264229822-en
OECD (2018b). Test No. 442E: In Vitro Skin Sensitisation: In Vitro Skin Sensitisation assays addressing the Key Event on activation of dendritic cells on the Adverse Outcome Pathway for Skin Sensitisation. OECD Guidelines for the Testing of Chemicals, Section 4. OECD Publishing, Paris. doi:10.1787/9789264264359-en.
OECD (2019). Test No. 442C: In Chemico Skin Sensitisation. Assays addressing the Adverse Outcome Pathway key event on covalent binding to proteins. OECD Guidelines for the Testing of Chemicals, Section 4. OECD Publishing, Paris. doi:10.1787/9789264229709-en
Pohanish, R. P. (ed.) (2012). Sittig’s Handbook of Toxic and Hazardous Chemicals and Carcinogens (Sixth Edition). Oxford, UK: William Andrew Publishing. doi:10.1016/b978-1-4377-7869-4.00010-2
Pound, P. and Bracken, M. B. (2014). Is animal research sufficiently evidence based to be a cornerstone of biomedical research? BMJ 348, g3387. doi:10.1136/bmj.g3387
Raunio, H. (2011). In silico toxicology – Non-testing methods. Front Pharmacol 2, 33. doi:10.3389/fphar.2011.00033
Roberts, D. W. and Patlewicz, G. (2009). Chemistry based nonanimal predictive modeling for skin sensitization. In J. Devillers (ed.), Ecotoxicology Modeling (61-83). Boston, MA, USA: Springer. doi:10.1007/978-1-4419-0197-2_3
Sailstad, D. M., Hattan, D., Hill, R. N. et al. (2001). ICCVAM evaluation of the murine local lymph node assay. Reg Toxicol Pharmacol 34, 249-257. doi:10.1006/rtph.2001.1496
Schmidt, M., Raghavan, B., Muller, V. et al. (2010). Crucial role for human toll-like receptor 4 in the development of contact allergy to nickel. Nat Immunol 11, 814-819. doi:10.1038/ni.1919
Smith Pease, C. K., Basketter, D. A. and Patlewicz, G. Y. (2003). Contact allergy: The role of skin chemistry and metabolism. Clin Exp Dermatol 28, 177-183. doi:10.1046/j.1365-2230.2003.01239.x
Teubner, W., Mehling, A., Schuster, P. X. et al. (2013). Computer models versus reality: How well do in silico models currently predict the sensitization potential of a substance. Regul Toxicol Pharmacol 67, 468-485. doi:10.1016/j.yrtph.2013.09.007
Thyssen, J. P., Linneberg, A., Menne, T. et al. (2007). The epidemiology of contact allergy in the general population – Prevalence and main findings. Contact Dermatitis 57, 287-299. doi:10.1111/j.1600-0536.2007.01220.x
UN – United Nations. (2017). Globally Harmonized System of Classification and Labelling of Chemicals (GHS) (Rev.7). doi:10.18356/e18d11a0-en
Urbisch, D., Mehling, A., Guth, K. et al. (2015). Assessing skin sensitization hazard in mice and men using non-animal test methods. Regul Toxicol Pharmacol 71, 337-351. doi:10.1016/j.yrtph.2014.12.008
Verheyen, G. R., Braeken, E., Van Deun, K. et al. (2017). Evaluation of in silico tools to predict the skin sensitization potential of chemicals. SAR QSAR Environ Res 28, 59-73. doi:10.1080/1062936x.2017.1278617
Warshaw, E. M., Hagen, S. L., Sasseville, D. et al. (2017). Occupational contact dermatitis in mechanics and repairers referred for patch testing: Retrospective analysis from the North American contact dermatitis group 1998-2014. Dermatitis 28, 47-57. doi:10.1097/der.0000000000000231
Wilm, A., Kuhnl, J. and Kirchmair, J. (2018). Computational approaches for skin sensitization prediction. Crit Rev Toxicol 48, 738-760. doi:10.1080/10408444.2018.1528207