Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks

Main Article Content

Sebastian Hoffmann
Elisa Aiassa
Michelle Angrish
Claire Beausoleil
Frederic Y. Bois
Laura Ciccolallo
Peter S. Craig
Rob B. M. de Vries
Jean Lou C. M. Dorne
Ingrid L. Druwe
Stephen W. Edwards
Chantra Eskes
Marios Georgiadis
Thomas Hartung
Aude Kienzler
Elisabeth A. Kristjansson
Juleen Lam
Laura Martino
Bette Meek
Rebecca L. Morgan
Irene Munoz-Guajardo
Pamela D. Noyes
Elena Parmelli
Aldert Piersma
Andrew Rooney
Emily Sena
Kristie Sullivan
José Tarazona
Andrea Terron
Kris Thayer
Jan Turner
Jos Verbeek
Didier Verloo
Mathieu Vinken
Sean Watford
Paul Whaley
Daniele Wikoff
Kate Willett
Katya Tsaioun


The workshop titled “Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks” was co-organized by the Evidence-based Toxicology Collaboration and the European Food Safety Authority (EFSA), and hosted by EFSA at its headquarters in Parma, Italy on October 2 and 3, 2019. The goal was to explore integration of systematic review methods with mechanistic evidence evaluation. Participants were invited to work on concrete products to advance the exploration of how evidence-based approaches can support the development and application of adverse outcome pathways (AOP) in chemical risk assessment. The workshop discussions were centered around three related themes: 1) assessing certainty in AOPs, 2) literature-based AOP development and 3) integrating certainty in AOPs and non-animal evidence into decision frameworks. Several challenges, mostly related to methodology, were identified, and largely determined the workshop recommendations. The workshop recommendations included the comparison and potential alignment of processes used to develop AOP and systematic review methodology, including the translation of vocabulary of evidence-based methods to AOP and vice versa, the development and improvement of evidence mapping and text mining methods and tools, as well as a call for a fundamental change in chemical risk and uncertainty assessment methodology if to be conducted based on AOPs and new approach methodologies (NAM). The usefulness of evidence-based approaches for mechanism-based chemical risk assessments was stressed, particularly the potential contribution of the rigor and transparency inherent to such approaches in building stakeholders’ trust for implementation of NAM evidence and AOP into chemical risk assessment.

Article Details

How to Cite
Hoffmann, S., Aiassa, E., Angrish, M., Beausoleil, . C. ., Bois, F. Y. ., Ciccolallo, L., Craig, P. S., de Vries, R. B. M. ., Dorne, J. L. C. M. ., Druwe, I. L., Edwards, S. W. ., Eskes, C., Georgiadis, M., Hartung, T. ., Kienzler, A., Kristjansson, E. A. ., Lam, J. ., Martino, L., Meek, B., Morgan, R. L. ., Munoz-Guajardo, I., Noyes, P. D. ., Parmelli, E. ., Piersma, A., Rooney, A., Sena, E., Sullivan, K., Tarazona, J., Terron, A., Thayer, K., Turner, J., Verbeek, J., Verloo, D., Vinken, M., Watford, S., Whaley, P., Wikoff, D., Willett, K. and Tsaioun, K. (2022) “Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks ”, ALTEX - Alternatives to animal experimentation. doi: 10.14573/altex.2202141.
Meeting Reports

Abbott, A. (2009). Toxicity testing gets a makeover. Nature 461, 158. doi:10.1038/461158a

Ågerstrand, M., Breitholtz, M., Rudén, C. (2011a). Comparison of four different methods for reliability evaluation of ecotoxicity data: a case study of non-standard test data used in environmental risk assessments of pharmaceutical substances. Environmental Sciences Europe 23(1):17. doi:10.1186/2190-4715-23-17

Ågerstrand, M., Küster, A., Bachmann, J. et al. (2011b). Reporting and evaluation criteria as means towards a transparent use of ecotoxicity data for environmental risk assessment of pharmaceuticals. Environ Poll 159(10):2487-92. doi:10.1016/j.envpol.2011.06.023

Alonso-Coello, P., Schünemann, H. J., Moberg, J. et al. (2016a). GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ 353:i2016. doi:10.1136/bmj.i2016

Alonso-Coello, P., Oxman, A. D., Moberg, J. et al. (2016b). GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ 353:i2089. doi:10.1136/bmj.i2089

Anderson, L. M., Petticrew, M., Rehfuess, E. et al. (2011). Using logic models to capture complexity in systematic reviews. Res Syn Meth 2: 33-42. doi:10.1002/jrsm.32

Ankley, G.T., Bennett, R. S., Erickson, R. J. et al. (2010). Adverse Outcome Pathways: A Conceptual Framework to Support Ecotoxicology Research and Risk Assessment. Environ Toxicol Chem 29(3): 730–41. doi:10.1002/etc.34

Ankley, G. T., Edwards, S. W. (2018). The Adverse Outcome Pathway: A Multifaceted Framework Supporting 21stCentury Toxicology. Curr Opin Toxicol 9:1-7. doi:10.1016/j.cotox.2018.03.004

Arnesdotter, E., Spinu, N., Firman, J. et al. (2021). Derivation, characterisation and analysis of an adverse outcome pathway network for human hepatotoxicity. Toxicology. 459:152856. doi:10.1016/j.tox.2021.152856

Arzuaga, X., Smith, M. T,, Gibbons, C. F. et al. (2019). Proposed Key Characteristics of Male Reproductive Toxicants as an Approach for Organizing and Evaluating Mechanistic Evidence in Human Health Hazard Assessments. Environ Health Perspect 127(6):65001. doi:10.1289/EHP5045

Bal-Price, A., Lein, P. J., Keil, K. P. et al. (2017). Developing and applying the adverse outcome pathway concept for understanding and predicting neurotoxicity. Neurotoxicology 59:240-55. doi:10.1016/j.neuro.2016.05.010

Bal-Price, A., Meek, M. B. (2017). Adverse outcome pathways: application to enhance mechanistic understanding of neurotoxicity. Pharmacol Ther 179:84-95. doi:10.1016/j.pharmthera.2017.05.006

Becker, R. A., Ankley, G. T., Edwards, S. W. et al. (2015).. Increasing Scientific Confidence in Adverse Outcome Pathways: Application of Tailored Bradford-Hill Considerations for Evaluating Weight of Evidence. Regulatory Toxicology and Pharmacol 72 (3): 514–37. doi:10.1016/j.yrtph.2015.04.004

Beronius, A., Molander, L., Zilliacus, J. et al (2018). Testing and refining the Science in Risk Assessment and Policy (SciRAP) web‐based platform for evaluating the reliability and relevance of in vivo toxicity studies. J Appl Toxicol 38(12):1460-70. doi:10.1002/jat.3648

Bozada, T. Jr., Borden, J., Workman, J. et al. (2021). Sysrev: A FAIR Platform for Data Curation and Systematic Evidence Review. Front Artif Intell 4:685298. doi:10.3389/frai.2021.685298

Burnett, S. D., Blanchette, A. D., Grimm, F. A. et al. (2019). Population-based toxicity screening in human induced pluripotent stem cell-derived cardiomyocytes. Toxicol Appl Pharmacol 381:114711. doi:10.1016/j.taap.2019.114711

Casati, S. (2018). Integrated Approaches to Testing and Assessment. Basic Clin Pharmacol Toxicol 123 Suppl 5:51-55. doi:10.1111/bcpt.13018

Darney, K., Lautz, L. S., Béchaux, C. et al. (2021). Human variability in polymorphic CYP2D6 metabolism: Implications for the risk assessment of chemicals in food and emerging designer drugs. Environ Int 156:106760. doi:10.1016/j.envint.2021.106760

de Vries, R. and Whaley, P. (2018). In Vitro Critical Appraisal Tool (IV-CAT): Tool Development Protocol. doi:10.5281/zenodo.1493498

de Vries, R. B. M., Angrish, M., Browne, P., et al. (2021). Applying evidence-based methods to the development and use of adverse outcome pathways. ALTEX 38(2):336-347. doi:10.14573/altex.2101211

Dishaw, L., Yost, E., Arzuaga, X. et al. (2020). A novel study evaluation strategy in the systematic review of animal toxicology studies for human health assessments of environmental chemicals. Environ Int 141:105736. doi:10.1016/j.envint.2020.105736

Dornbos, P., LaPres, J. J. (2018). Incorporating population-level genetic variability within laboratory models in toxicology: From the individual to the population. Toxicology 395:1-8. doi:10.1016/j.tox.2017.12.007

Ede, J. D., Lobaskin, V., Vogel, U. et al. (2020). Translating Scientific Advances in the AOP Framework to Decision Making for Nanomaterials. Nanomaterials 10(6):1229. doi:10.3390/nano10061229

EFSA (European Food Safety Authority) and EBTC (Evidence-Based Toxicology Collaboration) 2018. EFSA Scientific Colloquium 23: Evidence integration in risk assessment: the science of combining apples and oranges. EFSA supporting publication 2018: 16(3):EN-1396. 28 pp. doi:10.2903/sp.efsa.2018.EN-1396

EFSA PPR Panel (EFSA Panel on Plant Protection Products and their Residues), Ockleford, C. , Adriaanse, P. et al (2017). Scientific Opinion on the investigation into experimental toxicological properties of plant protection products having a potential link to Parkinson's disease and childhood leukaemia. EFSA Journal 15(3):4691, 325 pp. doi:10.2903/j.efsa.2017.4691

Fox, D.M. (2010). The convergence of science and governance: research, health policy, and American states. Univ of California Press.

Gilbert, M. E., Rovet, J., Chen, Z. et al. (2012). Developmental thyroid hormone disruption: prevalence, environmental contaminants and neurodevelopmental consequences. Neurotoxicology 33(4):842-52. doi:10.1016/j.neuro.2011.11.005

Gilmour, N., Kern, P. S., Alépée, N. et al. (2020). Development of a next generation risk assessment framework for the evaluation of skin sensitisation of cosmetic ingredients. Regul Toxicol Pharmacol 116:104721. doi:10.1016/j.yrtph.2020.104721

Guyatt, G. H., Oxman, A. D., Vist, G. E. et al (2008). GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336:924-6. doi:10.1136/bmj.39489.470347.AD

Guyatt, G. H., Oxman, A. D., Vist, G. et al (2011). GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol 64(4):407-15. doi:10.1016/j.jclinepi.2010.07.017

Hartung, T., Hoffmann, S., Stephens, M. (2013). Mechanistic validation. ALTEX 30(2):119-30. doi:10.14573/altex.2013.2.119

Higgins, J. P. T., Chandler, T. J. , Cumpston, M. et al. (2019). Cochrane Handbook for Systematic Reviews of Interventions. 2019. Chichester (UK), John Wiley & Sons. Available at: (last accessed Feb 14, 2022)

Hoffmann, S., de Vries, R. B. M., Stephens, M. L. et al (2017). A primer on systematic reviews in toxicology. Arch Toxicol 91(7):2551-2575. doi:10.1007/s00204-017-1980-3

Horvat, T., Landesmann, B., Lostia, A. et al. (2017). Adverse outcome pathway development from protein alkylation to liver fibrosis. Arch Toxicol 91(4):1523-1543. doi:10.1007/s00204-016-1814-8

Institute of Medicine. Finding What Works in Health Care: Standards for Systematic Reviews. 2011. Washington, DC: National Academies Press. doi:10.17226/13059

Ives, C., Campia, I., Wang, R. L. et al. (2017). Creating a Structured AOP Knowledgebase via Ontology-Based Annotations. Appl In Vitro Toxicol 1;3(4):298-311. doi:10.1089/aivt.2017.0017

Johnson, P. I., Koustas, E., Vesterinen, H. M. et al. (2016). Application of the Navigation Guide systematic review methodology to the evidence for developmental and reproductive toxicity of triclosan. Environ Int 92:716-28. doi:10.1016/j.envint.2016.03.009

Judson, R. S., Houck, K. A., Kavlock, R. J. et al. (2010). In vitro screening of environmental chemicals for targeted testing prioritization: the ToxCast project. Environ Health Perspect 118(4):485-92. doi:10.1289/ehp.0901392

Kavlock, R. J., Austin, C. P., Tice, R. R. (2009). Toxicity testing in the 21st century: implications for human health risk assessment. Risk Analysis 29(4):485-7. doi:10.1111/j.1539-6924.2008.01168.x

Klimisch, H. J., Andreae, M., Tillmann, U. (1997). A systematic approach for evaluating the quality of experimental toxicological and ecotoxicological data. Regul Toxicol Pharmacol 25(1):1-5. doi:10.1006/rtph.1996.1076

Knapen, D., Stinckens, E., Cavallin, J. E. et al. (2020). Toward an AOP Network-Based Tiered Testing Strategy for the Assessment of Thyroid Hormone Disruption. Environ Sci Technol 54(14):8491-8499. doi:10.1021/acs.est.9b07205

Knight, A. (2008). Non-animal methodologies within biomedical research and toxicity testing. 2008. ALTEX 25:213-231. doi:10.14573/altex.2008.3.213

Krithikadatta, J., Gopikrishna, V. and Datta, M. (2014). CRIS Guidelines (Checklist for Reporting In-vitro Studies): A concept note on the need for standardized guidelines for improving quality and transparency in reporting in-vitro studies in experimental dental research. J Conserv Dent 17(4), p.301. doi:10.4103/0972-0707.136338

La Merrill, M. A., Vandenberg, L. N., Smith, M. T. et al. (2019). Consensus on the key characteristics of endocrine-disrupting chemicals as a basis for hazard identification. Nat Rev Endocrinol 1-3. doi:10.1038/s41574-019-0273-8

Lam, J., Koustas, E., Sutton, P. et al. (2014). The Navigation Guide—evidence-based medicine meets environmental health: integration of animal and human evidence for PFOA effects on fetal growth. Environ Health Perspect 122(10):1040-51. doi:10.1289/ehp.1307923

Lam, J., Sutton, P., Kalkbrenner, A. et al. (2016). A systematic review and meta-analysis of multiple airborne pollutants and autism spectrum disorder. PLoS One 11(9):e0161851. doi:10.1371/journal.pone.0161851

Lam, J., Lanphear, B. P., Bellinger, D. et al. (2017). Developmental PBDE exposure and IQ/ADHD in childhood: a systematic review and meta-analysis. Environ Health Perspect 125(8):086001. doi:10.1289/EHP1632

Leist, M., Ghallab, A., Graepel, R. et al. (2017). Adverse outcome pathways: opportunities, limitations and open questions. Arch Toxicol 91(11):3477-3505. doi:10.1007/s00204-017-2045-3

Li, S. A., Alexander, P. E., Reljic, T. et al. (2018). Evidence to Decision framework provides a structured "roadmap" for making GRADE guidelines recommendations. J Clin Epidemiol 104:103-112. doi:10.1016/j.jclinepi.2018.09.007

Liu, Z., Shi, Q., Ding, D. et al. (2011). Translating clinical findings into knowledge in drug safety evaluation--drug induced liver injury prediction system (DILIps). PLoS Comput Biol 7(12):e1002310. doi:10.1371/journal.pcbi.1002310

Luderer, U., Eskenazi, B., Hauser, R. et al. (2019). Proposed key characteristics of female reproductive toxicants as an approach for organizing and evaluating mechanistic data in hazard assessment. Environ Health Perspect 127(7):075001. doi:10.1289/EHP4971

Marshall, I. J., Wallace, B.C. (2019). Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev 8(1):163. doi:10.1186/s13643-019-1074-9

Martin, P., Bladier, C., Meek, B., et al. (2018). Weight of Evidence for Hazard Identification: A Critical Review of the Literature. Environ Health Perspect 126(7), 076001. doi:10.1289/EHP3067

Meek, M. E., Palermo, C. M., Bachman, A. N. et al. (2014). Mode of Action Human Relevance (MOA/HR) Framework – Evolution of the Bradford Hill Considerations and Comparative Analysis of Weight of Evidence. J Appl Toxicol 34: 595-606. doi:10.1002/jat.2984

Mellor, C. L., Steinmetz, F. P., Cronin, M. T. (2016). The identification of nuclear receptors associated with hepatic steatosis to develop and extend adverse outcome pathways. Crit Rev Toxicol 46(2):138-52. doi:10.3109/10408444.2015.1089471

Moberg, J., Oxman, A.D., Rosenbaum, S. et al. (2018). The GRADE Evidence to Decision (EtD) framework for health system and public health decisions. Health Res Policy Syst 16(1):45. doi:10.1186/s12961-018-0320-2

Morgan, R.L., Thayer, K.A., Bero, L. et al. (2016). GRADE: Assessing the quality of evidence in environmental and occupational health. Environ Int 92-93:611-6. doi:10.1016/j.envint.2016.01.004

NRC – National Research Council (2007). Toxicity Testing in the 21st Century: A Vision and Strategy. 2007. Washington (DC): National Academies Press. doi: doi:10.17226/11970

NRC (2014). Review of EPA's Integrated Risk Information System (IRIS) Process. 2014. Washington (DC): National Academies Press. doi:10.17226/18764

NRC (2017). Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals. 2017. Washington (DC): National Academies Press. doi:10.17226/24758

NTP – National Toxicology Program (2016). NTP Monograph on Immunotoxicity Associated with Exposure to Perfluorooctanoic acid (PFOA) and perfluorooctane sulfonate (PFOS). 2016. U.S. Department of Health and Human Services. Available at (last accessed on Feb 14, 2022)

NTP (2019). NTP Monograph on the Systematic Review of Occupational Exposure to Cancer Chemotherapy and Adverse Health Outcomes. 2019. U.S. Department of Health and Human Services. Available at: (last accessed December 20, 2019)

O'Connor, A.M., Glasziou, P., Taylor, M. et al. (2020). A focus on cross-purpose tools, automated recognition of study design in multiple disciplines, and evaluation of automation tools: a summary of significant discussions at the fourth meeting of the International Collaboration for Automation of Systematic Reviews (ICASR). Syst Rev 9(1):100. doi:10.1186/s13643-020-01351-4

OECD (2016). Guidance Document for the Use of Adverse Outcome Pathways in Developing Integrated Approaches to Testing and Assessment (IATA). OECD Series on Testing and Assessment, No. 260. OECD Publishing, Paris. doi:10.1787/44bb06c1-en.

OECD (2018). “Users’ Handbook Supplement to the Guidance Document for Developing and Assessing Adverse Outcome Pathways,” OECD Series on Adverse Outcome Pathways, No. 1. OECD Publishing, Paris, doi:10.1787/5jlv1m9d1g32-en.

Parmelli, E., Amato, L., Oxman, A. D. et al. (2017). GRADE EVIDENCE TO DECISION (EtD) FRAMEWORK FOR COVERAGE DECISIONS. Int J Technol Assess Health Care 33(2):176-182. doi:10.1017/S0266462317000447

Perkins, E. J., Ashauer, R., Burgoon, L. et al (2019). Building and Applying Quantitative Adverse Outcome Pathway Models for Chemical Hazard and Risk Assessment. Environ Toxicol Chem 38(9):1850-1865. doi:10.1002/etc.4505

Pistollato, F., Madia, F., Corvi, R. et al. (2021). Current EU regulatory requirements for the assessment of chemicals and cosmetic products: challenges and opportunities for introducing new approach methodologies. Arch Toxicol 95, 1867–1897. doi:10.1007/s00204-021-03034-y

Rehfuess, E. A., Booth, A., Brereton, L., et al. (2018). Towards a taxonomy of logic models in health technology assessments; A priori, staged and iterative approaches. Res Synth Methods 9:13-24. doi:10.1002/jrsm.1254

Rhomberg, L. R., Goodman, J. E., Bailey, et al. (2013). A survey of frameworks for best practices in weight-of-evidence analyses. Crit Rev Toxicol 43(9), 753–784. doi:10.3109/10408444.2013.832727

Rooney, A. A., Boyles, A. L., Wolfe, M.S. et al. (2014). Systematic review and evidence integration for literature-based environmental health science assessments. Environ Health Perspect 122(7):711-8. doi:10.1289/ehp.1307972

Rooney, A. A., Boyles, A. L., Walker, V.R. (2015). Systematic Review, An Illustration of Increased Transparency in a Framework for Evaluating Immunotoxicity Associated with PFOA and PFOS Exposure. In Toxicological Effects of Perfluoroalkyl and Polyfluoroalkyl Substances (pp. 419-449). Humana Press, Cham. doi:10.1007/978-3-319-15518-0_16

Samuel, G. O., Hoffmann, S., Wright, R. A. et al. (2016). Guidance on assessing the methodological and reporting quality of toxicologically relevant studies: A scoping review. Environ Int 92-93:630-46. doi:10.1016/j.envint.2016.03.010

Schneider, K., Schwarz, M., Burkholder, I. et al. (2009). “ToxRTool”, a new tool to assess the reliability of toxicological data. Toxicol Lett 189(2):138-44. doi:10.1016/j.toxlet.2009.05.013

Schünemann, H., Hill., S, Guyatt, G. et al. (2011). The GRADE approach and Bradford Hill's criteria for causation. J Epidemiol Community Health 65(5):392-5. doi:10.1136/jech.2010.119933

Smith, M., T., Guyton, K. Z., Gibbons, C.F. et al. (2016). Key characteristics of carcinogens as a basis for organizing data on mechanisms of carcinogenesis. Environ Health Perspect 124(6):713-21. doi:10.1289/ehp.1509912

Tabernilla, A., Dos Santos Rodrigues, B., Pieters, A. et al. (2021). In Vitro Liver Toxicity Testing of Chemicals: A Pragmatic Approach. Int J Mol Sci 10;22(9):5038. doi:10.3390/ijms22095038

Tollefsen, K. E., Scholz, S., Cronin, M. T. et al. (2014). Applying Adverse Outcome Pathways (AOPs) to support Integrated Approaches to Testing and Assessment (IATA). Regul Toxicol Pharmacol 70(3):629-40. doi:10.1016/j.yrtph.2014.09.009

Vesterinen, H.M., Johnson, P.I., Atchley, D. S. et al. (2015). Fetal growth and maternal glomerular filtration rate: a systematic review. J Matern Fetal Neonatal Med 28(18):2176-81. doi:10.3109/14767058.2014.980809

Villeneuve, D. L., Crump, D., Garcia-Reyero, N. et al. (2014). Adverse Outcome Pathway (AOP) Development I: Strategies and Principles. Toxicol Sci 142 (2): 312–20. doi:10.1093/toxsci/kfu199

Viswanathan, M., Ansari, M., Berkman, N. D. et al. (2012). Assessing the risk of bias of individual studies when comparing medical interventions. Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Publication No. 12-EHC047-EF). Available at: (last accessed Feb 14, 2022)

Vinken, M., Landesmann, B., Goumenou, M. et al. (2013). Development of an adverse outcome pathway from drug-mediated bile salt export pump inhibition to cholestatic liver injury. Toxicol Sci 136(1):97-106. doi:10.1093/toxsci/kft177

Whaley, P., Piggott, T., Morgan, R. L. et al. (2022). Biological plausibility in environmental health systematic reviews: a GRADE concept paper. J Clin Epidem, S0895-4356(22)00050-6. Advance online publication. doi:10.1016/j.jclinepi.2022.02.011

Wittwehr, C., Amorim, M.J., Clerbaux, L.A. et al. (2021). Understanding COVID-19 through adverse outcome pathways - 2nd CIAO AOP Design Workshop. ALTEX 38(2):351-357. doi:10.14573/altex.2102221

Wolffe, T. A. M., Whaley, P., Halsall, C. et al (2019). Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management. Environ Int 130:104871. doi:10.1016/j.envint.2019.05.065

Woodruff, T.J. and Sutton P (2014). The Navigation Guide systematic review methodology: a rigorous and transparent method for translating environmental health science into better health outcomes. Environ Health Perspect 122(10):1007-14. doi:10.1289/ehp.1307175

Most read articles by the same author(s)

1 2 3 4 5 6 7 8 9 10 > >>