How evidence-based methodologies can help identify and reduce uncertainty in chemical risk assessment
Article Sidebar
Main Article Content
Abstract
Evidence-based methodology, in particular systematic review, is increasingly being applied in environmental, public, and occupational health to increase the transparency, comprehensiveness, and objectivity of the processes by which existing evidence is gathered, assessed, and synthesized in answering research questions. This development is also changing risk assessment practices and will impact the assessment of uncertainties in the evidence for risks to human health that are posed by exposure to chemicals. The potential of evidence-based methodology for characterizing uncertainties in risk assessment has been widely recognized, while its contribution to uncertainty reduction is yet to be fully elucidated. We therefore present some key aspects of the evidence-based approach to risk assessment, showing how they can contribute to the identification and the assessment of uncertainties. We focus on the pre-specification of an assessment methodology in a protocol, comprehensive search strategies, study selection using predefined eligibility criteria, critical appraisal of individual studies, and an evidence integration and uncertainty characterization process based on certainty of evidence frameworks that are well-established in health care research. We also provide examples of uncertainty in risk assessment and discuss how evidence-based methodology could address those. This perspective, which neither claims to be comprehensive nor complete, is intended to stimulate discussion of the topic and to motivate detailed exploration of how evidence-based methodology contributes to characterization of uncertainties, and how it will lead to uncertainty reduction in the conduct of health risk assessment.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Articles are distributed under the terms of the Creative Commons Attribution 4.0 International license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is appropriately cited (CC-BY). Copyright on any article in ALTEX is retained by the author(s).
Beck, N. B., Becker, R. A., Erraguntla, N. et al. (2016). Approaches for describing and communicating overall uncertainty in toxicity characterizations: U.S. Environmental Protection Agency’s integrated risk information system (IRIS) as a case study. Environ Int 89-90, 110-128. doi:10.1016/j.envint.2015.12.031
Benbrook, C. M. (2019). How did the US EPA and IARC reach diametrically opposed conclusions on the genotoxicity of glyphosate-based herbicides? Environ Sci Eur 31, 2. doi:10.1186/s12302-018-0184-7
Berkman, N. D., Santaguida, P. L., Viswanathan, M. et al. (2014). The Empirical Evidence of Bias in Trials Measuring Treatment Differences. https://www.ncbi.nlm.nih.gov/sites/books/NBK253184/
Bero, L., Chartres, N., Diong, J. et al. (2018). The risk of bias in observational studies of exposures (ROBINS-E) tool: Concerns arising from application to observational studies of exposures. Syst Rev 7, 242. doi:10.1186/s13643-018-0915-2
Bhat, V. S., Meek, M. E. B., Valcke, M. et al. (2017). Evolution of chemical-specific adjustment factors (CSAF) based on recent international experience; increasing utility and facilitating regulatory acceptance. Crit Rev Toxicol 47, 729-749. doi:10.1080/10408444.2017.1303818
Cano-Sancho, G., Ploteau, S., Matta, K. et al. (2019). Human epidemiological evidence about the associations between exposure to organochlorine chemicals and endometriosis: Systematic review and meta-analysis. Environ Int 123, 209-223. doi:doi:10.1016/j.envint.2018.11.065
Chvátalová, V. (2019). A critical evaluation of EFSA’s environmental risk assessment of genetically modified maize MON810 for honeybees and earthworms. Environ Sci Eur 31, 52. doi:10.1186/s12302-019-0238-5
Cooper, G. S., Lunn, R. M., Agerstrand, M. et al. (2016). Study sensitivity: Evaluating the ability to detect effects in systematic reviews of chemical exposures. Environ Int 92-93, 605-610. doi:10.1016/j.envint.2016.03.017
Dankovic, D. A., Naumann, B. D., Maier, A. et al. (2015). The Scientific Basis of Uncertainty Factors Used in Setting Occupational Exposure Limits. J Occup Environ Hyg 12, Suppl 1, S55-S68. doi:10.1080/15459624.2015.1060325
De Vries, R. B. M., Angrish, M., Browne, P. et al. (2021). Applying evidence-based methods to the development and use of adverse outcome pathways. ALTEX 38, 336-347. doi:10.14573/altex.2101211
Deveau, M., Chen, C.-P., Johanson, G. et al. (2015). The global landscape of occupational exposure limits – Implementation of harmonization principles to guide limit selection. J Occup Environ Hyg 12, Suppl 1, S127-144. doi:10.1080/15459624.2015.1060327
Dorman, D. C., Chiu, W., Hales, B. F. et al. (2018). Systematic reviews and meta-analyses of human and animal evidence of prenatal diethylhexyl phthalate exposure and changes in male anogenital distance. J Toxicol Environ Health B Crit Rev 21, 207-226. doi:10.1080/10937404.2018.1505354
Eddy, D. M. (2005). Evidence-based medicine: A unified approach. Health Aff (Millwood) 24, 9-17. doi:10.1377/hlthaff.24.1.9
EFSA (2010). Application of systematic review methodology to food and feed safety assessments to support decision making. EFSA J 8, 1637. doi:10.2903/j.efsa.2010.1637
EFSA (2015). Principles and process for dealing with data and evidence in scientific assessments. EFSA J 13, 4121. doi:10.2903/j.efsa.2015.4121
EFSA, Gundert-Remy, U., Bodin, J. et al. (2017). Bisphenol A (BPA) hazard assessment protocol. EFSA Supporting Publication 14, 1354E. doi:10.2903/sp.efsa.2017.EN-1354
EFSA Scientific Committee (2018a). Guidance on uncertainty analysis in scientific assessments. EFSA J 16, e05123. doi:10.2903/j.efsa.2018.5123
EFSA Scientific Committee (2018b). The principles and methods behind EFSA’s guidance on uncertainty analysis in scientific assessment. EFSA J 16, e05122. doi:10.2903/j.efsa.2018.5122
Gusenbauer, M. and Haddaway, N. R. (2021). What every researcher should know about searching – Clarified concepts, search advice, and an agenda to improve finding in academia. Res Synth Methods 12, 136-147. doi:10.1002/jrsm.1457
Guyatt, G. H., Oxman, A. D., Vist, G. E. et al. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336, 924-926. doi:10.1136/bmj.39489.470347.AD
Hassauer, C. and Roosen, J. (2020). Toward a conceptual framework for food safety criteria: Analyzing evidence practices using the case of plant protection products. Saf Sci 127, 104683. doi:10.1016/j.ssci.2020.104683
Hoffmann, S., De Vries, R. B. M., Stephens, M. L. et al. (2017). A primer on systematic reviews in toxicology. Arch Toxicol 9, 2551-2575. doi:10.1007/s00204-017-1980-3
Howard, B. E., Phillips, J., Tandon, A. et al. (2020). SWIFT-active screener: Accelerated document screening through active learning and integrated recall estimation. Environ Int 138, 105623. doi:10.1016/j.envint.2020.105623
Kabat, G. C. (2019). Rapid response to: Probable carcinogenicity of glyphosate. BMJ 365, l1613. doi:10.1136/bmj.l1613
Kass, G. E. N. and Lodi, F. (2020). Letter to the editor regarding the article ‘EFSA’s toxicological assessment of aspartame: Was it even-handedly trying to identify possible unreliable positives and unreliable negatives?’ Arch Public Health 78, 14. doi:10.1186/s13690-020-0395-4
Kogevinas, M. (2019). Probable carcinogenicity of glyphosate. BMJ 365, l1613. doi:10.1136/bmj.l1613
Koustas, E., Lam, J., Sutton, P. et al. (2014). The navigation guide – Evidence-based medicine meets environmental health: Systematic review of nonhuman evidence for PFOA effects on fetal growth. Environ Health Perspect 122, 1015-1027. doi:doi:10.1289/ehp.1307177
Lam, J., Koustas, E., Sutton, P. et al. (2014). The navigation guide – Evidence-based medicine meets environmental health: Integration of animal and human evidence for PFOA effects on fetal growth. Environ Health Perspect 122, 1040-1051. doi:10.1289/ehp.1307923
Li, J., Pega, F., Ujita, Y. et al. (2020). The effect of exposure to long working hours on ischaemic heart disease: A systematic review and meta-analysis from the WHO/ILO joint estimates of the work-related burden of disease and injury. Environ Int 142, 105739. doi:10.1016/j.envint.2020.105739
Lynch, H. N., Goodman, J. E., Tabony, J. A. et al. (2016). Systematic comparison of study quality criteria. Regul Toxicol Pharmacol 76, 187-198. doi:10.1016/j.yrtph.2015.12.017
Mahood, Q., Van Eerd, D. and Irvin, E. (2014). Searching for grey literature for systematic reviews: Challenges and benefits. Res Synth Methods 5, 221-234. doi:10.1002/jrsm.1106
Marshall, I. J. and Wallace, B. C. (2019). Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev 8, 163. doi:10.1186/s13643-019-1074-9
Martin, P., Bladier, C., Meek, B. et al. (2018). Weight of evidence for hazard identification: A critical review of the literature. Environ Health Perspect 126, 076001. doi:10.1289/EHP3067
Matta, K., Ploteau, S., Coumoul, X. et al. (2019). Associations between exposure to organochlorine chemicals and endometriosis in experimental studies: A systematic review protocol. Environ Int 124, 400-407. doi:10.1016/j.envint.2018.12.063
Meek, M. E., Boobis, A., Cote, I. et al. (2014). New developments in the evolution and application of the WHO/IPCS framework on mode of action/species concordance analysis. J Appl Toxicol 34, 1-18. doi:10.1002/jat.2949
Millstone, E. P. and Dawson, E. (2019). EFSA’s toxicological assessment of aspartame: Was it even-handedly trying to identify possible unreliable positives and unreliable negatives? Arch Public Health 77, 34. doi:10.1186/s13690-019-0355-z
Moher, D., Shamseer, L., Clarke, M. et al. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 4, 1. doi:10.1186/2046-4053-4-1
Morgan, R. L., Thayer, K. A., Bero, L. et al. (2016). GRADE: Assessing the quality of evidence in environmental and occupational health. Environ Int 92-93, 611-616. doi:10.1016/j.envint.2016.01.004
Morgan, R. L., Thayer, K. A., Bero, L. et al. (2018). Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes. Environ Int 121, 1027-1031. doi:10.1016/j.envint.2018.07.015
Morgan, R. L., Beverly, B., Ghersi, D. et al. (2019). GRADE guidelines for environmental and occupational health: A new series of articles in Environment International. Environ Int 128, 11-12. doi:10.1016/j.envint.2019.04.016
NRC – National Research Council (2014). Review of EPA’s Integrated Risk Information System (IRIS) Process. Washington, DC, USA: The National Academies Press.
NTP – National Toxicology Program (2015). Handbook for Preparing Report on Carcinogens Monographs. https://ntp.niehs.nih.gov/ntp/roc/handbook/roc_handbook_508.pdf
NTP (2016). Monograph on Immunotoxicity Associated With Exposure to Perfluorooctanoic Acid (PFOA) and Perfluorooctane Sulfonate (PFOS). https://ntp.niehs.nih.gov/ntp/ohat/pfoa_pfos/pfoa_pfosmonograph_508.pdf
OECD (2019). Guiding Principles and Key Elements for Establishing a Weight of Evidence for Chemical Assessment. OECD Series on Testing and Assessment, No. 311. OECD Publishing, Paris. https://doi.org/10.1787/f11597f6-en
Pega, F., Momen, N.C., Ujita, Y. et al. (2021). Systematic reviews and meta-analyses for the WHO/ILO joint estimates of the work-related burden of disease and injury. Environ Int 155, 106605. doi:10.1016/j.envint.2021.106605
RAC and SCOEL (2016). ECHA Committee for Risk Assessment (RAC) and Scientific Committee on Occupational Exposure Limits (SCOEL) Joint Opinion to resolve differences in scientific opinion as regards exposure levels for N-methyl-2-pyrrolidone. https://echa.europa.eu/documents/10162/13579/rac_joint_nmp_opinion_en.pdf/e4b4f43b-a3bd-a7c0-08be-16c3886593e7
Rhomberg, L. R., Goodman, J. E., Bailey, L. A. et al. (2013). A survey of frameworks for best practices in weight-of-evidence analyses. Crit Rev Toxicol 43, 753-784. doi:10.3109/10408444.2013.832727
Robinson, C., Portier, C. J., Cavoski, A. et al. (2020). Achieving a high level of protection from pesticides in Europe: Problems with the current risk assessment procedure and solutions. Eur J Risk Regul 11, 450-480. doi:10.1017/err.2020.18
Rooney, A. A., Boyles, A. L., Wolfe, M. S. et al. (2014). Systematic review and evidence integration for literature-based environmental health science assessments. Environ Health Perspect 122, 711-718. doi:10.1289/ehp.1307972
Rooney, A. A., Cooper, G. S., Jahnke, G. D. et al. (2016). How credible are the study results? Evaluating and applying internal validity tools to literature-based assessments of environmental health hazards. Environ Int 92-93, 617-629. doi:10.1016/j.envint.2016.01.005
Rudén, C. (2001). The use and evaluation of primary data in 29 trichloroethylene carcinogen risk assessments. Regul Toxicol Pharmacol 34, 3-16. doi:10.1006/rtph.2001.1482
Samuel, G. O., Hoffmann, S., Wright, R. A. et al. (2016). Guidance on assessing the methodological and reporting quality of toxicologically relevant studies: A scoping review. Environ Int 92-93, 630-646. doi:10.1016/j.envint.2016.03.010
Schenk, L. (2010). Comparison of data used for setting occupational exposure limits. Int J Occup Environ Health 16, 249-262. doi:10.1179/107735210799160255
Schenk, L. and Johanson, G. (2011). A quantitative comparison of the safety margins in the European indicative occupational exposure limits and the derived no-effect levels for workers under REACH. Toxicol Sci 121, 408-416. doi:10.1093/toxsci/kfr056
Schenk, L., Deng, U. and Johanson, G. (2015). Derived no-effect levels (DNELs) under the European chemicals regulation REACH – An analysis of long-term inhalation worker-DNELs presented by industry. Ann Occup Hyg 59, 416-438. doi:10.1093/annhyg/meu103
Schreider, J., Barrow, C., Birchfield, N. et al. (2010). Enhancing the credibility of decisions based on scientific conclusions: Transparency is imperative. Toxicol Sci 116, 5-7. doi:10.1093/toxsci/kfq102
Schulz, K. F., Chalmers, I., Hayes, R. J. et al. (1995). Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273, 408-412. doi:10.1001/jama.273.5.408
Schünemann, H., Hill, S., Guyatt, G. et al. (2011). The GRADE approach and Bradford Hill’s criteria for causation. J Epidemiol Community Health 65, 392-395. doi:10.1136/jech.2010.119933
Schünemann, H. J., Brozek, J., Guyat, G. et al. (2013). GRADE handbook for grading quality of evidence and strength of recommendations. https://gdt.gradepro.org/app/handbook/handbook.html
Stephens, M. L. et al. (2013). Evidence-based toxicology for the 21st century: Opportunities and challenges. ALTEX 30, 74-103. doi:10.14573/altex.2013.1.074
Stephens, M. L., Andersen, M., Becker, R. A. et al. (2016). The emergence of systematic review in toxicology. Toxicol Sci 152, 10-16. doi:10.1093/toxsci/kfw059
Thayer, K. A., Wolfe, M. S., Rooney, A. A. et al. (2014). Intersection of systematic review methodology with the NIH reproducibility initiative. Environ Health Perspect 122, A176-177. doi:10.1289/ehp.1408671
van Luijk, J., Popa, M., Swinkels, J. et al. (2019). Establishing a health-based recommended occupational exposure limit for nitrous oxide using experimental animal data – A systematic review protocol. Environ Res 178, 108711. doi:10.1016/j.envres.2019.108711
Whaley, P., Halsall, C., Agerstrand, M. et al. (2016). Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations. Environ Int 92-93, 556-564. doi:10.1016/j.envint.2015.11.002
Whaley, P., Aiassa, E., Beausoleil, C. et al. (2020a). Recommendations for the conduct of systematic reviews in toxicology and environmental health research (COSTER). Environ Int 143, 105926. doi:10.1016/j.envint.2020.105926
Whaley, P., Edwards, S. W., Kraft, A. et al. (2020b). Knowledge organization systems for systematic chemical assessments. Environ Health Perspect 128, 125001. doi:10.1289/EHP6994
Wikoff, D., Urban, J. D., Harvey, S. et al. (2018). Role of risk of bias in systematic review for chemical risk assessment: A case study in understanding the relationship between congenital heart defects and exposures to trichloroethylene. Int J Toxicol 37, 125-143. doi:10.1177/1091581818754330
Wikoff, D., Haws, L., Ring, C. et al. (2019). Application of qualitative and quantitative uncertainty assessment tools in developing ranges of plausible toxicity values for 2,3,7,8-tetrachlorodibenzo-p-dioxin. J Appl Toxicol 39, 1293-1310. doi:10.1002/jat.3814
Wikoff, D., Urban, J. D., Ring, C. et al. (2020a). Development of a range of plausible noncancer toxicity values for 2,3,7,8-tetrachlorodibenzo-p-dioxin based on effects on sperm count: Application of systematic review methods and quantitative integration of dose response using meta-regression. Toxicol Sci 179, 162-182. doi:10.1093/toxsci/kfaa171
Wikoff, D., Lewis, R. J., Erraguntla, H. et al. (2020b). Facilitation of risk assessment with evidence-based methods – A framework for use of systematic mapping and systematic reviews in determining hazard, developing toxicity values, and characterizing uncertainty. Regul Toxicol Pharmacol 118, 104790. doi:10.1016/j.yrtph.2020.104790
Wolffe, T. A. M., Whaely, P., Halsall, C. et al. (2019). Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management. Environ Int 130, 104871. doi:10.1016/j.envint.2019.05.065
Woodruff, T. J. and Sutton, P. (2014). The navigation guide systematic review methodology: A rigorous and transparent method for translating environmental health science into better health outcomes. Environ Health Perspect 122, 1007-1014. doi:10.1289/ehp.1307175
Yost, E. E., Euling, S. Y., Weaver, J. A. et al. (2019). Hazards of diisobutyl phthalate (DIBP) exposure: A systematic review of animal toxicology studies. Environ Int 125, 579-594. doi:10.1016/j.envint.2018.09.038