EFSA’s framework for evidence-based scientific assessments: A case study on uncertainty analysis

Main Article Content

Elisa Aiassa
Caroline Merten
Laura Martino


To provide a sound scientific advice in support of the European decision-making process in food and feed safety, the European Food Safety Authority (EFSA) defined the principles for producing ‘evidence-based scientific assessments’ (impartiality, methodological rigour, transparency and engagement) and, to help fulfilling them, developed cross-cutting methodological approaches. This paper focusses on two of these approaches: conducting scientific assessments in four steps - with an emphasis on developing a protocol for the assessment a priori - and analysing uncertainty. An overview of the 4-step approach and the methods for addressing uncertainty is given and a case-study on uncertainty analysis, developed in collaboration with the German Federal Institute for Risk Assessment, is illustrated. The main advantage related to the implementation of protocols and uncertainty analysis is the increase in the scientific value of the outputs. However, experience and further capacity-building is needed to better incorporate uncertainty analysis in the planning phase (protocol) of the scientific assessment process. The case-study is based on exposure in humans. Nonetheless it provides an example of a framework for evidence-based scientific assessments that is applicable also to other types of evidence, including the one arising from New Approach Methodologies. Adopting the proposed framework, that covers in the planning and implementation phase an analysis of the uncertainties, is also expected to foster the integration of multiple evidence sources, including alternative methods and testing strategies, in the regulatory scientific assessment process.

Article Details

How to Cite
Aiassa, E., Merten, C. and Martino, L. (2021) “EFSA’s framework for evidence-based scientific assessments: A case study on uncertainty analysis”, ALTEX - Alternatives to animal experimentation. doi: 10.14573/altex.2004211.

Bell, S.M., Chang, X., Wambaugh, J.F. et al. (2018). In vitro to in vivo extrapolation for high throughput prioritization and decision making, Toxicol in Vitro 47, 213-227. doi:10.1016/j.tiv.2017.11.016

Casati, S. (2018): Integrated Approaches to Testing and Assessment. Basic & Clinical Pharmacology & Toxicology 123, 51–55. doi:10.1111/bcpt.13018

EC (2002). Regulation (EC) No 178/2002 of the European Parliament and of the Council of 28 January 2002 laying down the general principles and requirements of food law, establishing the European Food Safety Authority and laying down procedures in matters of food safety. https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=celex%3A32002R0178

EC - European Commission, Chief Scientific Advisors, Scientific Advice Mechanism (SAM),. (2019). Scientific Advice to European Policy in a Complex World. doi:10.2777/80320

EFSA - European Food Safety Authority (2010). Application of systematic review methodology to food and feed safety assessments to support decision making. Retrieved from http://www.efsa.europa.eu/en/efsajournal/pub/1637.htm

EFSA (2014). Guidance on expert knowledge elicitation in food and feed safety risk assessment (1831-4732). Retrieved from http://www.efsa.europa.eu/en/efsajournal/pub/3734

EFSA (2015a). Principles and process for dealing with data and evidence in scientific assessments. Retrieved from http://www.efsa.europa.eu/en/efsajournal/pub/4121

EFSA (2015b). Editorial: Increasing robustness, transparency and openness of scientific assessments. EFSA J 13, e13031. doi:10.2903/j.efsa.2015.e13031

EFSA (2016): EFSA Strategy 2020. Trusted science for safe food. Protecting consumers’ health with independent scientific advice on the food chain. European Food Safety Authority. http://bookshop.europa.eu/en/efsa-strategy-2020-pbTM0116289/

EFSA (2017). Workshop on the trial of the EFSA guidance document on uncertainty analysis in scientific assessments 22 - 23 June 2017 Parma, Italy (2397-8325). Retrieved from https://efsa.onlinelibrary.wiley.com/doi/abs/10.2903/sp.efsa.2017.EN-1313

EFSA (2018a). Implementation of PROMETHEUS 4-step approach for evidence use in EFSA scientific assessments: benefits, issues, needs and solutions (2397-8325). Retrieved from https://efsa.onlinelibrary.wiley.com/doi/abs/10.2903/sp.efsa.2018.EN-1395

EFSA (2018b). Evaluation of the impact of glyphosate and its residues in feed on animal health. EFSA J 16, e05283. doi:10.2903/j.efsa.2018.5283

EFSA (2018c). Protocol for the scientific opinion on the Tolerable Upper Intake Level of dietary sugars. EFSA J 16, e05393. doi:10.2903/j.efsa.2018.5393

EFSA (2019). Guidance on communication of uncertainty in scientific assessments. EFSA J 17, e05520. doi:10.2903/j.efsa.2019.5520

EFSA and BfR. (2019). International Conference on Uncertainty in Risk Analysis. EFSA Supporting Publication, 1689. Retrieved from https://www.efsa.europa.eu/en/supporting/pub/en-1689

EFSA, Gundert-Remy, U., Bodin, J. et al. (2017). Bisphenol A (BPA) hazard assessment protocol. EFSA Supporting Publications 14, 1354E. doi:10.2903/sp.efsa.2017.EN-1354

EFSA, Schendel Thomas, J. C., Lindtner, O. et al. (2018). Guidelines for Uncertainty Analysis: Application of the respective documents of EFSA and BfR for exposure assessments. EFSA supporting publication 2018: 15( 7): EN‐1472. 113 pp. doi: 10.2903/sp.efsa.2018.EN‐1472 (2397-8325). Retrieved from https://efsa.onlinelibrary.wiley.com/doi/abs/10.2903/sp.efsa.2018.EN-1472

EFSA Panel on Animal Health Welfare (2017). Bluetongue: control, surveillance and safe movement of animals. EFSA J 15, e04698. doi:10.2903/j.efsa.2017.4698

EFSA Panel on Biological Hazards (2016). Evaluation of the safety and efficacy of Listex™ P100 for reduction of pathogens on different ready-to-eat (RTE) food products. EFSA J 14, e04565. doi:10.2903/j.efsa.2016.4565

EFSA Panel on Contaminants in the Food Chain (2018). Risk for animal and human health related to the presence of dioxins and dioxin-like PCBs in feed and food. EFSA J 16, e05333. doi:10.2903/j.efsa.2018.5333

EFSA Panel on Food Additives, Flavoring, Processing Aids and Food Contact Materials. (2008). Scientific Opinion on the Safety of aluminium from dietary intake - Scientific Opinion (1831-4732). Retrieved from https://efsa.onlinelibrary.wiley.com/doi/abs/10.2903/j.efsa.2008.754

EFSA Panel on Food Additives and Nutrient Sources added to Food (2015). Risk assessment for peri- and post-menopausal women taking food supplements containing isolated isoflavones. EFSA J 13, 4246. doi:10.2903/j.efsa.2015.4246

EFSA Panel on Nutrition Novel Foods (2019). Dietary reference values for sodium. EFSA J 17, e05778. doi:10.2903/j.efsa.2019.5778

EFSA Panel on Plant Health (2017). Pest risk assessment of Eotetranychus lewisi for the EU territory. EFSA Journal, 15(10), e04878. doi:10.2903/j.efsa.2017.4878

EFSA Scientific Committee (2018a). Guidance on Uncertainty Analysis in Scientific Assessments. EFSA Journal, 16(1), 39. doi:10.2903/j.efsa.2018.5123

EFSA Scientific Committee (2018b). Scientific Opinion on the principles and methods behind EFSA’s Guidance on Uncertainty Analysis in Scientific Assessment on The principles and methods behind EFSA's Guidance on Uncertainty Analysis in Scientific Assessment. EFSA J 16, e05122. doi:10.2903/j.efsa.2018.5122

EFSA Scientific Committee (2017a). Guidance on the use of the weight of evidence approach in scientific assessments. EFSA J 15, 4971, 15, 69. doi:10.2903/j.efsa.2017.4971

EFSA Scientific Committee (2017b). Guidance on the assessment of the biological relevance of data in scientific assessments. EFSA Journal 2017; 15,4970, 15, 73. doi:10.2903/j.efsa.2017.4970

EU ANSA, (2018). Approaches to assess and manage scientific uncertainty: examples from EU ANSA Agencies. Retrieved from https://publications.europa.eu/en/publication-detail/-/publication/9880c8bc-83eb-11e8-ac6a-01aa75ed71a1.

Groothuis, F. A., Heringa M .B., Nicol, B. et al. (2015): dose metric considerations in in vitro assays to improve quantitative in vitro- in vivo dose extrapolations. Toxicology, 332, 30-40. doi:10.1016/j.tox.2013.08.012

Heinemeyer G., Kreienbrock, L., Schümann, M. et al. (2015). Guidelines on Uncertainty Analysis in Exposure Assessments. Recommendation of the Committee for Exposure Assessment and Standardisation of the Federal Institute for Risk Assessment (BfR). Series: BfR-Wissenschaft, Vol. 03/2015, 56 pp. Berlin, DE: Bundesinstitut für Risikobewertung (BfR). Retrieved from https://www.bfr.bund.de/cm/350/guidelines-on-uncertainty-analysis-in-exposure-assessments.pdf

Heseker, H., Oeppining, A., and Vohmann, C. (2003). Verzehrsstudie zur Ermittlung der Lebensmittelaufnahme von Säuglingen und Kleinkindern für die Abschätzung eines akuten Toxizitätsrisikos durch Rückstände von Pflanzenschutzmitteln (VELS). Paderborn DE: Universität Paderborn. doi.org/10.1007/s00003-006-0118-y

Higgins J. P. T, Thomas, J., Chandler, J. et al. (editors). (2019). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Retrieved from https://training.cochrane.org/handbook

ICCVAM – Interagency coordinating Committee on the validation of alternative methods (2017): A Strategic Roadmap for Establishing New Approaches to Evaluate the Safety of Chemicals and Medical Products in the United States. https://ntp.niehs.nih.gov/ntp/about_ntp/bsc/2017/december/meetingmaterials/iccvam_roadmap_508.pdf

IPCS - International Programme on Chemical Safety (2017). Guidance Document on Evaluating and Expressing Uncertainty in Hazard Assessment, second edition. IPCS Harmonization Project Document No. 11. World Health Organisation, Geneva. Retrieved from http://www.inchem.org/documents/harmproj/harmproj/harmproj11.pdf.

Kerr, N. L. (1998). HARKing: Hypothesizing After the Results are Known. Personality and Social Psychology Review 2, 196-217. doi:10.1207/s15327957pspr0203_4

Munafo, M. R., Nosek, B. A., Bishop, D. V. M., et al. (2017). A manifesto for reproducible science. Nature Human Behaviour 1. doi:UNSP 002110.1038/s41562-016-0021

NRC – National Research Council (2007): Toxicity Testing in the 21st Century: A Vision and a Strategy. Washington, DC: The National Academies Press. doi:10.17226/11970.

OHAT-NTP. (2019). Handbook for Conducting a Literature-Based Health Assessment Using OHAT Approach for Systematic Review and Evidence Integration. Available at: https://ntp.niehs.nih.gov/ntp/ohat/pubs/handbookmarch2019_508.pdf.

SAPEA (Science Advice for Policy by European Academies). (2019). Making sense of science for policy under conditions of complexity and uncertainty. Berlin: SAPEA. doi:10.26356/MASOS.

Shamseer, L., Moher, D., Clarke, M. et al. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ, 349, g7647. doi:10.1136/bmj.g7647

Wetmore, B. A. (2015): Quantitative in vitro-to-in vivo extrapolation in a high-throughput environment. Toxicology, 332, 94-101. doi:10.1016/j.tox.2014.05.012

Wilk-Zasadna, I., Bernasconi, C., Pelkonen, O., and Coecke, S. (2015). Biotransformation in vitro: An essential consideration in the quantitative in vitro-to-in vivo extrapolation (QIVIVE) of toxicity data. Toxicology 332, 8–19. doi:10.1016/j.tox.2014.10.006

Woodruff, T. J., and Sutton, P. (2014). The Navigation Guide systematic review methodology: a rigorous and transparent method for translating environmental health science into better health outcomes. Environ Health Perspect, 122(10), 1007-1014. doi:10.1289/ehp.1307175

WHO - World Health Organisation (2012). WHO Handbook for Guideline Development. Retrieved from http://apps.who.int/iris/bitstream/10665/75146/1/9789241548441eng.pdf