Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do

Main Article Content

Paul Whaley
Bas J. Blaauboer
Jan Brozek
Elaine A. Cohen Hubal
Kaitlyn Hair
Sam Kacew
Thomas B. Knudsen
Carol F. Kwiatkowski
David T. Mellor
Andrew F. Olshan
Matthew J. Page
Andrew A. Rooney
Elizabeth G. Radke
Larissa Shamseer
Katya Tsaioun
Peter Tugwell
Daniele Wikoff
Tracey J. Woodruff


Systematic reviews are fast increasing in prevalence in the toxicology and environmental health literature. However, how well these complex research projects are being conducted and reported is unclear. Since editors have an essential role in ensuring the scientific quality of manuscripts being published in their journals, a workshop was convened where editors, systematic review practitioners, and research quality control experts could discuss what editors can do to ensure the systematic reviews they publish are of sufficient scientific quality. Interventions were explored along four themes: setting standards; reviewing protocols; optimizing editorial workflows; and measuring the effectiveness of editorial interventions. In total, 58 editorial interventions were proposed. Of these, 26 were shortlisted for being potentially effective, and 5 were prioritized as short-term actions that editors could relatively easily take to improve the quality of published systematic reviews. Recent progress in improving systematic reviews is summarized, and outstanding challenges to further progress are highlighted.

Article Details

How to Cite
Whaley, P., Blaauboer , B. J., Brozek, J., Cohen Hubal, E. A., Hair, K., Kacew, S., Knudsen , T. B., Kwiatkowski, C. F., Mellor, D. T., Olshan, A. F., Page, M. J., Rooney, A. A., Radke, E. G., Shamseer , L. ., Tsaioun, K., Tugwell, P., Wikoff, D. and Woodruff, T. J. (2021) “Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do”, ALTEX - Alternatives to animal experimentation, 38(3), pp. 513-522. doi: 10.14573/altex.2106111.
Meeting Reports

Blanco, D., Altman, D., Moher, D. et al. (2019). Scoping review on interventions to improve adherence to reporting guidelines in health research. BMJ Open 9, e026589. doi:10.1136/bmjopen-2018-026589

Brazma, A., Hingamp, P., Quackenbush, J. et al. (2001). Minimum information about a microarray experiment (MIAME)-toward standards for microarray data. Nat Gen 29, 365-371. doi:10.1038/ng1201-365

Cohen Hubal, E. A., Frank, J. J., Nachman, R. et al. (2020). Advancing systematic-review methodology in exposure science for environmental health decision making. J Exp Sci Environ Epidemiol 30, 906-916. doi:10.1038/s41370-020-0236-0

French, S. D., Green, S. E., O’Connor, D. A. et al. (2012). Developing theory-informed behaviour change interventions to implement evidence into practice: A systematic approach using the theoretical domains framework. Implement Sci 7, 38. doi:10.1186/1748-5908-7-38

Galipeau, J., Barbour, V., Baskin, P. et al. (2016). A scoping review of competencies for scientific editors of biomedical journals. BMC Med 14, 16. doi:10.1186/s12916-016-0561-2

Goldkuhle, M., Narayan, V. M., Weigl, A. et al. (2018). A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer. BMJ Open 8, e020869. doi:10.1136/bmjopen-2017-020869

Heim, A., Ravaud, P., Baron, G. et al. (2018). Designs of trials assessing interventions to improve the peer review process: A vignette-based survey. BMC Med 16, 191. doi:10.1186/s12916-018-1167-7

Higgins, J. P., Lasserson, T., Chandler, J. et al. (2019a). Methodological Expectations of Cochrane Intervention Reviews (MECIR). Cochrane.

Higgins, J. P., Thomas, J., Chandler, J. et al. (2019b). Cochrane Handbook for Systematic Reviews of Interventions. Wiley. doi:10.1002/9781119536604.

Hoffmann, S., de Vries, R. B. M., Stephens, M. L. et al. (2017). A primer on systematic reviews in toxicology. Arch Toxicol 91, 2551-2575. doi:10.1007/s00204-017-1980-3

Ioannidis, J. P. A. (2016). The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 94, 485-514. doi:10.1111/1468-0009.12210

Moher, D., Liberati, A., Tetzlaff, J. et al. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med 6, e1000097. doi:10.1371/journal.pmed.1000097

Moher, D., Galipeau, J., Alam, S. et al. (2017). Core competencies for scientific editors of biomedical journals: Consensus statement. BMC Med 15, 167. doi:10.1186/s12916-017-0927-0

Nosek, B. A. and Lakens, D. (2014). Registered reports: A method to increase the credibility of published results. Soc Psychol 45, 137-141. doi:10.1027/1864-9335/a000192

Nosek, B. A., Alter, G., Banks, G. C. et al. (2015). Scientific standards. Promoting an open research culture. Science 348, 1422-1425. doi:10.1126/science.aab2374

Nyhan, K., Haugh, D., Nardini, H. G. et al. (2020). Librarian Peer Reviewer Database. Connecting experts in systematic searching to the journal editors who need them.

Pachito, D. V., Pega, F., Bakusic, J. et al. (2021). The effect of exposure to long working hours on alcohol consumption, risky drinking and alcohol use disorder: A systematic review and meta-analysis from the WHO/ILO joint estimates of the work-related burden of disease and injury. Environ Int 146, 106205. doi:10.1016/j.envint.2020.106205

Page, M. J., Shamseer, L., Altman, D. G. et al. (2016). Epidemiology and reporting characteristics of systematic reviews of biomedical research: A cross-sectional study. PLoS Med 13, e1002028. doi:10.1371/journal.pmed.1002028

Page, M. J. and Moher, D. (2017). Evaluations of the uptake and impact of the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement and extensions: A scoping review. Syst Rev 6, 263. doi:10.1186/s13643-017-0663-8

Pega, F., Momen, N. C., Ujita, Y. et al. (2021). Systematic reviews and meta-analyses for the WHO/ILO joint estimates of the work-related burden of disease and injury. Environ Int 155, 106605. doi:10.1016/j.envint.2021.106605

Pollock, M., Fernandes, R. M. and Hartling, L. (2017). Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions. BMC Med Res Methodol 17, 48. doi:10.1186/s12874-017-0325-5

Pussegoda, K., Turner, L., Garritty, C. et al. (2017). Systematic review adherence to methodological or reporting quality. Syst Rev 6, 131. doi:10.1186/s13643-017-0527-2

Romeo, S., Zeni, O., Sannino, A. et al. (2021). Genotoxicity of radiofrequency electromagnetic fields: Protocol for a systematic review of in vitro studies. Environ Int 148, 106386. doi:10.1016/j.envint.2021.106386

Rooney, A. A., Boyles, A. L., Wolfe, M. S. et al. (2014). Systematic review and evidence integration for literature-based environmental health science assessments. Environ Health Perspect 122, 711-718. doi:10.1289/ehp.1307972

Sanchez, K. A., Foster, M., Nieuwenhuijsen, M. J. et al. (2020). Urban policy interventions to reduce traffic emissions and traffic-related air pollution: Protocol for a systematic evidence map. Environ Int 142, 105826. doi:10.1016/j.envint.2020.105826

Sgargi, D., Adam, B., Budnik, L. T. et al. (2020). Protocol for a systematic review and meta-analysis of human exposure to pesticide residues in honey and other bees’ products. Environ Res 186, 109470. doi:10.1016/j.envres.2020.109470

Sheehan, M. C. and Lam, J. (2015). Use of systematic review and meta-analysis in environmental health epidemiology: A systematic review and comparison with guidelines. Curr Environ Health Rep 2, 272-283. doi:10.1007/s40572-015-0062-z

Sheehan, M. C., Lam, J., Navas-Acien, A. et al. (2016). Ambient air pollution epidemiology systematic review and meta-analysis: A review of reporting and methods practice. Environ Int 92-93, 647-656. doi:10.1016/j.envint.2016.02.016

Simpson, J. A. and Weiner, E. S. (1989). The Oxford English Dictionary. 2nd edition. Oxford, UK: Clarendon.

Stevens, A., Shamseer, L., Weinstein, E. et al. (2014). Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: Systematic review. BMJ 348, g3804. doi:10.1136/bmj.g3804

Sutton, P., Chartres, N., Rayasam, S. D. et al. (2021). Reviews in environmental health: How systematic are they? Environ Int 152, 106473. doi:10.1016/j.envint.2021.106473

van Luijk, J. A. K. R., Popa, M., Swinkels, J. et al. (2019). Establishing a health-based recommended occupational exposure limit for nitrous oxide using experimental animal data – A systematic review protocol. Environ Res 178, 108711. doi:10.1016/j.envres.2019.108711

Verbeek, J., Oftedal, G., Feychting, M. et al. (2021). Prioritizing health outcomes when assessing the effects of exposure to radiofrequency electromagnetic fields: A survey among experts. Environ Int 146, 106300. doi:10.1016/j.envint.2020.106300

Whaley, P., Halsall, C., Ågerstrand, M. et al. (2016). Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations. Environ Int 92-93, 556-564. doi:10.1016/j.envint.2015.11.002

Whaley, P. and Isalski, M. (2020). CREST_Triage.

Whaley, P., Aiassa, E., Beausoleil, C. et al. (2020a). Recommendations for the conduct of systematic reviews in toxicology and environmental health research (COSTER). Environ Int 143, 105926. doi:10.1016/j.envint.2020.105926

Whaley, P., Edwards, S. W., Kraft, A. et al. (2020b). Knowledge organization systems for systematic chemical assessments. Environ Health Perspect 128, 125001. doi:10.1289/EHP6994

Woodruff, T. J. and Sutton, P. (2014). The navigation guide systematic review methodology: A rigorous and transparent method for translating environmental health science into better health outcomes. Environ Health Perspect 122, 1007-1014. doi:10.1289/ehp.1307175

Most read articles by the same author(s)

1 2 > >>