The long way from raw data to NAM-based information: Overview on data layers and processing steps
Article Sidebar
Main Article Content
Abstract
Toxicological test methods generate raw data and provide instructions on how to use these to determine a final outcome such as a classification of test compounds as hits or non-hits. The data processing pipeline provided in the test method description is often highly complex. Usually, multiple layers of data, ranging from a machine-generated output to the final hit definition, are considered. Transition between each of these layers often requires several data processing steps. As changes in any of these processing steps can impact the final output of new approach methods (NAMs), the processing pipeline is an essential part of a NAM description and should be included in reporting templates such as the ToxTemp. The same raw data, processed in different ways, may result in different final outcomes that may affect the readiness status and regulatory acceptance of the NAM, as an altered output can affect robustness, performance, and relevance. Data management, processing, and interpretation are therefore important elements of a comprehensive NAM definition. We aim to give an overview of the most important data levels to be considered during the development and application of a NAM. In addition, we illustrate data processing and evaluation steps between these data levels. As NAMs are increasingly standard components of the spectrum of toxicological test methods used for risk assessment, awareness of the significance of data processing steps in NAMs is crucial for building trust, ensuring acceptance, and fostering the reproducibility of NAM outcomes.
Plain language summary
Toxicological test methods initially generate raw data. These need to be further processed to determine a final outcome, such as the classification of test compounds as hits or non-hits. The processing of the raw data is often highly complex and proceeds stepwise. This process generates many layers of data connected by several processing steps. Any change to these processing steps can impact the final output of new approach methods (NAMs). This means that the same raw data, processed in different ways, may result in different final outcomes. Data management, processing and interpretation are therefore considered important elements of a comprehensive NAM definition. We illustrate data processing and evaluation steps that play an important role. Awareness of the significance of data processing steps in NAMs is crucial for building trust, ensuring acceptance, and fostering the reproducibility of NAM outcomes.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Articles are distributed under the terms of the Creative Commons Attribution 4.0 International license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is appropriately cited (CC-BY). Copyright on any article in ALTEX is retained by the author(s).
Aschner, M., Ceccatelli, S., Daneshian, M. et al. (2017). Reference compounds for alternative test methods to indicate developmental neurotoxicity (DNT) potential of chemicals: Example lists and criteria for their selection and use. ALTEX 34, 49-74. doi:10.14573/altex.1604201
Bal-Price, A., Hogberg, H. T., Crofton, K. M. et al. (2018). Recommendation on test readiness criteria for new approach methods in toxicology: Exemplified for developmental neurotoxicity. ALTEX 35, 306-352. doi:10.14573/altex.1712081
Beattie, K. A., Verma, M., Brennan, R. J. et al. (2024). Quantitative systems toxicology modeling in pharmaceutical research and development: An industry-wide survey and selected case study examples. CPT Pharmacometrics Syst Pharmacol 13, 2036-2051. doi:10.1002/psp4.13227
Bell, S., Abedini, J., Ceger, P. et al. (2020). An integrated chemical environment with tools for chemical safety testing. Toxicol In Vitro 67, 104916. doi:10.1016/j.tiv.2020.104916
Blum, J., Masjosthusmann, S., Bartmann, K. et al. (2023). Establishment of a human cell-based in vitro battery to assess developmental neurotoxicity hazard of chemicals. Chemosphere 311, 137035. doi:10.1016/j.chemosphere.2022.137035
Callegaro, G., Kunnen, S. J., Trairatphisan, P. et al. (2021). The human hepatocyte TXG-MAPr: Gene co-expression network modules to support mechanism-based risk assessment. Arch Toxicol 95, 3745-3775. doi:10.1007/s00204-021-03141-w
Casey, W. M., Chang, X., Allen, D. G. et al. (2018). Evaluation and optimization of pharmacokinetic models for in vitro to in vivo extrapolation of estrogenic activity for environmental chemicals. Environ Health Perspect 126, 97001. doi:10.1289/ehp1655
Chang, X., Tan, Y. M., Allen, D. G. et al. (2022). IVIVE: Facilitating the use of in vitro toxicity data in risk assessment and decision making. Toxics 10, 232. doi:10.3390/toxics10050232
Collen, E., Tanaskov, Y., Holzer, A. K. et al. (2024). Elements and development processes for test methods in toxicology and human health-relevant life science research. ALTEX 41, 142-148. doi:10.14573/altex.2401041
Crofton, K. M., Mundy, W. R., Lein, P. J. et al. (2011). Developmental neurotoxicity testing: Recommendations for developing alternative methods for the screening and prioritization of chemicals. ALTEX 28, 9-15. doi:10.14573/altex.2011.1.009
Crouzet, T., Grignard, E., Brion, F. et al. (2023). ReadEDTest: A tool to assess the readiness of in vitro test methods under development for identifying endocrine disruptors. Environ Int 174, 107910. doi:10.1016/j.envint.2023.107910
Daniel, A. B., Choksi, N., Abedini, J. et al. (2022). Data curation to support toxicity assessments using the integrated chemical environment. Front Toxicol 4, 987848. doi:10.3389/ftox.2022.987848
Delp, J., Gutbier, S., Cerff, M. et al. (2018). Stage-specific metabolic features of differentiating neurons: Implications for toxicant sensitivity. Toxicol Appl Pharmacol 354, 64-80. doi:10.1016/j.taap.2017.12.013
Delp, J., Cediel-Ulloa, A., Suciu, I. et al. (2021). Neurotoxicity and underlying cellular changes of 21 mitochondrial respiratory chain inhibitors. Arch Toxicol 95, 591-615. doi:10.1007/s00204-020-02970-5
Di Tillio, F. and Beltman, J. B. (2024). Developing quantitative adverse outcome pathways: An ordinary differential equation-based computational framework. Comput Toxicol 32, 100330. doi:10.1016/j.comtox.2024.100330
Ebmeyer, J., Najjar, A., Lange, D. et al. (2024). Next generation risk assessment: An ab initio case study to assess the systemic safety of the cosmetic ingredient, benzyl salicylate, after dermal exposure. Front Pharmacol 15, 1345992. doi:10.3389/fphar.2024.1345992
Ferrario, D., Brustio, R. and Hartung, T. (2014). Glossary of reference terms for alternative test methods and their validation. ALTEX 31, 319-335. doi:10.14573/altex.1403311
Feshuk, M., Kolaczkowski, L., Dunham, K. et al. (2023). The ToxCast pipeline: Updates to curve-fitting approaches and database structure. Front Toxicol 5, 1275980. doi:10.3389/ftox.2023.1275980
Filer, D. L., Kothiya, P., Setzer, R. W. et al. (2017). tcpl: The ToxCast pipeline for high-throughput screening data. Bioinformatics 33, 618-620. doi:10.1093/bioinformatics/btw680
Gabbert, S., Mathea, M., Kolle, S. N. et al. (2022). Accounting for precision uncertainty of toxicity testing: Methods to define borderline ranges and implications for hazard assessment of chemicals. Risk Anal 42, 224-238. doi:10.1111/risa.13648
Harrill, J. A., Viant, M. R., Yauk, C. L. et al. (2021). Progress towards an OECD reporting framework for transcriptomics and metabolomics in regulatory toxicology. Regul Toxicol Pharmacol 125, 105020. doi:10.1016/j.yrtph.2021.105020
Hartung, T. (2009). Toxicology for the twenty-first century. Nature 460, 208-212. doi:10.1038/460208a
Hartung, T., De Vries, R., Hoffmann, S. et al. (2019). Toward good in vitro reporting standards. ALTEX 36, 3-17. doi:10.14573/altex.1812191
Hartung, T. (2023a). ToxAIcology – The evolving role of artificial intelligence in advancing toxicology and modernizing regulatory science. ALTEX 40, 559-570. doi:10.14573/altex.2309191
Hartung, T. (2023b). Artificial intelligence as the new frontier in chemical risk assessment. Front Artif Intell 6, 1269932. doi:10.3389/frai.2023.1269932
Hartung, T., King, N. M. P., Kleinstreuer, N. et al. (2024). Leveraging biomarkers and translational medicine for preclinical safety – Lessons for advancing the validation of alternatives to animal testing. ALTEX 41, 545-566. doi:10.14573/altex.2410011
Holzer, A. K., Dreser, N., Pallocca, G. et al. (2023). Acceptance criteria for new approach methods in toxicology and human health-relevant life science research – Part I. ALTEX 40, 706-712. doi:10.14573/altex.2310021
Jochum, K., Miccoli, A., Sommersdorf, C. et al. (2024). Comparative case study on NAMs: Towards enhancing specific target organ toxicity analysis. Arch Toxicol 98, 3641-3658. doi:10.1007/s00204-024-03839-7
Kappenberg, F., Brecklinghaus, T., Albrecht, W. et al. (2020). Handling deviating control values in concentration-response curves. Arch Toxicol 94, 3787-3798. doi:10.1007/s00204-020-02913-0
Kappenberg, F., Grinberg, M., Jiang, X. et al. (2021). Comparison of observation-based and model-based identification of alert concentrations from concentration-expression data. Bioinformatics 37, 1990-1996. doi:10.1093/bioinformatics/btab043
Kessel, H. E., Masjosthusmann, S., Bartmann, K. et al. (2023). The impact of biostatistics on hazard characterization using in vitro developmental neurotoxicity assays. ALTEX 40, 619-634. doi:10.14573/altex.2210171
Kleinstreuer, N. C., Ceger, P., Watt, E. D. et al. (2017). Development and validation of a computational model for androgen receptor activity. Chem Res Toxicol 30, 946-964. doi:10.1021/acs.chemrestox.6b00347
Kleinstreuer, N. C., Hoffmann, S., Alépée, N. et al. (2018). Non-animal methods to predict skin sensitization (II): An assessment of defined approaches. Crit Rev Toxicol 48, 359-374. doi:10.1080/10408444.2018.1429386
Kleinstreuer, N. and Hartung, T. (2024). Artificial intelligence (AI) – It’s the end of the tox as we know it (and I feel fine). Arch Toxicol 98, 735-754. doi:10.1007/s00204-023-03666-2
Knudsen, T., Martin, M., Chandler, K. et al. (2013). Predictive models and computational toxicology. Methods Mol Biol 947, 343-374. doi:10.1007/978-1-62703-131-8_26
Krebs, A., Nyffeler, J., Rahnenfuhrer, J. et al. (2018). Normalization of data for viability and relative cell function curves. ALTEX 35, 268-271. doi:10.14573/1803231
Krebs, A., Waldmann, T., Wilks, M. F. et al. (2019). Template for the description of cell-based toxicological test methods to allow evaluation and regulatory use of the data. ALTEX 36, 682-699. doi:10.14573/altex.1909271
Krebs, A., Nyffeler, J., Karreman, C. et al. (2020a). Determination of benchmark concentrations and their statistical uncertainty for cytotoxicity test data and functional in vitro assays. ALTEX 37, 155-163. doi:10.14573/altex.1912021
Krebs, A., van Vugt-Lussenburg, B. M. A., Waldmann, T. et al. (2020b). The EU-ToxRisk method documentation, data processing and chemical testing pipeline for the regulatory use of new approach methods. Arch Toxicol 94, 2435-2461. doi:10.1007/s00204-020-02802-6
Krishna, S., Berridge, B. and Kleinstreuer, N. (2021). High-throughput screening to identify chemical cardiotoxic potential. Chem Res Toxicol 34, 566-583. doi:10.1021/acs.chemrestox.0c00382
Krug, A. K., Balmer, N. V., Matt, F. et al. (2013). Evaluation of a human neurite growth assay as specific screen for developmental neurotoxicants. Arch Toxicol 87, 2215-2231. doi:10.1007/s00204-013-1072-y
La, V. N. T., Nicholson, S., Haneef, A. et al. (2023). Inclusion of control data in fits to concentration-response curves improves estimates of half-maximal concentrations. J Med Chem 66, 12751-12761. doi:10.1021/acs.jmedchem.3c00107
Lasch, A., Lichtenstein, D., Marx-Stoelting, P. et al. (2020). Mixture effects of chemicals: The difficulty to choose appropriate mathematical models for appropriate conclusions. Environ Pollut 260, 113953. doi:10.1016/j.envpol.2020.113953
Leist, M., Efremova, L. and Karreman, C. (2010). Food for thought ... Considerations and guidelines for basic test method descriptions in toxicology. ALTEX 27, 309-317. doi:10.14573/altex.2010.4.309
Leist, M., Hasiwa, N., Daneshian, M. et al. (2012). Validation and quality control of replacement alternatives – Current status and future challenges. Toxicol Res 1, 8-22. doi:10.1039/c2tx20011b
Leist, M., Hasiwa, N., Rovida, C. et al. (2014). Consensus report on the future of animal-free systemic toxicity testing. ALTEX 31, 341-356. doi:10.14573/altex.1406091
Leist, M. and Hengstler, J. G. (2018). Essential components of methods papers. ALTEX 35, 429-432. doi:10.14573/altex.1807031
Leontaridou, M., Urbisch, D., Kolle, S. N. et al. (2017). The borderline range of toxicological methods: Quantification and implications for evaluating precision. ALTEX 34, 525-538. doi:10.14573/altex.1606271
Leontaridou, M., Gabbert, S. and Landsiedel, R. (2019). The impact of precision uncertainty on predictive accuracy metrics of non-animal testing methods. ALTEX 36, 435-446. doi:10.14573/altex.1810111
Lynch, C., Sakamuru, S., Ooka, M. et al. (2024). High-throughput screening to advance in vitro toxicology: Accomplishments, challenges, and future directions. Annu Rev Pharmacol Toxicol 64, 191-209. doi:10.1146/annurev-pharmtox-112122-104310
Maertens, A., Golden, E., Luechtefeld, T. H. et al. (2022). Probabilistic risk assessment – The keystone for the future of toxicology. ALTEX 39, 3-29. doi:10.14573/altex.2201081
Magel, V., Blum, J., Dolde, X. et al. (2024). Inhibition of neural crest cell migration by strobilurin fungicides and other mitochondrial toxicants. Cells 13, 2057. doi:10.3390/cells13242057
Mansouri, K., Moreira-Filho, J. T., Lowe, C. N. et al. (2024). Free and open-source QSAR-ready workflow for automated standardization of chemical structures in support of QSAR modeling. J Cheminform 16, 19. doi:10.1186/s13321-024-00814-3
Najjar, A., Kuhnl, J., Lange, D. et al. (2024a). Next-generation risk assessment read-across case study: Application of a 10-step framework to derive a safe concentration of daidzein in a body lotion. Front Pharmacol 15, 1421601. doi:10.3389/fphar.2024.1421601
Najjar, A., Wilm, A., Meinhardt, J. et al. (2024b). Evaluation of new alternative methods for the identification of estrogenic, androgenic and steroidogenic effects: A comparative in vitro/in silico study. Arch Toxicol 98, 251-266. doi:10.1007/s00204-023-03616-y
NTP (2018). NTP research report on National Toxicology Program approach to genomic dose-response modeling: NTP RR 5. Durham, NC. National Toxicology Program (5), 1-44. doi:10.22427/ntp-rr-5
OECD (2005). Guidance Document on the Validation and International Acceptance of New or Updated Test Methods for Hazard Assessment. OECD Series on Testing and Assessment, No. 34. OECD Publishing, Paris. doi:10.1787/e1f1244b-en
OECD (2017a). Guidance Document for the Use of Adverse Outcome Pathways in Developing Integrated Approaches to Testing and Assessment (IATA). OECD Series on Testing and Assessment, No. 260. OECD Publishing, Paris. doi:10.1787/44bb06c1-en
OECD (2017b). Guidance Document for Describing Non-Guideline In Vitro Test Methods. OECD Series on Testing and Assessment, No. 211. OECD Publishing, Paris. doi:10.1787/9789264274730-en
OECD (2018). Guidance Document on Good In Vitro Method Practices (GIVIMP). OECD Series on Testing and Assessment, No. 286. OECD Publishing, Paris. doi:10.1787/9789264304796-en
OECD (2021). Guideline No. 497: Defined Approaches on Skin Sensitisation. OECD Guidelines fot the Testing of Chemicals, Section 4. OECD Publishing Paris. doi:10.1787/b92879a4-en
OECD (2023). OECD Omics Reporting Framework (OORF): Guidance on reporting elements for the regulatory use of omics data from laboratory-based toxicology studies. OECD Series on Testing and Assessment, No. 390. OECD Publishing, Paris. doi:10.1787/6bb2e6ce-en
OECD (2024). (Q)SAR assessment framework: Guidance for the Regulatory Assessment of (Quantitative) Structure Activity Relationship Models and Predictions, Second Edition. OECD Series on Testing and Assessment, No. 405. doi:10.1787/bbdac345-en
Pallocca, G. and Leist, M. (2022). On the usefulness of animals as a model system (part II): Considering benefits within distinct use domains. ALTEX 39, 531-539. doi:10.14573/altex.2207111
Pamies, D., Leist, M., Coecke, S. et al. (2022). Guidance document on good cell and tissue culture practice 2.0 (GCCP 2.0). ALTEX 39, 30-70. doi:10.14573/altex.2111011
Perkins, E. J., Ashauer, R., Burgoon, L. et al. (2019). Building and applying quantitative adverse outcome pathway models for chemical hazard and risk assessment. Environ Toxicol Chem 38, 1850-1865. doi:10.1002/etc.4505
Petersen, E. J., Elliott, J. T., Gordon, J. et al. (2023). Technical framework for enabling high quality measurements in new approach methodologies (NAMs). ALTEX 40, 174-186. doi:10.14573/altex.2205081
Polak, S., Tylutki, Z., Holbrook, M. et al. (2019). Better prediction of the local concentration-effect relationship: The role of physiologically based pharmacokinetics and quantitative systems pharmacology and toxicology in the evolution of model-informed drug discovery and development. Drug Discov Today 24, 1344-1354. doi:10.1016/j.drudis.2019.05.016
Reinke, E. N., Reynolds, J., Gilmour, N. et al. (2025). The skin allergy risk assessment-integrated chemical environment (sara-ice) defined approach to derive points of departure for skin sensitization. Curr Res Toxicol 8, 100205. doi:10.1016/j.crtox.2024.100205
Samuel, G. O., Hoffmann, S., Wright, R. A. et al. (2016). Guidance on assessing the methodological and reporting quality of toxicologically relevant studies: A scoping review. Environ Int 92-93, 630-646. doi:10.1016/j.envint.2016.03.010
Schmeisser, S., Miccoli, A., von Bergen, M. et al. (2023). New approach methodologies in human regulatory toxicology – Not if, but how and when! Environ Int 178, 108082. doi:10.1016/j.envint.2023.108082
Schmidt, B. Z., Lehmann, M., Gutbier, S. et al. (2017). In vitro acute and developmental neurotoxicity screening: An overview of cellular platforms and high-throughput technical possibilities. Arch Toxicol 91, 1-33. doi:10.1007/s00204-016-1805-9
Smirnova, L., Hogberg, H. T., Leist, M. et al. (2024). Revolutionizing developmental neurotoxicity testing – A journey from animal models to advanced in vitro systems. ALTEX 41, 152-178. doi:10.14573/altex.2403281
Strickland, J., Truax, J., Corvaro, M. et al. (2022). Application of defined approaches for skin sensitization to agrochemical products. Front Toxicol 4, 852856. doi:10.3389/ftox.2022.852856
Sturla, S. J. (2018). Point of departure. Chem Res Toxicol 31, 2-3. doi:10.1021/acs.chemrestox.7b00341
Suciu, I., Delp, J., Gutbier, S. et al. (2023). Definition of the neurotoxicity-associated metabolic signature triggered by berberine and other respiratory chain inhibitors. Antioxidants (Basel) 13, 49. doi:10.3390/antiox13010049
Tsaioun, K., Blaauboer, B. J. and Hartung, T. (2016). Evidence-based absorption, distribution, metabolism, excretion (ADME) and its interplay with alternative toxicity methods. ALTEX 33, 343-358. doi:10.14573/altex.1610101
Verheijen, M., Tong, W., Shi, L. et al. (2020). Towards the development of an omics data analysis framework. Regul Toxicol Pharmacol 112, 104621. doi:10.1016/j.yrtph.2020.104621
Watt, E. D. and Judson, R. S. (2018). Uncertainty quantification in ToxCast high throughput screening. PLoS One 13, e0196963. doi:10.1371/journal.pone.0196963
Wheeler, M. W. (2023). An investigation of non-informative priors for Bayesian dose-response modeling. Regul Toxicol Pharmacol 141, 105389. doi:10.1016/j.yrtph.2023.105389
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J. et al. (2016). The FAIR guiding principles for scientific data management and stewardship. Sci Data 3, 160018. doi:10.1038/sdata.2016.18
Zhu, T., Cao, S., Su, P. C. et al. (2013). Hit identification and optimization in virtual screening: Practical recommendations based on a critical literature analysis. J Med Chem 56, 6560-6572. doi:10.1021/jm301916b
Zobl, W., Bitsch, A., Blum, J. et al. (2024). Protectiveness of NAM-based hazard assessment – Which testing scope is required? ALTEX 41, 302-319. doi:10.14573/altex.2309081