Food for Thought ... on Validation . A Puzzle or a Mystery : an Approach Founded on New Science

Over the past two decades, requisite criteria have been established for an in vitro method to be “validated”, and formal processes have been created through which a proposed test method is evaluated prior to being considered for regulatory acceptance (Locke and Goldberg, 2006). The European Union, the United States and others have established processes by which new test methods are reviewed and formally declared valid. It must be emphasized, however, that formal validation does not guarantee regulatory acceptance. Moreover, regulators do not necessarily require test methods to be validated by a formal process; instead, authorities responsible for ensuring methods used for substantiating safety simply need to be convinced that the proposed method performs to its intended use in measuring the endpoint(s) in question for regulatory approval. The EU was first to establish a formal validation process by creating the European Centre for the Validation of Alternative Methods (ECVAM1), which was established to conduct research, develop non-animal methods, and implement validation studies. The EU Cosmetic Directive2, which was amended seven times and then “recast” into a consolidated version of the Cosmetics Directive3, prohibits animal testing for ingredients, final formulations, and for marketing cosmetic products. ECVAM’s European Scientific Advisory Committee (ESAC1) reviews proposed non-animal methods, and when appropriate, declares them as valid for their intended use. These methods are also reviewed by the European Commission’s Scientific Committee on Consumer Safety (SCCS4), which provides a recommendation as to the validity of a test method. Furthermore, the methods that have been Summary The notion of test method validation, when applied to in vitro models, has been subjected to standards, practices, and regulatory mandates that may represent significant barriers to the development of new methods based on target toxicological pathways, mode of action, and mechanistic endpoints. Currently, there is an expectation that toxicity test methods used for regulatory purposes must be formally validated by a legislated body. In this manuscript, we review the genesis of current formal validation programs, their purpose, their justification for future need, and an examination of the scientific process as the driving force. With the collision of three major international events in the scientific and regulatory communities (viz. EU Cosmetic Directive, EU REACH legislation, and the NRC publication, Toxicity Testing in the 21st Century: A Vision and a Strategy), there may be a need for a shift in the current validation paradigm. Further, test method validity may be recognized by the appropriate authorities when specific criteria and the fundamental precepts of reliability and reproducibility have been adequately met.


Food for Thought … on Validation. A Puzzle or a Mystery: an Approach Founded on New Science
Neil Wilcox1 and Alan Goldberg 21 Office of Cosmetics and Colors, Food and Drug Administration, College Park, MD, USA; 2 Johns Hopkins University Bloomberg School of Public Health, Center for Alternatives to Animal Testing, Baltimore, MD, USA

Background
Over the past two decades, requisite criteria have been established for an in vitro method to be "validated", and formal processes have been created through which a proposed test method is evaluated prior to being considered for regulatory acceptance (Locke and Goldberg, 2006).The European Union, the United States and others have established processes by which new test methods are reviewed and formally declared valid.It must be emphasized, however, that formal validation does not guarantee regulatory acceptance.Moreover, regulators do not necessarily require test methods to be validated by a formal process; instead, authorities responsible for ensuring methods used for substantiating safety simply need to be convinced that the proposed method performs to its intended use in measuring the endpoint(s) in question for regulatory approval.
The EU was first to establish a formal validation process by creating the European Centre for the Validation of Alternative Methods (ECVAM 1 ), which was established to conduct research, develop non-animal methods, and implement validation studies.The EU Cosmetic Directive 2 , which was amended seven times and then "recast" into a consolidated version of the Cosmetics Directive3 , prohibits animal testing for ingredients, final formulations, and for marketing cosmetic products.ECVAM's European Scientific Advisory Committee (ESAC 1 ) reviews proposed non-animal methods, and when appropriate, declares them as valid for their intended use.These methods are also reviewed by the European Commission's Scientific Committee on Consumer Safety (SCCS 4 ), which provides a recommendation as to the validity of a test method.Furthermore, the methods that have been

Summary
The notion of test method validation, when applied to in vitro models, has been subjected to standards, practices, and regulatory mandates that may represent significant barriers to the development of new methods based on target toxicological pathways, mode of action, and mechanistic endpoints.Currently, there is an expectation that toxicity test methods used for regulatory purposes must be formally validated by a legislated body.In this manuscript, we review the genesis of current formal validation programs, their purpose, their justification for future need, and an examination of the scientific process as the driving force.With the collision of three major international events in the scientific and regulatory communities (viz.EU Cosmetic Directive, EU REACH legislation, and the NRC publication, Toxicity Testing in the 21 st Century: A Vision and a Strategy), there may be a need for a shift in the current validation paradigm.Further, test method validity may be recognized by the appropriate authorities when specific criteria and the fundamental precepts of reliability and reproducibility have been adequately met.

Keywords: in vitro, 21 st century toxicity testing, test method validation
Disclaimer: The views expressed in this manuscript are those of the author(s) alone.They do not represent the views or opinions of the US FDA or any entity affiliated with the US FDA.
Internationally, 30 countries are members of the OECD, which publishes test guidelines (TG) when consensus is reached between all member nations.This process is recognized by its participating members.Although they are obliged to accept data generated by OECD TG's15 , each member country retains the right to request additional data from other test methods, including animal models, when deemed necessary.

New toxicology
The 2007 National Research Council's seminal publication, Toxicity Testing in the 21 st Century: A Vision and a Strategy (NRC Tox21c, 2007), clearly has set a new direction focusing on mechanistic toxicological endpoints in lieu of empirical animal testing.Since current formal validation processes tend to use data generated from animal models as their "gold standard" for comparison, it begs the question as to whether these processes are adequately designed for toxicity testing of the future.As a result, it is appropriate to re-examine the basis upon which formal validation processes currently are established.Few will challenge the need for new methods that are more predictive for relevant toxicological endpoints in the target species of interest, viz., humans.Currently, ECVAM and ICCVAM are charged with validating alternative methods through formal processes, and there is a general assumption in the toxicological community that such formal validation is requisite for regulatory acceptance, which is not always the case.Although regulators require proposed safety testing methods to be valid for their intended use, and expert review is desirable, validation through a formal process is not required by the regulatory agencies.
It is certainly preferred to have a standardized test that has undergone a formal validation process available for hazard identification, but this is impractical for the more specific dimensions of the risk assessment process, viz., exposure assessment and risk characterization.When evaluating a safety assessment for a regulated product, the test developer must demonstrate that the exposure characteristics (viz., dose, route of administration and duration of exposure) are safe for their intended use or claim (FD&C Act16 ).The point here is that the product must be declared validated based upon these ESAC reviews are submitted to the appropriate EU institutions, such as the European Food Safety Authority (EFSA5 ), the European Chemical Agency (EChA6 ), the European Pharmacopoeia7 , and the European Medicines Evaluation Agency (EMEA8 ), as well as the Organization for Economic and Co-operative Development The US Government established the Interagency Coordinating Committee on the Validation of Alternative Methods (ICC-VAM12 ), which mandated, in part, to establish a formal process by which in vitro methods would be validated and recommended for regulatory acceptance.ICCVAM establishes expert working groups from within its 15 member agencies, who in turn collaborate with the NTP Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM 12 ).When defined acceptance criteria have been met, ICCVAM will recommend review of the method by a Federal Advisory Committee that will accept, modify, or reject the ICCVAM recommendation.Finally, if the Federal Advisory Committee considers the method valid for a particular application or intended use, ICCVAM will submit the committee's recommendation to its member agencies, who decide individually whether they will formally accept the method.Acceptance of the method does not require the agency to use this method exclusively, and reviewers are allowed to request whatever additional information is needed, which may include animal testing.
Japan created the Japanese Centre for the Validation of Alternative Methods (JaCVAM 13 ), and Korea has recently established the Korean Center for the Validation of Alternative Methods (KoCVAM 14 ), as part of the National Institute of Food and Drug Safety (NIFDS) in the Korean Food and Drug Administration.Both JaCVAM and KoCVAM participate in the international arena on alternatives to animal testing.Although Canada does not have a center dedicated to methods validation, they do have a review process in place.evolution of new test methods were identified: test development; prevalidation; validation (involving a formal inter-laboratory study with the testing of coded chemicals); independent assessment; and progression toward regulatory acceptance.ECVAM has implemented a prevalidation scheme, which includes three main phases: protocol refinement, protocol transfer, and protocol performance (Balls et al., 1995).Goldberg et al. suggested an approach to validation based on routine scientific approaches and methods development (Goldberg et al., 1993(Goldberg et al., , 1997a,b),b).This approach will be useful for mechanistic-based assays.To improve the formal system, ECVAM ( 2004) published the "Modular Approach to the ECVAM Principles on Test Validity" that makes the validation process more flexible by breaking down the various steps in validation into independent modules and defining for each module the information needed for assessing test validity -a significant improvement (Hartung et al., 2004).
In 1999, ICCVAM published its Evaluation of the Validation Status of Toxicological Methods: General Guidelines for Submissions to ICCVAM (ICCVAM, 1999).This publication also contains ICCVAM Validation and Regulatory Acceptance Criteria as part of an appendix.The Acceptance Criteria list several important items that need to be met depending on a method's intended use.These guidelines were heavily influenced by the ECVAM approach but gave a clearer statement of the necessary criteria and offered methods developers some specific guidance.
In 2009, at the Developmental Neurotoxicity (DNT 2) workshop hosted by the Johns Hopkins University Center for Alternatives to Animal Testing (CAAT17 ) in Washington DC, Crofton et al. identified criteria and instructions for methods developers that would allow a method to fully meet validation criteria, if developed with the principles identified.In fact, if methods met all criteria identified, they would, by definition, be scientifically validated methods (Crofton et al., 2011).
In developing the toxicology of the 21 st century and a resultant human toxicology based on the in vitro use of human cells in culture, the newest technologies -omics, high throughput and high content systems -have progressed in most cases.However, the ability to interpret and incorporate these approaches into an industrial or regulatory scheme has not yet become incorporated.Thus, one could ask, "Is validation a puzzle or a mystery?" substantiated for safety based upon its intended use, and it is up to the manufacturer to ensure to the satisfaction of the regulator that the product is safe.This scenario precludes the development of a toxicity testing method to adequately demonstrate the safety of all products, which are vast in their numbers with wide and varied applications.As a result, the individual(s) (e.g., safety assessor) responsible for satisfying regulatory requirements for safety need(s) to address each product individually; similarly, regulators need to review each product on its own merits and ensure that the safety assessor has developed a portfolio demonstrating that the test method is relevant to the specific claim and measures the appropriate endpoints to substantiate safety.In other words, the manufacturer needs to determine that the product is safe under the prescribed conditions of use.As stated earlier, however, there is still a place for the formal validation of methods used in hazard testing.
As a result, regulatory agencies such as the FDA and the EPA, among others, do not require that a method used to substantiate safety of a proposed product be "formally validated" through either the ECVAM or ICCVAM process.FDA Centers with premarket authority may be able to determine that a proposed method is "valid" for its intended use by requiring the sponsor of a proposed FDA-regulated product to submit data to this end.

The validation process
Validation is the process by which the reliability and relevance of a procedure are established for a specific purpose.(Frazier, 1990;Balls et al., 1990).More importantly, it is not a process to develop new approaches, optimize approaches, or compare one approach to another.It is a process that verifies the method or procedure in question performs as intended.
In the early stages of the formal validation process, methods were included that did not meet the above standard.Thus, in 1995, based upon experience gained during several large-scale validation studies, ECVAM published recommendations concerning the practical and logistical aspects of validating alternative test methods (ECVAM workshop report 5).Five main stages in the These phenomena have created a world-wide renaissance in toxicological testing and, in turn, have driven a new industry pursuing novel testing models and targeting relevant endpoints.As a result, technological developments in test method design characterizing human exposure to chemicals of toxicological concern have proceeded rapidly and will continue to do so in the foreseeable future.Consequently, government organizations (viz., ECVAM and ICCVAM) originally established to validate non-animal test methods, will need not only to prepare for an increase in demand for reviewing methods as they are developed at an unprecedented pace, but will also need to actively contribute to establishing new validation models for methods targeting mechanistic and pathway-based toxicity as envisioned in the NRC Tox21c.Participation in this initiative by ICCVAM, ECVAM and JaCVAM is imperative, as one would anticipate the transition to new toxicity testing models and new validation strategies to occur over many years.

Science and progress
Toxicology evolved over time as a trial and error activity initially implemented by our ancestors as a survival technique.Paracelsus characterized toxicology when he coined the now well-known axiom, "All substances are poisons, there is none which is not a poison.The right dose differentiates a poison from a remedy."Toxicology has since been formalized as a scientific discipline with its roots in the industrial revolution commencing in the 19 th century and evolving into various toxicological disciplines in the mid to late 20 th century.It was during this latter time that standardized animal tests were developed as methods for safety testing prior to human use or environmental application.
Although these methods have served an important role in public health, they were empirical in nature and were rarely linked to understanding the underlying mechanism of action eliciting the untoward reaction or to knowing whether the animal response was relevant to humans.As a result, a "safety factor" was incorporated into the assessment algorithm to account for the unknown gap between exposure and toxicity.One example of calculating such a "margin of safety" (MOS) is to divide the "no observable adverse effect level" (NOAEL) by the "systemic exposure dose" (SED), with the MOS required to be a factor of at least 100.This standard approach to safety testing has been used for decades; unfortunately, it has led to a culture of accepting empirical data without challenging the relevance of the data in humans and with little understanding of the underlying mechanistic basis for the potential toxicity.
This approach has become such an accepted standard that when scientists started suggesting that mechanistically-based, in vitro test methods would be superior (and reduce animal suffering), the idea was dismissed by many as not good science.As This concept is wonderfully described by Malcolm Gladwell in his recent book, What the Dog Saw, in a chapter called "Open Secrets" (Gladwell, 2009).Puzzles are 'transmitter-dependent'; they turn on what we know, thus a puzzle grows simpler with the addition of each new piece of information.Mysteries, on the other hand are 'receiver-dependent'; they are solved by the skill of the receiver or seeker of the information.To understand mysteries involves experience and insight.(Think of Columbo or Charlie Chan as a model).
The puzzle aspect of in vitro toxicology is to provide the methods that allow the collection of data for the evaluation of chemicals and their risk assessment, enabling one to make a decision.Developmental neurotoxicity testing (DNT) improves with each new piece of information.The DNT workshops18 add new information and provide a more complete picture, but DNT remains a puzzle.
Conversely, acute toxicity testing, skin irritation, allergic contact dermatitis (hypersensitivity testing), and possibly endocrine disruptors have all the necessary pieces (assays) in order to solve these mysteries available in the published literature (McKim et al., 2010; also see 19 ).Some methods are better than others, and it is the receiver who has to decide which works for the products they are reviewing or testing.In these four areas, acute toxicity, irritation, hypersensitivity, and endocrine disruption testing, the available assays are open secrets -they are available for all to see.Fortunately, many are beginning to see the open secrets that will help interpret the data.

The "perfect storm"
The premise upon which the current approaches to validation were founded is sound but did not anticipate three international events that would have a profound impact on the research and development of methods intended to replace animals in toxicological testing: The EU Cosmetic Regulation (formerly known as the 7 th Amendment to the Cosmetic Directive); the EU Registration, Evaluation, and Authorization of Chemicals (REACH 20 ) program; and the U.S. National Research Council's seminal publication, Toxicity in the 21 st Century: A Vision and a Strategy.These separate and unrelated events are driving the need to examine the use of new and developing science in addition to traditional animal model approaches of safety testing.These independent events, although significantly different in their intended goals, have created the "perfect storm" in the ubiquitous arena of toxicology.They have converged from vastly different directions and resulted in a common outcome -to drive the eventual replacement of animal testing, an approach based upon empiricism and correlative relationships extrapolated across species, with a science-based approach targeting toxicological mechanisms and pathways applicable to humans.mentioned above, several events evolved that have now resulted in broad scientific agreement that non-animal methods may be the best way forward because they will allow scientists to understand the mechanisms of toxicity, including metabolism of test compounds and interaction between organ systems.
The notion of validating non-animal tests was conceived on the premise that the tests must be able to predict toxicity, as well as a standardized reference.This resulted in a default approach of identifying a test method to act as a "gold standard" against which the proposed method would be compared to an established validated method.This process incorporates the philosophy of "replacing one test method with another"; in other words, one animal test would be replaced with one non-animal test, despite the fact that many scientists proclaimed the need for a "battery of in vitro tests" to replace an animal test (Goldberg, 1987).The premise for such an approach was an attempt to duplicate the pathway or cascade of events illustrative of the in vivo response to the toxicant.
The current approach to validation requires the proposed in vitro test method to identify specific endpoint(s), which quite likely will be a cellular mechanism, and to compare that against an established in vivo method.For example, a human corneal epithelial (HCE-T) model for ocular irritancy measures alterations in the barrier function.In vitro data, averaged from replicate assays, were compared to respective Draize rabbit eye irritation data using linear regression with Pearson's correlation analysis.Data indicated that barrier function alterations in the HCE-T model correlated with ocular irritancy and corneal toxicity (Kruszewski et al., 1997).This type of correlation effectively illustrates the current approach to the formal validation process; that is, an ocular mechanism of irritation, corneal cell barrier function, is compared to subjective rabbit eye irritation data.In the event that the HCE-T model were to be validated against the Draize Eye test, one would be comparing mechanistic data generated from human corneal cells in vitro against empirical data observed in the rabbit eye in vivo.This is a good example of why the NRC Tox21c, which calls for a paradigm shift in toxicity testing, by default demands a new approach in the validation process as well.Rather than attempting to validate in vitro mechanistic data against in vivo observational data, the NRC vision requires that the mechanism and/or pathway that leads to a toxic reaction should be understood first, and then a method to measure the perturbation of the mechanism or pathway would be developed.Clearly, if this latter scenario is followed, the animal data, where the fundamental mechanism of action is unknown, would not be used for comparison, and a new validation model would be required to substantiate the validity of the new mechanistic model.This example represents a current approach to validation; that is, retrofitting in vitro mechanistic data against in vivo data of unknown etiology at the cellular and molecular levels.With that said, modeling an in vitro system that mimics the whole animal and the exceedingly complex interactions between organ systems will be a major challenge as this vision is implemented over time.
Although the current approach to formal validation worked well to establish criteria necessary for validation, it was founded on the perceived reliability and accuracy of animal data without knowing the underlying mechanisms or pathways leading to toxicity.Such an approach had the inherent disadvantages of animal-to-animal variations, not to mention the myriad problems associated with extrapolating animal data to humans.

New science
While scientific discovery in biology has advanced in multiple disciplines over the past several years, traditional animal testing has remained a steadfast approach to safety testing of chemicals and formulations from a regulatory perspective.As eloquently stated in the NRC Tox21c vision, there are many tools and technologies that are either currently available or in the development stages that will move the science of toxicity testing forward.These new technologies eventually will obviate the need for animal testing, and they will be superior to animal testing.They will allow specific mechanisms and pathways to be identified that will predict toxicity a priori, as opposed to empirical observations after the fact in animals.Computational methods (e.g., structure-activity-relationships -SAR) are available for characterizing chemicals based upon known physiological properties that allow biological activity such as potential toxicity to be predicted.Similarly, there have been many advances in cellular and molecular biology, omics technologies (e.g., bioinformatics, genomics, proteomics and robotics) that, along with computational analysis, have led to a better understanding of toxicity pathways and already have resulted in advances in rapid "high-throughput" in vitro systems used to screen potential pharmaceutical candidates.Envisaged in the NRC report are in vitro assays, which play a significant role in the overall process of eventually replacing animals.The clear advantage of this vision is that scientists will be able to predict toxicity based upon an in-depth knowledge of the sequence of events at the cellular, subcellular, and molecular levels in specific metabolic pathways, leading to rapid advancements in all types of products -from life-saving drugs to cosmetics -and greatly reducing the time, money, and animals heretofore required (van Vliet, 2011).

The way forward
The current formalized approach to validation involves lengthy and expensive processes that require validating in vitro data against in vivo (animal) data, which may or may not be relevant (OECD 9 ) (involving the EU and US National Coordinators for Test Guideline Program), the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH 10 ), and the International Cooperation on Harmonization of Technical Requirements for Registration of Veterinary Products (VICH 11 ) for formal acceptance.