Beyond Animal Testing Index: Benchmarking Tool for a World Beyond Animal Testing

While the original definition of replacement focuses on the replacement of the use of animals in science, a more contemporary definition focuses on accelerating the development and use of predictive and robust models, based on the latest science and technologies, to address scientific questions without the use of animals. The transition to animal free innovation is on the political agenda in and outside the European Union. The Beyond Animal Testing Index (BATI) is a benchmarking instrument designed to provide insight into the activities and contribution of research institutes in the transition to animal free innovation. The BATI allows participating organizations to learn from each other and stimulates continuous improvement. The BATI was modelled after the Access to Medicine Index, which benchmarks pharmaceutical companies on their efforts to make medicines widely available in developing countries. A prototype of the BATI was field tested with three Dutch academic medical centers and two universities in 2020-2021. The field test demonstrated the usability and effectiveness of the BATI as a benchmarking tool. Analyses were performed across five different domains. The participating institutes concluded that the BATI served as an internal, as well as an external stimulus to share, learn and improve institutional strategies towards the transition to animal free innovation. The BATI also identified gaps in the development and implementation of 3R technologies. Hence, the BATI might be a suitable instrument for monitoring the effectiveness of policies. BATI version 1.0 is ready to be used for benchmarking at a larger scale.


Introduction
In September 2021, The European Parliament adopted a resolution calling the European Commission for plans and actions to accelerate the transition to innovation without the use of animals in research, regulatory testing, and education, or in short the transition to animal free innovation. 1 In its response, the European Commission considered that such a transition is best supported by focusing on and intensifying current efforts, reinforcing existing structures and networks, and by identifying potential areas where new actions could be undertaken within the Commission's remit. In 2019, the Dutch government launched the Transition Program for Innovation without the use of animals (TPI) after a policy advice by its National Committee for the protection of animals used for scientific purposes (NCad). 2,3 This program was recently extended until 2024. Although both programs have the similar aim to stimulate the transition to animal free innovations, neither of these two programs include the development of a comprehensive benchmark tool to monitor the progress and results of the transition to animal free innovations even though such a benchmarking tool was proposed by the project 'Faster from Innovation to Humans', a predecessor of TPI. 4 The societal, political, and scientific desire to reduce animal testing would benefit from a tool that allows visualization of the opportunities for this transition, its successes, and the identification of areas where additional work is required. The 'Beyond Animal Testing Index' (BATI) is such a tool. 5 The BATI has been specifically designed to provide insight in the efforts and contributions organizations make in the transition to animal free innovation. Furthermore, it allows organizations to learn from and inspire each other, and to provide organizations with incentives regarding the implementation of research practices without the use of animals for the benefit of science. The BATI ranks research organizations with regards to the full implementation of the concept of the 3Rs -Replacement, Reduction and Refinement of animal research updated by the UK National Centre for the 3Rs -culminating in the transition to animal free innovations. 6 The analytical framework of the BATI was drafted after that of the Access to Medicine Index (AMI). 7 The AMI benchmarks pharmaceutical companies regarding their effort to make medicines accessible in low-and middle-income countries and has proven to be successful in continuously improving the performance of organizations as is expected by society. The objectives of the BATI are similar to those of the AMI. These are to capture society's expectations regarding the transition to animal free innovations and to incentivize continuous improvement regarding the development and implementation of animal free innovations and the 3Rs.
The development of the BATI was initiated by the Leiden University Medical Centre (LUMC) and the University of Applied Sciences Utrecht (HU) in collaboration with the Netherlands Organization for Applied Scientific Research (TNO). In analogy with the Access to Medicine Index, a beta-version of the BATI was developed together with national and international stakeholders through multiple stakeholder consultations. Early 2019, a prototype of the BATI was introduced to representatives of the Netherlands Federation of Academic Medical Centers (NFU), Universities of the Netherlands, the Collective of Health Funds in the Netherlands (SGF), the NGO 'Proefdiervrij' (its mission is a future without animal testing), and ZonMW (its objective is to promote quality and innovation of health research in order to make health care better and to keep it ALTEX, accepted manuscript published August 8, 2023 doi:10.14573/altex.2304161 affordable). 8,9,10,11,12 The BATI was presented during a BATI-workshop at the FELASA congress in 2019 (Stegmeijer et al., 2019). The beta-version of the BATI was finalized at the end of 2019. The beta-version was put to the test with two universities and three academic medical centers in Groningen, Leiden, and Utrecht in the Netherlands in 2020-2021. This field test was funded by the Dutch ministry of Agriculture, Nature, and Food Quality (LNV), the dossier holder for animals used for scientific purposes.

BATI analytical framework
The analytical framework of the BATI consists of five domains: (A) Management and Stakeholders, (B) Research and Development, (C) Education and Training, (D) Investments and Grants, and (E) Intellectual Property. These domains represent aspects in which an organization can contribute to the transition to animal free innovations. The organization is assessed for its commitment to the transition, the level of transparency and its performance, the pillars of the framework. Performance covers the actual implementation of organizational policies with regards to the development and implementation of animal free innovations. Indicators, 29 in total, were developed to demarcate the data collection per domain or pillar. Each domain is divided into themes represented by different indicators each of which is distributed among one of the pillars (Table 1). The domains, themes and indicators are explained in supplementary material 1 13 . The indicator-based scoring matrix and scoring criteria are explained in supplementary material 2 14 . Indicators were formulated as statements representing targets based on the transition ambition. The formulation of the statements determined within which pillar indicators were ranked (I = commitment, II = transparency, III = performance). For the purpose of the field testing of the BATI prototype, all indicators were considered equally important, because there were no prior reasons to consider one indicator more important than another. As a result, a domain or pillar with a higher number of indicators contributed more to the ranking than a domain or pillar with less indicators. This may be reconsidered for future BATI editions in consultation with stakeholders. The contribution of each domain to the BATI score is presented in Fig.1. The contribution of each pillar to the BATI score is presented in Fig. 2. The absolute scores per indicator, on a scale from 1 to 5, were converted to a relative benchmark (Explained in more detail in supplementary material 3 15 ). Hence, the BATI results in a relative ranking of participating organizations.

Domain A -Management and stakeholders
This domain covers the governance and management structure of the organization from the top of the organization i.e., the Board of Directors to the lower management levels, as well as the goals and strategies adopted by the organization to accelerate the transition to animal free innovations, to stimulate innovation in research, and to ensure full implementation of the 3Rs. It ALTEX, accepted manuscript published August 8, 2023 doi:10.14573/altex.2304161 4 also covers the organization's connections with relevant stakeholders (governmental, non-governmental and in the private sector). Management and Stakeholder is the basis on which the principles of the 3Rs and the transition to animal free innovation can be further developed and disseminated throughout the organization. Domain A has eight indicators divided into five themes: governance, strategy, animal welfare and ethical review body, process management (public reporting, performance management and incentives for internal organization), and stakeholder engagement.

Domain B -Research and development
This domain assesses the research departments of the organization including the Biological Service Unit (BSU) i.e., animal facility. Research and Development (R&D) is the process by which new knowledge, new methods, new products, or new procedures are investigated and developed. Within Public research organizations, R&D is most often centered around specific research fields or profiles, such as cancer, toxicology, immunity, and infection. Research projects are defined and conducted within one or more of these research fields. This domain analyses how R&D is organized and stimulated within the organization and to which extent the 3Rs, new approach methodologies (NAMs) and the transition to animal free innovations are embedded. NAMs are defined as any technology, methodology, approach, or combination that can provide information on biological pathways, including in silico, in chemico, in vitro, and ex vivo approaches (ECHA, 2016). For our purpose we extended the definition of NAM to also include the use of ex vivo animal tissue and other species that are considered less sentient and to cover biomedical research as well. Furthermore, it considers the presence of quality standards, the levels of collaboration within and beyond the organization and the sharing of research data (open science). Domain B has seven indicators divided into three themes (quality standards, new approach methods, and data sharing (open science)).

2.3
Domain C -Education and training Education and training are core tasks of an academic (medical) center. New generations of researchers should be trained with the latest innovations in research and stimulated to develop a broad and innovative mindset. This domain not only assesses the education of students but also the ability of the organization to act at other levels of 'Capacity Building'. Capacity Building is the process of developing and strengthening the skills, instincts, abilities, processes, and resources that organizations and communities need to survive, adapt, and thrive in a fast-changing world. This domain assesses capacity development in human resources, including acquiring, retaining, and improving knowledge and skills of the organization's staff. Domain C has eight indicators divided into five themes (educational program to initial qualifications, continuing professional development (CPD), training need assessment, awareness, and accessibility to training outside of the organization). 'Awareness' covers the awareness of an organization's staff and personnel of new developments in animal testing and the transition to animal-free innovations and of responsible research conduct.

Domain D -Investments and grants
This domain assesses the actions and policies of the organization on financing innovation in research projects related to NAMs and their implementation from core funds and through grants obtained from funders. In the context of the BATI Domain D has three indicators divided across two themes (investments and grants).

2.5
Domain E -Intellectual property This domain assesses the organization's policy on intellectual property (IP). IP can be valuable for both the organization and society. If IP is protected by a patent for a fixed period, it ensures that the invention is disclosed to the wider community at the same time it can provide an opportunity for recouperation of research and development costs and ensures that an individual company/ private organization cannot take advantage and make potentially large profits from research funded through public money. The monetary or scientific value of IP largely depends on how the organization manages and executes its IP rights. Through a license, a patented invention can be used by other parties. Depending on the license, parties are allowed to participate in further research, development or commercializing the product. The BATI assesses the number of patents owned by the institute relevant to the focus areas of the Index, and evaluates whether it contributes and stimulates, or inhibits knowledge exchange. Furthermore, the Index assesses how the institute organizes their licensing practices, including the selection of partners. Domain E has three indicators divided across three themes (patents, licenses, and partnerships).

BATI pilot study
3.1 Participating organizations Five institutes were recruited for the beta testing project: the Leiden University Medical Center (LUMC) in Leiden, The Netherlands); University Medical Center Utrecht (UMCU) and Utrecht University (UU) both in Utrecht, The Netherlands; University Medical Center Groningen (UMCG) and University of Groningen (RUG) both in Groningen, The Netherlands. In Utrecht and Groningen, the university and the medical center are tightly linked and share organizational and infrastructural structures. Therefore, the Utrecht centers were considered as if they were one entity as were the two centers in Groningen. Consequently, this report contains the benchmarking results for three organizational entities -Groningen, Leiden, and Utrecht. Wherever organization is used in the rest of the report it refers to one of these three. Within each organization, the animal welfare body, or the director of the Biological Services Unit (BSU which is or includes the animal facility) served as the first point of contact.

Data collection and analysis
The aim was to collect per organization all relevant information and policies on the 3Rs, new approach methodologies (funding, development, and implementation) and animal experimentation from existing sources and legal accountability reports. This approach was taken to not increase the administrative burden of the respective organizations. Information was collected from ALTEX, accepted manuscript published August 8, 2023 doi:10.14573/altex.2304161 5 documentation available in the public domain, like annual reports, policy statements and institutional websites complemented with internal documents, and targeted interviews.
The collected documentation was categorized by indicator. Datasets were analyzed and scored independently by two members of the BATI research team using the scoring matrix. The assessments were compared. In the exceptional cases where differences were observed, the assessments were discussed in the wider team until agreement was reached. Most discussions focused on the categorization of information per indicator. Initial results per organization were shared with the respective organizations to give them the opportunity to provide additional information that was missed and to comment on the interpretations of the results. The feedback and additional information provided were evaluated by the BATI research team and a decision was made whether to include newly provided information. For example, if an organization added information about a project to improve communication through its external website and the BATI team concluded that this project was beyond the planning phase and resourced with evidence of deliverables, this information was included. Subsequently, the analysis was repeated with the new information included. The data were analyzed per organization. For each organization the total absolute score per indicator was calculated and grouped into themes, domains, or pillars. The total absolute scores were converted to relative scores. The rankings are based on these relative scores. The data and analysis of the data are in Supplementary material 3 15 . The BATI results were anonymized for publication, using the code names Alpha, Bravo and Charlie (not necessarily in alphabetic order of the names of the organizations).
The interviews were conducted in a standardized manner using a set of predetermined questions. The organizations nominated interviewees. These initial interviews prompted follow-up interviews with other members in the same or other departments/ units of the organization ('snowballing'). For example, if the nominated interviewee was a member of the legal department and referred during their interview to documents, policies, or persons in other departments, that reference would trigger an interview request to that department or more specifically to a particular staff member in that department. The information obtained through interviews was added to the documentation as a 'narrative'. The narratives are short quotes or observations taken from the interviews. These were used to further interpret the findings from existing sources. For this version of the BATI, the narratives were not included in the calculations of the benchmark but served to confirm the presence and active use of relevant documentation.
The same is true for the results of the questionnaire, which was circulated among the employees (mainly animal technicians) of the participating organizations. The questionnaire served to assess the opinion about the organization's research policies and actions regarding the 3Rs, knowledge of the program Transition to Animal Free Innovations, and to what extent there is a safe working atmosphere supportive of and stimulating new ideas and developments, and room for improvement at the working place. The questionnaire is in Supplementary material 4 16 . The responses, 108 in total evenly distributed across organizations, were anonymized per organization but remained traceable to the respective organizations.

Ranking by domain
The overall ranking of the three organizations split into domains is presented in Figure 3 (and Supplementary material 3 15 ). The organizations differ from each other regarding the total ranking and at the individual domain level. The ranking orders at the domain level are not necessarily the same as the overall ranking. For example, the overall ranking ranked Charlie first, then Bravo and Alpha third, where Alpha ranked second and Bravo third for Capacity building (Domain C). This illustrates that organizations can excel in different areas. On the x-axis the relative score of the organizations on a scale of 0 to 100 with each shade corresponding to one of the three pillars, and on the y-axis the three anonymized organizations with at the top the overall achievable scores per pillar, corresponding with the number of indicators that make up each pillar.

4.2
Ranking by pillar The overall ranking of the three organizations split into pillars is presented in Figure 4 (and Supplementary material 3 15 ). The overall ranking of the three organizations shows that Alpha and Bravo score equally high for their commitment and at the level of implementation. The difference in overall score between Alpha and Bravo is explained by the difference in transparency. Charlie reached almost maximal scores for both commitment and transparency. The difference in scores for commitment and transparency is explained by the quantity and content of relevant documentation in the public domain, as well as shared internal documents and information.

4.3
Ranking by theme per domain The ranking by themes within domains is presented in supplementary material 5 17 .

Discussion and conclusion
The BATI was designed after the Access to Medicine Index with the aim to provide insight into the efforts and contributions research institutions make in the transition to animal free innovation and to reward them for their effort. The BATI provides organizations incentives to learn from and inspire each other regarding the implementation of research practices without the use of animals for the benefit of science. The prototype of the BATI was field tested with five institutes in the Netherlands of which two sets of two had partly intertwined or shared management and operations. It was therefore decided to benchmark the two institutes in Groningen and the two institutes in Utrecht together explaining the results presented for three entities: the collective of Groningen institutes, the collective of Utrecht institutes and the LUMC in Leiden. The BATI resulted in an overall ranking, a ranking by domain, pillar and by themes within domains. The ranking orders at the domain level were not necessarily the same as the overall ranking, illustrating the power of the BATI providing organizations with information on areas where they are ahead and others where improvements are possible. The BATI resulted in a ranking based on existing documentation present in the public domain and within the organization verified and incidentally complemented with information obtained through targeted interviews and results from a questionnaire among BSU staff. Stakeholders and participating organizations were very pleased with the fact that the BATI did not demand additional reports or datasets. The participating organizations consented to a closed meeting during which the benchmarking results were presented, not anonymized. The representatives largely agreed with the results and conclusions presented. They used this platform to learn from each other and engage in discussions, and they decided on future collaborations incentivizing 3R initiatives and initiatives driving the transition to animal free innovations. The BATI team conclude that publicly available information supplemented with targeted interviews provide sufficient information to perform the benchmark and that this open and transparent approach does not lead to a 'blaming and shaming' culture.
With feedback from the participating organizations the BATI team prepared the BATI version 1.0, which will serve as the basis for the next edition of the benchmark. The next edition will benchmark the same organizations plus other academic (medical) centers in the Netherlands where animals are used for scientific purposes. It will be staged over a period of two years. A typical life cycle of the benchmark resembles that of the Access to Medicine Index. In the first year, the BATI analytical framework for that edition will be constructed and decided upon by the stakeholders (including the participating organizations). Consideration should be given to the inclusion of criteria that were not included in this first edition such as those addressing other aspects contributing to the transition to animal free innovations, like actions to facilitate the use of human derived materials. 18 Also, the equal weight of indicators should be revisited as it resulted in a bias towards the domains and pillars with more indicators. Towards the end of the first year, data collection will commence. Data collection will continue into the second year after which data analysis will follow and a benchmark will be published. The stakeholder consultation and engagement are important for the acceptance and usefulness of the BATI. The more stakeholders are engaged the higher the willingness to