Essential Components of Methods Papers

Methods papers are important for the progress of biomedical research, as they provide the essential tools to explore new questions and help to better answer old ones. However, it is often not clear how a methods paper differs from a methods protocol. Confusion between these two very different types of publication is widespread. The resultant misunderstanding contributes to a relatively poor reputation of methods research in biology despite the fact that many Nobel prizes have been awarded specifically for method development. Here, the key components of a methods paper are summarized: (i) methods description, (ii) performance standards, (iii) applicability domains, (iv) evidence for advances compared to the state-of-the-art, (v) exemplification of the method by practical application. In addition, information domains are discussed that are desirable but may be provided on a case-by-case basis or over the course of a series of papers: (vi) method robustness, (vii) accuracy and (viii) precision measures, including various quantifications of method performance, and (ix) measures of uncertainty, including a sensitivity analysis. Finally, elements of the overall framing of the method description are highlighted. These include the scientific, technical and, e.g., toxicological rationale for the method, and also the prediction model, i.e., the procedure used to transform primary data into new information. Received July 3, 2018; © The Authors, 2018. ALTEX 35(3), 429-432. doi:10.14573/altex.1807031 Correspondence: Marcel Leist, PhD In vitro Toxicology and Biomedicine, Dept inaugurated by the Doerenkamp-Zbinden foundation, University of Konstanz, Universitaetsstr. 10, 78464 Konstanz, Germany (marcel.leist@uni-konstanz.de) This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International license (http://creativecommons.org/ licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is appropriately cited.

are many dedicated collections of protocols (e.g., Current Protocols).The separation is relatively clear, but not mutually exclusive: There are also journals that contain method descriptions and protocols (e.g., Methods in Enzymology).
Which of the two types of publication corresponds to a cookbook, i.e., the detailed instruction of all steps required to make a method work?It is the protocol, sometimes also referred to as the "standard operating procedure (SOP)."In contrast to a protocol, methods papers do not usually contain a detailed procedure.They refer to general principles and provide a rough layout of what needs to be done to make a method work.This general layout may be used by an expert in the field (in patent literature this would be termed "a person skilled in the art") to

Does a methods paper allow for immediate implementation of the described method?
There is some confusion about the definition of a "methods paper."The purpose of this BenchMarks essay is not to add further ballast to this issue by providing another definition.We rather recommend to obtain some inspiration from the many journals and books devoted to this issue.Particularly instructive is the example of Nature Protocols vs Nature Methods.From this case, we see that there seems to be a fundamental difference between a protocol and a method description.This dichotomy also is seen by many journals focusing on methods (e.g., ALTEX, Journal of Neuroscience Methods, Journal of Immunological Methods, etc.) but not publishing protocols.Vice versa, there In Shakespeare's Hamlet, a lot of value is given to a methods approach (Though this be madness, yet there is method in 't).In modern times, methods papers often have been the basis of major progress in science.Examples can be found in the speeches delivered upon the award of Nobel prizes related to the polymerase chain reaction (PCR) method (K.Mullis), the radioimmunoassay (R. Yallow), patch clamping technology (B.Sackmann, E. Neher), the generation of induced pluripotent stem cells (S.Yamanaka), in vitro fertilization (G.Edwards), knock-out mice (M.Capecchi, O. Smithies, M. Evans), computer tomography (A.Cormack, G. Hounsfield), (cryo)-electron microscopy (J.Dubochet, J. Frank, R. Henderson, E. Ruska), green fluorescent protein (O.Shimomura, M. Chalfie, R. Tsien), super-resolution microscopy (S.Hell) and many more.The protocols for these methods have been refined, modified, optimized, sometimes in hundreds of variants, but the overall principle has remained the same since their discovery.These examples give some orientation on why we need both methods and protocols, and why they represent two different categories.

The practical way from method to protocol
It would appear practical if all methods papers were accompanied directly by the original SOP.However, this coupling is not as straightforward as it appears.First, SOPs can become outdated quickly because certain materials are no longer available or major changes in technology occur.Second, a given SOP may depend on proprietary technology or unique skills, and this would require a new SOP for adaptation in another laboratory.Third, and most important, information that can be transferred in written form is limited.From a certain complexity level on, hands-on training is the best way to communicate the application of a new method, and it is important to realize that this can only partially be substituted by published SOPs.Last, but not least, there is a limit of parameters that can be defined when an SOP is compiled.No matter how meticulously this is implemented, there will always be details that have not been or could not be defined.In summary, there are limits to the exact definition of a method by an SOP, at least when biological material is involved.

Specification of outcomes vs definition of procedures
How can the dilemma of having to define endless series of details that need to be considered, and then still having to add further sets to patch gaps in a protocol be avoided?The problem of this "specific definition approach" is well known from the field of doping and illicit drugs, where new legal gaps arise no matter how comprehensive one tries to be.The alternative that has been chosen in the field of narcotics is to ban all substances that cause addiction, craving, withdrawal symptoms, or that result in performance enhancement above a certain threshold.
In the domain of methods, this is equivalent to defining "performance standards."Translated to cooking, this would mean to define for poached eggs that some of the yolk should still be liquid, while the egg white should be solid.Such a performance standard is a definition of what is considered a positive result work out an SOP, i.e., to define exact materials, chemicals, individual manual or robotic procedures to be executed, and routines on how to treat and combine the output data.An example of a typical method description from the field of cooking would be "Prepare two poached eggs for each plate and cover them with two tablespoons of Béchamel sauce; then garnish with chives."Note that the typical methods description would not give certain details (e.g., salt added to poaching water, number of chives used) and, thus, dishes would look and taste different if produced by different cooks.The corresponding protocol would detail all the manual steps required to produce poached eggs, the volume and temperature of the water used, the details on cutting chives to certain fragment sizes, and so on.
This leads to the essential question whether a methods paper allows the implementation of the described method within a relatively short period of time.The answer to this question is complex because it may depend on the intended use of the method.In an ideal world, a methods paper should provide sufficient information so that a specialist can immediately apply the method.However, in practice, methods will usually not work immediately and this is also not expected; rather some optimization and adaptation steps and training, e.g., of complex manual procedures, may be necessary.Thus, the implementation of a functioning SOP may take from days up to several months.By and large, the question whether short term implementation is possible could be answered with "YES" on the basis of the situation described above.However, difficulties arise when the supposedly small adaptations and optimizations play a critical role in the method outcome.This is of particular importance in regulated fields such as toxicology.However, it also applies to biomedical research in general.Research results from one given assay may be largely divergent depending on, e.g., the mouse strain, age, sex, diet; the type of cell culture dishes; the purity of chemicals, distilled water, cell populations used; or the exact room temperature during the experiment.In such cases, the answer would be "NO" (the method cannot be directly applied to yield reproducible results across laboratories): An exact SOP would be required to make sure that testing occurs under very similar conditions by different operators, in different locations and over time.

Why we need methods papers
What is the benefit of a methods paper if it does not necessarily allow for the immediate use of the described method?Why not have mainly protocol collections, just as the many cookbooks, cooking magazines and cooking shows?For one, it is important to note that many methods papers do allow application of the described method given that sufficient resources, time and experience are available.However, there is a second, fundamental issue: new methods are part of the progress of science.They can be discoveries in themselves with far-reaching implications, and thus, they need to be published, evaluated and peer-reviewed like other original research papers, while protocols rather require a kind of auditing to avoid mistakes and to assure sufficient information density and adherence to technical rules and conventions.obtained by the correct use of the method.It can substitute a complex set of technical instructions (e.g., corrections for local air pressure, chicken diet and egg size for the production of poached eggs).For many methods in biology, such performance standards would be based on the results of known positive and negative controls for which the expected (and required) result is pre-specified.It would then be assumed that the method is working correctly (despite any modifications and adaptations) as long as the performance standards are fulfilled.

What are the key components of a methods paper?
After these introductory considerations, it should be clear that a methods paper and a laboratory protocol fulfil different purposes.From here, the next step is to summarize the key components of a methods paper, and to differentiate them from helpful but not obligatory additions.Two features discussed above are (i) a reasonable description of procedures and materials to an extent that will allow a specialist to assess and to reproduce the method; (ii) the definition of performance standards, at least for one exemplary application of the method.Notably, this implies that performance standards are never absolute but are made fit-forpurpose, i.e., for a specific use of the method.They need to be changed or entirely re-defined for other uses.Vice versa, each method, as defined by its performance standards, has a certain applicability domain, e.g., for certain chemicals or within a certain range of input or output parameters.It may not work reliably outside this domain.Thus, (iii) information on likely (or proven) applicability domains is the third key component of a methods description (Fig. 1).

The sine qua non of methods papers
Many manuscripts on methods comply with the above considerations, at least to a major extent.Weaknesses on performance standards (see (ii)) and absence of information on applicability domains (see (iii)) can be argued to be acceptable in some situations.However, two additional aspects are essential (Fig. 1).Many other journal editors also feel that the two following elements are non-negotiable: (iv) A methods paper must demonstrate how the new method goes beyond the state-of-the-art.This means that data need to be provided on how a problem is solved better, faster or with less uncertainty than with previously available methods.Ideally, there would be a quantification (or a rough estimate) on how much the new method differs from the previous gold standard with respect to its performance.As a methods paper is part of the progress of science, it needs to demonstrate which new (or better) approach it enables.In the category of standard research papers, this criterion would be called "extent of novelty."For methods papers, it is often not called "novelty", but nevertheless, the best methods papers make it possible to explore largely unknown terrain, while mediocre methods papers do not significantly change the toolset available to explore new questions or to drill deeper in a particular direction.
(v) A methods paper must also contain at least one use case.This issue is often confused with the aforementioned aspect of novelty.There can indeed be some overlap, as the case can also Fig. 1: Components of a methods paper Three groups of components contribute to a good methods publication.The framing of the method involves an introduction to the problem and outlines the challenge addressed.Moreover, it helps the reader understand the new method by outlining its scientific and technical background/rationale.The frame will also address the issue of data display and interpretation, if possible by presenting a prediction model.One of the three essential core components is the method description itself, including information on the applicability domain and a definition of performance standards.The second component shows how the method goes beyond the state of the art by defining the scientific advance made possible by the method.The third component exemplifies the use of the method, and thereby provides information on scope and impact of the method for solving actual problems.Ideally, several of the add-on components will also be found, although this is dependent on the described method and on the space available in an initial publication.If not provided with the original method description, these components are important topics of follow-up publications to complete information on method performance. of success.In many of these instances, it appears that a small but critical experimental detail differed.Such problems can be avoided to a large extent if an analysis is performed on the method to find out how strongly the end result is affected by intentional (e.g., incubation time) or unintentional (e.g., fluctuation of room temperature) variations.Such a "sensitivity analysis" should ideally be performed for all major experimental parameters and variations.The outcome of such experiments is information on how strongly the method depends on a given parameter.For some parameters, large variations will not have a major influence on outcome.This information will support planning and implementation of experiments.Small fluctuations of other parameters will strongly influence the results, e.g., temperature, age or passage of cells, purities and concentrations of ingredients.This should be considered during experimental design and will influence the reproducibility of data sets.

Framing
Besides all the above points, the framing of a methods paper can affect both its overall quality and its acceptance by the target audience.This "frame" partially refers to the paper introduction: Here it should be clearly outlined why there is a need for a new method.Ideally, a technical/scientific background will be provided.Information should also be given on the overall challenge that is addressed by the method, and how this challenge has been addressed by other methods.
Another part of the frame deals with issues such as the interpretation of data obtained with the method.This could address the data structure, the way data are displayed, and the model according to which the data should be interpreted (prediction model).

Outlook
These considerations are mainly oriented towards biomedical research -often with toxicological applications and the establishment of new approach methods (NAM) of animal-free experimentation in mind.However, they can be generalized to apply not only to classical cell-based in vitro methods, but also to (bio)chemical approaches and computational methods.Also for these, sensitivity analyses can be performed, performance standards set, robustness and uncertainty factors determined, and example applications shown.Sometimes the vocabulary is different in other fields, but principles should be easily transferable.
show the superiority of the method compared to the state-ofthe-art.This overlap is, however, not necessary.The above point shows the generic performance of a new tool.For instance, a ceramic knife could be demonstrated to have advantages compared to steel knives, as it does not corrode and it does not dent at the edge.This can also be nicely measured and quantified.It does, however, not prove a practical applicability.In most real-life situations, rust is not important for kitchen knives, and a knife that dents (metal) may be preferable in many cases to one that splinters (ceramic).A real-world use case should be demonstrated to give an idea of the usefulness of the new tool or method.This should not be defined by a possibly irrelevant parameter (e.g., the new method can measure human body weight with microgram uncertainty instead of gram uncertainty), but by a performance criterion derived from a relevant problem.This latter point is related to the criteria of scope and impact for standard research papers.It is important to demonstrate that the method is likely to have a broad scope, rather than addressing a minor, possibly artificial, niche problem.This is independent of the novelty of the method.

Nice to know
The world of methods papers is very colorful and heterogeneous, so that no rigid rule applies to all of them.However, in addition to the main points (i-v above), some additional aspects may be considered (Fig. 1).These may be added in various combinations, and they all tend to increase the weight and usefulness of a methods paper: (vi) Robustness: Any information on how reproducibly a method works is useful.This also implies the identification of critical steps.Notably, robustness not only applies to the stability of experimental outcomes within one experiment.It also includes the question of how much the assay results change over time.The presentation of historic controls, i.e., data from reference compounds obtained over longer time periods, can help to clarify this point.
(vii) Accuracy: This involves information on false negatives and false positives, i.e., sensitivity and specificity of the method.
(viii) For analytical methods, a set of classical analytical parameters is useful to know: limit of detection, limit of quantification, linear range, baseline variation (noise), signal-noise ratio, stability over time, transferability within one laboratory, transferability and reproducibility between different laboratories, etc. Measurement and presentation of all these data can easily go beyond the scope of one single publication.Subsets of this information may be chosen or qualitative estimates may be given initially.In such cases, follow-up papers may then present more quantitative, in depth data sets.
(ix) Quantification of uncertainty: Very frequently, two labs use the same method with different outcomes or different rates