Table of Contents

Title Page

Copyright
Contributors
Chapter 1: Introduction
1. References

Part I: Contextual Backgrounds

Chapter 1: Reproducibility, Objectivity, Invariance
1. 1.1 Introduction
2. 1.2 Reproducibility in the Empirical Sciences
3. 1.3 Objectivity
4. 1.4 Invariance and Symmetry
5. 1.5 Summary
6. References
Chapter 2: Reproducibility between Production and Prognosis
1. 2.1 Preliminary Remarks: Three Myths
2. 2.2 How Does Reproducibility Connect with Production?
3. 2.3 How Does Production Connect with Continuity?
4. 2.4 How Does Continuity Connect with Scientific Rationality?
5. 2.5 How Does Scientific Rationality Connect with Prognosis?
6. 2.6 How Do Prediction and Prognosis Connect with Reproducibility?
7. 2.7 Concluding Remarks
8. References
Chapter 3: Stability and Replication of Experimental Results: A Historical Perspective
1. 3.1 Experiments and Their Reproduction in the Development of Science
2. 3.2 Repetition of Experiments
3. 3.3 The Power of Replicability
4. 3.4 Cases of Failed Replication
5. 3.5 Doing Science without Replication and Replicability
6. 3.6 What Can We Learn from History?
7. Acknowledgments
8. References
Chapter 4: Reproducibility of Experiments: Experimenters' Regress, Statistical Uncertainty Principle, and the Replication Imperative
1. 4.1 Introduction
2. 4.2 The Experimenter's Regress
3. 4.3 The Statistical Uncertainty Principle
4. 4.4 The Replication Imperative
5. References

Part II: Statistical Issues

Chapter 5: Statistical Issues in Reproducibility
1. 5.1 Introduction
2. 5.2 A Random Sample
3. 5.3 Structures of Variation
4. 5.4 Regression Models
5. 5.5 Model Development and Selection Bias
6. 5.6 Big and High-Dimensional Data
7. 5.7 Bayesian Statistics
8. 5.8 Conclusions
9. Acknowledgments
10. References
Chapter 6: Model Selection, Data Distributions, and Reproducibility
1. 6.1 Introduction
2. 6.2 Bayesian Model Selection and Relation to Minimum Description Length
3. 6.3 Extending BMS (and NML#): BMS*
4. 6.4 Replication Variance and Reproducibility
5. 6.5 Final Remark
6. References
Chapter 7: Reproducibility from the Perspective of Meta-Analysis
1. 7.1 Introduction
2. 7.2 Basics of Meta-Analysis
3. 7.3 Meta-Analysis of Mind-Matter Experiments: A Case Study
4. 7.4 Summary
5. References
Chapter 8: Why Are There So Many Clustering Algorithms, and How Valid Are Their Results?
1. 8.1 Introduction
2. 8.2 Supervised and Unsupervised Learning
3. 8.3 Cluster Validity as Easiness in Classification
4. 8.4 Applying Clustering-Quality Measures to Data
5. 8.5 Other Clustering Models
6. 8.6 Summary
7. References

Part III: Physical Sciences

Chapter 9: Facilitating Reproducibility in Scientific Computing: Principles and Practice
1. 9.1 Introduction
2. 9.2 A Culture of Reproducibility
3. 9.3 Statistical Overfitting
4. 9.4 Performance Reporting in High-Performance Computing
5. 9.5 Numerical Reproducibility
6. 9.6 High-Precision Arithmetic in Experimental Mathematics and Mathematical Physics
7. 9.7 Reproducibility in Symbolic Computing
8. 9.8 Why Should We Trust the Results of Computation?
9. 9.9 Conclusions
10. References
Chapter 10: Methodological Issues in the Study of Complex Systems
1. 10.1 Introduction
2. 10.2 Definitions of Complexity
3. 10.3 Complexity and Meaning
4. 10.4 Beyond Stationarity and Ergodicity
5. 10.5 Conclusions
6. Acknowledgments
7. References
Chapter 11: Rare and Extreme Events
1. 11.1 Introduction
2. 11.2 Statistics of Extremes
3. 11.3 Predictions of Extreme Events
4. 11.4 Evolving Systems Exposed to Extreme Events
5. 11.5 Conclusions
6. Acknowledgments
7. References
Chapter 12: Science under Societal Scrutiny: Reproducibility in Climate Science
1. 12.1 Reproducibility Challenges for Climate Science
2. 12.2 Reproducibility in Observational Climate Science
3. 12.3 Reproducibility in Climate Modeling
4. 12.4 Reproducibility in Paleoclimatology
5. 12.5 Conclusions and Recommendations
6. References

Part IV: Life Sciences

Chapter 13: From Mice to Men: Translation from Bench to Bedside
1. 13.1 The Drug Development Process
2. 13.2 Contributions of Animals to Medical Progress
3. 13.3 Translation Challenges in Different Fields of Research
4. 13.4 Increasing Translational Success: Summary and Conclusions
5. References
Chapter 14: A Continuum of Reproducible Research in Drug Development
1. 14.1 Introduction
2. 14.2 The Strategy of the Magic Bullet
3. 14.3 Specialists and Generalists
4. 14.4 From Single-Target to Multi-Target Drugs
5. 14.5 Conclusions
6. References
Chapter 15: Randomness as a Building Block for Reproducibility in Local Cortical Networks
1. 15.1 Introduction
2. 15.2 Spike Trains and Reproducibility
3. 15.3 Spike Trains
4. 15.4 Neuronal Populations
5. 15.5 Summary
6. References
Chapter 16: Neural Reuse and In-Principle Limitations on Reproducibility in Cognitive Neuroscience
1. 16.1 Introduction
2. 16.2 The Erosion of Modular Thinking
3. 16.3 Intrinsic Limits on Reproducibility
4. 16.4 Going Forward
5. References
Chapter 17: On the Difference between Persons and Things – Reproducibility in Social Contexts
1. 17.1 The Problem of Other Minds and Its Evolutionary Dimension
2. 17.2 Understanding the Inner Experience of Others
3. 17.3 Identifying the Neural Mechanisms of Understanding Others
4. 17.4 Abduction of the Functional Roles of Neural Networks
5. 17.5 Psychopathology of the Inner Experience of Others
6. 17.6 Conclusions
7. References

Part V: Social Sciences

Chapter 18: Order Effects in Sequential Judgments and Decisions
1. 18.1 Introduction
2. 18.2 Question Order Effects and QQ Equality
3. 18.3 No Order Effect Model and Saturated Model
4. 18.4 The Anchor Adjustment Model
5. 18.5 The Repeat Choice Model
6. 18.6 The Quantum Model
7. 18.7 Concluding Comments
8. References
Chapter 19: Reproducibility in the Social Sciences
1. 19.1 Introduction
2. 19.2 Reproducibility as a Current Problem in the Social Sciences
3. 19.3 “Reproductions Have No Meaningful Scientific Value”
4. 19.4 Reaction from the Blogosphere
5. 19.5 Conclusion
6. References
Chapter 20: Accurate But Not Reproducible? The Possible Worlds of Public Opinion Research
1. 20.1 Introduction
2. 20.2 Reproducibility: A Missing Criterion in Public Opinion Research?
3. 20.3 Big Data versus Science: The Breakthrough of Modern Polling
4. 20.4 The Birth of a Statistical Myth
5. 20.5 Generating Trust ¹⁰
6. 20.6 The Possible Worlds of Public Opinion Research
7. 20.7 Looping Effects between Measurement and Measured
8. 20.8 Swarms of Possible Worlds
9. References
Chapter 21: Depending on Numbers
1. 21.1 Introduction
2. 21.2 Statistical Error
3. 21.3 Translation
4. 21.4 Statistical and Substantive Significance
5. 21.5 Irreproducible Numbers
6. 21.6 Reproducing Calculations
7. References
Chapter 22: Science Between Trust and Control: Non-Reproducibility in Scholarly Publishing
1. 22.1 Introduction
2. 22.2 Reproducibility as the Touchstone for Distinguishing Science from Non-Science
3. 22.3 Contested Claims: The Story behind STAP
4. 22.4 The Structural Gap between the Production and Representation of Scientific Facts
5. 22.5 The Increasing Awareness of Reproducibility Problems
6. 22.6 The New Transparency: Bridging the Gap in Scholarly Publishing
7. 22.7 Conclusions
8. References

Part VI: Wider Perspectives

Chapter 23: Repetition with a Difference: Reproducibility in Literature Studies
1. 23.1 Introduction
2. 23.3 Language and Difference
3. 23.3 Mimesis, Imitatio, and Parody
4. 23.4 Literary Translation: Domesticating versus Foreignizing
5. 23.5 Reproducing Cultural Significance
6. 23.6 Conclusions
7. Acknowledgments
8. References
Chapter 24: Repetition Impossible: Co-Affection by Mimesis and Self-Mimesis
1. 24.1 Introduction
2. 24.2 Repetition within the Philosophy of Time
3. 24.3 Re-Presenting Forgetting
4. 24.4 Repetition, Co-Affection and Trauma: Identity and Coping with the Past
5. 24.5 The Dialectics of Remembering and Forgetting
6. 24.6 Co-Affection and Memorizing Recall
7. 24.7 Mimesis
8. References
Chapter 25: Relevance Criteria for Reproducibility: The Contextual Emergence of Granularity
1. 25.1 Introduction
2. 25.2 Contrast Classes, Coarse Grains, Partition Cells
3. 25.3 Two Examples
4. 25.4 Contextual Emergence
5. 25.5 Ontological Relativity: Beyond Fundamentalism and Relativism
6. Acknowledgments
7. References
Chapter 26: The Quest for Reproducibility Viewed in the Context of Innovation Societies
1. 26.1 Introduction
2. 26.2 A Genealogical Sketch of “Innovation”
3. 26.3 Reframing Scientific Ethos: Sound Science
4. 26.4 Reproducibility and Innovation: A Regulative Dual
5. 26.5 Making the Implicit Explicit I: Social Robustness of Science
6. 26.6 Making the Implicit Explicit II: Responsible Research and Innovation
7. 26.7 Mertonian Norms Challenged Anew: Institutional Reflexivity and Responsiveness
8. References
Index

End User License Agreement

List of Illustrations

Chapter 5: Statistical Issues in Reproducibility

Figure 5.1 Measurements of the velocity of light by Newcomb in 1882. Three confidence intervals are shown on the right.
Figure 5.2 Histogram (left) and empirical cumulative distribution function (right) for the Newcomb data. The dashed and dotted lines represent normal distributions obtained from fits to the data without and with two outliers, respectively.
Figure 5.3 Student's measurements of the concentration of nitrogen in aspartic acid. The bars represent confidence intervals based on the $c05-math-108$ -test for the first 32, 64, 96, and 128 of 131 observations.

Chapter 6: Model Selection, Data Distributions, and Reproducibility

Figure 6.1a Case A – the data distributions are the binomial distributions for the number of successes in 10 trials predicted by model instances with $c06-math-224$ in the range from 0.0 to 1.0 in steps of 0.1.
Figure 6.1b Case A – the toy example imposes a geometric prior on the data distributions. Due to their one-to-one correspondence with the priors on data distributions, the model instances have the same priors.
Figure 6.2 Case B – 11 additional “partner” distributions are each a mixture of one of the binomials with a uniform distribution, and are shown to the right of the binomial partner.
Figure 6.2b Case B – prior probabilities for the 22 distributions: the binomial distribution is assumed to be three times less likely than its partner.
Figure 6.2c Case B – the prior of each model instance is just the sum of its two best matching partner distributions.
Figure 6.3 Case A – posterior probabilities for the data distributions following an outcome of three successes in 10 trials. For Case A, these are also the posterior probabilities of the corresponding model instances.
Figure 6.3 Case B – posterior probabilities for the 22 data distributions, following an outcome of three successes in 10 trials.
Figure 6.3 Case B – posterior probabilities for the model instances are the sum of the posterior probabilities for the two partner distributions.
Figure 6.4 Predicted distribution for the mean number of successes expected in ten trials, based on posterior probabilities following an outcome of three observed successes in 10 trials. Solid lines connecting dots are for Case A, and dashed lines connecting triangles are for Case B.

Chapter 7: Reproducibility from the Perspective of Meta-Analysis

Figure 7.1 Forest plot of the beta-blocker data from Carlin (1992). See text for explanation.
Figure 7.2 Forest plot of the magnesium data from Sterne and Egger (2001). See text for explanation.
Figure 7.3 Funnel plots of standard deviations of the estimates (SE) versus log odds ratio for the beta blocker (a) and the magnesium data (b) using fixed-effects estimates. See text for explanation.
Figure 7.4 Funnel plots of standard deviations of the estimates (SE) versus log odds ratio for the beta blocker (a) and the magnesium data (b) using random-effects estimates. See text for explanation.
Figure 7.5 Histogram of $c07-math-156$ -scores under experimental conditions (data from Radin and Nelson (1989)). The bell curve represents the distribution expected under the null hypothesis of no effect.
Figure 7.6 Normal probability plots of the standardized residuals for the FCR, RCR1, RCR0, and RCRS model (from upper left to lower right).
Figure 7.7 (a) and (b) are histograms of 5,000 test $c07-math-310$ -values simulated under the null hypotheses of no selection and no effect, respectively. (c) and (d) are profiles of weights $c07-math-311$ estimated within the RCRS model, when there is selection (d), and when there is not (c). Squares: averages of estimated $c07-math-312$ -profiles from 5,000 simulated data. Circles: $c07-math-313$ -profiles underlying the simulations. Broken lines: 10 $c07-math-314$ -profiles randomly selected from the 5,000 simulated profiles (providing an impression of the variability to be expected).

Chapter 8: Why Are There So Many Clustering Algorithms, and How Valid Are Their Results?

Figure 8.1 Percentage of incorrectly classified instances versus perturbation size for datasets of Type I (left) and of Type VI (right) for $c08-math-377$ . With WEKA's simple $c08-math-378$ -means there is no preferred value for $c08-math-379$ (the number of clusters) as the results show no stability of the error rate with respect to perturbation size.
Figure 8.2 Analysis of one dataset of Type III on the left shows $c08-math-395$ as the largest value of $c08-math-396$ for which there is stability (in learned classifier accuracy) to perturbations when we consider the clustering results of WEKA's simple $c08-math-397$ -means as a supervised instance. On the right we have a similar (and typical) result of our analysis, but for dataset of Type IV. Again, $c08-math-398$ is the model-order identified for WEKA's simple $c08-math-399$ -means matching the number of components of the mixture generating the data.
Figure 8.3 Reproducing previous studies, we determine the model order of WEKA's simple $c08-math-410$ -means for the datasets of Type II (left) and Type V (right). The analysis with the CQM method shows correctly that the largest values of $c08-math-411$ where there is stability in supervised learning to perturbation are $c08-math-412$ and $c08-math-413$ , respectively. We illustrate the plot for one randomly chosen dataset out of the 50 of each data type.
Figure 8.4 Distribution with mixture of two components with proportions $c08-math-427$ as per Eq. (8.3). Left: $c08-math-428$ , middle: $c08-math-429$ , right: $c08-math-430$ .
Figure 8.5 Plots for the analysis of the dataset Leukemia with WEKA's simple $c08-math-452$ -means. The classifier is WEKA's NaiveBayes. Left: Our analysis shows the preferred values of $c08-math-453$ and $c08-math-454$ since the supervised learning problem shows the same error rate for $c08-math-455$ until $c08-math-456$ , while $c08-math-457$ does not start increasing until $c08-math-458$ . On the other hand, $c08-math-459$ suffers from as early as $c08-math-460$ . Right: Once $c08-math-461$ is determined, we analyze the different clustering obtained with different seeds.
Figure 8.6 For the dataset Leukemia and the subclass ALL, obtained with an earlier clustering, we see that simple $c08-math-475$ -means with $c08-math-476$ preferrably (and maybe with $c08-math-477$ ) yields sub-clusters.
Figure 8.7 Identification of the number of clusters (components) in the mixture learned by EM on the simple data of Estivill-Castro (2011, Figure 1). Left: Resilience to perturbation of NaiveBayes applied to a supervised-learning problem by forcing a crisp clustering of EM results selecting the most likely cluster. Middle: Resilience to perturbation of NaiveBayes applied to several supervised learning problems by choosing a class as per the distribution of EM results. Right: Resilience to perturbation of Voronoi partition classification applied to several supervised-learning problems by choosing a class as per the distribution of EM results.

Chapter 9: Facilitating Reproducibility in Scientific Computing: Principles and Practice

Figure 9.1 Performance development from 1994 to 2014 of the top 500 computers: medium curve = #1 system; lower curve = #500 system; upper curve = sum of #1 through #500. See top500.org/statistics/perfdevel/.
Figure 9.2 Minimum backtest length (in years) versus number of trials.
Figure 9.3 Final optimized strategy applied to the input dataset. Note that the Sharpe ratio is 1.32, indicating a fairly effective strategy on this dataset.
Figure 9.4 Final optimized strategy applied to the new dataset. Note that the Sharpe ratio is $c09-math-008$ , indicating a completely ineffective strategy on this dataset.
Figure 9.5 Performance plot of run time as a function of number of objects: parallel system (lower curve) vs. vector system (upper curve)]. Note that for all data points from the table except for the last entry, the vector system is faster than the parallel system.

Chapter 10: Methodological Issues in the Study of Complex Systems

Figure 10.1 Three patterns used to demonstrate the notion of complexity. Typically, the pattern in the middle is intuitively judged as most complex. The left pattern is a periodic sequence of black and white pixels, whereas the pattern on the right is constructed as a random sequence of black and white pixels. (Reproduced from Grassberger (1986) with permission.)
Figure 10.2 Two classes of complexity measures: Monotonic complexity measures essentially are measures of randomness, typically based on syntactic information. Convex complexity measures vanish for complete randomness and can be related to the concept of pragmatic information.

Chapter 12: Science under Societal Scrutiny: Reproducibility in Climate Science

Figure 12.1 Monthly average maximum temperatures during August at Potsdam (gray circles and connecting lines) from 1893 to 2013 exhibiting large interannual variations due to weather phenomena and short-term climatic variability. For investigations of longer-term trends, averages over extended periods of time are more meaningful, as indicated by the 30-year climatological averages at the beginning and at the end of the time series (horizontal black lines).
Figure 12.3 Climate model intercomparison of projections for future global temperatures under different emission scenarios for the fourth (left) and fifth (right) assessment report of the Intergovernmental Panel on Climate Change. (Reproduced from Knutti and Sedláček (2013) with permission by Nature Publishing Group.)
Figure 12.2 Global surface air temperature anomalies (relative to the average over the period 1971–2000) for the four different datasets and analysis methods described in the text.
Figure 12.4 Ensemble reconstructions of Northern-hemisphere surface air temperature anomalies (with respect to the reference period 1370–1420) over the last millennium (Frank et al. 2010). The shaded regions (from light to dark gray) indicate the ranges covered by 90%, 70%, and 50% of the ensemble reconstructions. The solid black line shows a climate model simulation (Feulner 2011) which compares very well with the reconstructed temperatures.

Chapter 13: From Mice to Men: Translation from Bench to Bedside

Figure 13.1 Study design features for 232 publications for the period from 1997 to 2011. Left panel: randomization; right panel: conflict of interest statement.
Figure 13.2 Incidence of experiments with outcome “tumor progression” versus quality score for 1,538 experiments described in 232 publications.

Chapter 15: Randomness as a Building Block for Reproducibility in Local Cortical Networks

Figure 15.1 (from Carandini 2004): Response of a single cell to an identical stimulus. (a) three different trials, and (b) average over seven trials.
Figure 15.2 Different types of spike trains. The $c015-math-034$ -axis shows time (arbitrary units) and each peak corresponds to a spike. (a) Poisson spiking (CV = 1), (b) regular spiking (CV = 0), and (c) random spike bursts (CV $c015-math-035$ ).
Figure 15.3 Inhibition with leakage for lower voltage bounds of $c015-math-077$ (solid), $c015-math-078$ (dashed), or $c015-math-079$ (dotted). Bold lines, left axis: CV of output spike train; thin lines, right axis: output rate for an input rate of 100Hz. On the $c015-math-080$ -axis, we plot the probability $c015-math-081$ that the next spike is excitatory. So $c015-math-082$ corresponds to excess inhibition, while $c015-math-083$ corresponds to excess excitation. The $c015-math-084$ -axis is the CV of the resulting output spike train. For excess inhibition the output CV is close to 1, regardless of the value of $c015-math-085$ . In contrast to the model without lower voltage bound, the rate is not fixed by insisting on a CV of 1.
Figure 15.4 (from Lengler et al. 2013): Response of a heterogeneous (dark) and a homogeneous (light) network with balanced excitation and inhibition for Poisson input of varying rates. (a) Reaction to input which is not perfectly Poisson. The plot shows that the heterogeneous network has strictly smaller CVs. Moreover, the homogeneous network shows strongly fluctuating responses to similar inputs. (b) Reaction to Poisson input that is disturbed by spontaneous bursts of input synchronization. The $c015-math-087$ -axis gives the fraction of input spikes that belong to bursts. The input rate is constant. For the heterogeneous network the output rate is robust and concentrated, while the homogeneous network shows much less predictable responses. The shaded areas show the standard deviations. (c) Behavior if the output of the population is fed in as input to an additional population; populations 1, 2, and 3 are shown (from solid to dotted). For the heterogeneous network the spike trains of different neurons are decorrelated as the signal propagates, while the homogeneous network increases cross-correlations. (d) Reaction of the network to a small number of neurons ( $c015-math-088$ -axis) each of which feeds one input spike into the network within a time interval of 10 ms (solid lines), 20 ms (dashed lines), or 30 ms (dotted lines). The $c015-math-089$ -axis shows the time of the first spike in the population. The curves start at the input size where all 100 trials produced at least one spike. The heterogeneous network can be activated by fewer input spikes and reacts faster.

Chapter 16: Neural Reuse and In-Principle Limitations on Reproducibility in Cognitive Neuroscience

Figure 16.1 Panel A offers a spatial map of motor cortex, showing which local regions exhibited neural activity during movements of the digit, wrist, etc. Panel B summarizes the total amount of the mapped area showing neural activity during the specified movements. Note both the similarities and differences in the spatial map during the two training phases and reacquisition. The same behavior can be supported by somewhat different neural arrangements. (Reprinted from Scheiber (2001) with permission license # 3577790361692.)
Figure 16.2 Top panel shows neural activation during a theory of mind task, and bottom panel during a change detection task. The left side of each panel displays the group average activation maps, and the right side the percentage of individuals who had activation at each location in the group average map. Note that for the stricter comparisons (social vs. random, and relational vs. match, which show differences in activity in the experimental task vs. a control task, rather than vs. baseline activity, and thus represent the regions of the brain specific to the experimental task) very little of the group average activation was displayed by more than 50% of individual participants. (Reprinted from Barch et al. 2013 with permission license # 3577790735856.)
Figure 16.3 Functional fingerprints for selected regions of the brain. Clockwise from top left: left intraparietal sulcus, left auditory cortex, right auditory cortex, and right anterior cingulate. The lines represents the relative amount of the overall activity (with confidene intervals) for each region that was observed in each of 20 task domains.

Chapter 18: Order Effects in Sequential Judgments and Decisions

Figure 18.1 Test of the no order effect model. Left panel: All 72 data sets with some specifically selected to have order effects. Right panel: 66 data sets not specifically selected to have order effects. See text for discussion.
Figure 18.2 Left panel: Quantile plot for the chi-square test of the anchor-adjustment model. Values should fall on a line with unit slope and zero intercept. Right panel: Scatter plot relating the bias parameters obtained from each order. To satisfy the QQ equality, the correlation must equal +1.0.
Figure 18.3 Left panel: Quantile plot for the chi-square test of the repeat choice model. Values should fall on a line with unit slope and zero intercept. Right panel: Scatter plot relating the context effect to the difference in marginal choice probabilities. The correlation is predicted to equal +1.0.
Figure 18.4 Left panel: Quantile plot for the chi-square test of the quantum model. Values should fall on the line with unit slope and zero intercept. Right panel: Scatter plot relating the two order effects in the minor diagonal of Table 18.5c. The observed correlation of $c018-math-171$ is close to the predicted slope of $c018-math-172$ .

Chapter 22: Science Between Trust and Control: Non-Reproducibility in Scholarly Publishing

Figure 22.1 Types of accompanying materials for an original paper

Chapter 25: Relevance Criteria for Reproducibility: The Contextual Emergence of Granularity

Figure 25.1 Two chiral versions of the molecule thalidomide, also known under the brand name Contergan^® (also Softenon^®); left: $c025-math-054$ -thalidomide acts as a sedative, right: $c025-math-055$ -thalidomide is acutely teratogenic and caused thousands of babies with deformed extremities in the 1950s.

List of Tables

Chapter 5: Statistical Issues in Reproducibility

Table 5.1 The probabilities of getting the same result in the replication study, for the four possible cases of correct null hypothesis $c05-math-071$ and alternative hypothesis $c05-math-072$ and significance of test results in the original study. * Replication of wrong results is undesirable.
Table 5.2 Levels of validation for same and different features (middle columns) in the original and follow-up study. The different terms in the left column are related to different types of validation in the right column.

Chapter 6: Model Selection, Data Distributions, and Reproducibility

Table Matrix 6.1 Prior probabilities for data distributions (left margin), distributions predicted by model instances (right margin), and model outcomes (bottom margin). A posterior version of the same matrix can be produced where the margins are conditioned on the observed data outcome $c06-math-083$ . As explained in the text, virtually all the equations and predictions in this chapter can be derived and explained with reference to this matrix. Data outcomes in bolded columns are explained in Section 6.4.2.
Table 6.1 follows the form of Matrix 6.1, with probabilities corresponding to the priors and data distributions for the toy example, for Case A described in the text. Case A assumes all distributions to be binomial, based on the corresponding value of $c06-math-208$ (from 0.0 to 1.0 in steps of 0.1).

Chapter 9: Facilitating Reproducibility in Scientific Computing: Principles and Practice

Table 9.1 Run times on parallel and vector systems for different problem sizes (data for Fig. 9.5).

Chapter 13: From Mice to Men: Translation from Bench to Bedside

Table 13.1 Outcome distribution for drugs in the “main focus” category and in the “comparison” category, within category “drug type.”

Chapter 18: Order Effects in Sequential Judgments and Decisions

Table 18.1a: white-black
Table 18.1b: black-white
Table 18.1c: order effects
Table 18.2 joint probabilities for the no order effect model
Table 18.3a: anchor-adjust joint probabilities for A–B order
Table 18.3b: anchor-adjust joint probabilities for B–A order
Table 18.3c: anchor-adjust predicted order effects
Table 18.4a: repeat-choice joint probabilities for A–B order
Table 18.4b: repeat-choice joint probabilities for B–A order
Table 18.4c: repeat-choice predicted order effects
Table 18.5a: quantum model joint probabilities for A–B order
Table 18.5b: quantum model joint probabilities for B–A order
Table 18.5c: quantum model predicted order effects

Reproducibility

Principles, Problems, Practices, and Prospects

Edited by

Harald Atmanspacher

Collegium Helveticum, University and ETH Zurich, Zurich, Switzerland

Sabine Maasen

Munich Center for Technology in Society, Technical University, Munich, Germany

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Classification: LCC Q175.32.O27 R47 2016 | DDC 001.4/2–dc23 LC record available at