Table of Contents

Cover

Title Page

Preface

References

Chapter 1: The need for more than one random-effect term when fitting a regression line

1.1 A data set with several observations of variable Y at each value of variable X
1.2 Simple regression analysis: Use of the software GenStat to perform the analysis
1.3 Regression analysis on the group means
1.4 A regression model with a term for the groups
1.5 Construction of the appropriate F test for the significance of the explanatory variable when groups are present
1.6 The decision to specify a model term as random: A mixed model
1.7 Comparison of the tests in a mixed model with a test of lack of fit
1.8 The use of REsidual Maximum Likelihood (REML) to fit the mixed model
1.9 Equivalence of the different analyses when the number of observations per group is constant
1.10 Testing the assumptions of the analyses: Inspection of the residual values
1.11 Use of the software R to perform the analyses
1.12 Use of the software SAS to perform the analyses
1.13 Fitting a mixed model using GenStat's Graphical User Interface (GUI)
1.14 Summary
1.15 Exercises
References

Chapter 2: The need for more than one random-effect term in a designed experiment

2.1 The split plot design: A design with more than one random-effect term
2.2 The analysis of variance of the split plot design: A random-effect term for the main plots
2.3 Consequences of failure to recognize the main plots when analysing the split plot design
2.4 The use of mixed modelling to analyse the split plot design
2.5 A more conservative alternative to the F and Wald statistics
2.6 Justification for regarding block effects as random
2.7 Testing the assumptions of the analyses: Inspection of the residual values
2.8 Use of R to perform the analyses
2.9 Use of SAS to perform the analyses
2.10 Summary
2.11 Exercises
References

Chapter 3: Estimation of the variances of random-effect terms

3.1 The need to estimate variance components
3.2 A hierarchical random-effects model for a three-stage assay process
3.3 The relationship between variance components and stratum mean squares
3.4 Estimation of the variance components in the hierarchical random-effects model
3.5 Design of an optimum strategy for future sampling
3.6 Use of R to analyse the hierarchical three-stage assay process
3.7 Use of SAS to analyse the hierarchical three-stage assay process
3.8 Genetic variation: A crop field trial with an unbalanced design
3.9 Production of a balanced experimental design by ‘padding’ with missing values
3.10 Specification of a treatment term as a random-effect term: The use of mixed-model analysis to analyse an unbalanced data set
3.11 Comparison of a variance component estimate with its standard error
3.12 An alternative significance test for variance components
3.13 Comparison among significance tests for variance components
3.14 Inspection of the residual values
3.15 Heritability: The prediction of genetic advance under selection
3.16 Use of R to analyse the unbalanced field trial
3.17 Use of SAS to analyse the unbalanced field trial
3.18 Estimation of variance components in the regression analysis on grouped data
3.19 Estimation of variance components for block effects in the split-plot experimental design
3.20 Summary
3.21 Exercises
References

Chapter 4: Interval estimates for fixed-effect terms in mixed models

4.1 The concept of an interval estimate
4.2 Standard errors for regression coefficients in a mixed-model analysis
4.3 Standard errors for differences between treatment means in the split-plot design
4.4 A significance test for the difference between treatment means
4.5 The least significant difference (LSD) between treatment means
4.6 Standard errors for treatment means in designed experiments: A difference in approach between analysis of variance and mixed-model analysis
4.7 Use of R to obtain SEs of means in a designed experiment
4.8 Use of SAS to obtain SEs of means in a designed experiment
4.9 Summary
4.10 Exercises
References

Chapter 5: Estimation of random effects in mixed models: Best Linear Unbiased Predictors (BLUPs)

5.1 The difference between the estimates of fixed and random effects
5.2 The method for estimation of random effects: The best linear unbiased predictor (BLUP) or ‘shrunk estimate’
5.3 The relationship between the shrinkage of BLUPs and regression towards the mean
5.4 Use of R for the estimation of fixed and random effects
5.5 Use of SAS for the estimation of random effects
5.6 The Bayesian interpretation of BLUPs: Justification of a random-effect term without invoking an underlying infinite population
5.7 Summary
5.8 Exercises
References

Chapter 6: More advanced mixed models for more elaborate data sets

6.1 Features of the models introduced so far: A review
6.2 Further combinations of model features
6.3 The choice of model terms to be specified as random
6.4 Disagreement concerning the appropriate significance test when fixed- and random-effect terms interact: ‘The great mixed-model muddle’
6.5 Arguments for specifying block effects as random
6.6 Examples of the choice of fixed- and random-effect specification of terms
6.7 Summary
6.8 Exercises
References

Chapter 7: Three case studies

7.1 Further development of mixed modelling concepts through the analysis of specific data sets
7.2 A fixed-effects model with several variates and factors
7.3 Use of R to fit the fixed-effects model with several variates and factors
7.4 Use of SAS to fit the fixed-effects model with several variates and factors
7.5 A random coefficient regression model
7.6 Use of R to fit the random coefficients model
7.7 Use of SAS to fit the random coefficients model
7.8 A random-effects model with several factors
7.9 Use of R to fit the random-effects model with several factors
7.10 Use of SAS to fit the random-effects model with several factors
7.11 Summary
7.12 Exercises
References

Chapter 8: Meta-analysis and the multiple testing problem

8.1 Meta-analysis: Combined analysis of a set of studies
8.2 Fixed-effect meta-analysis with estimation only of the main effect of treatment
8.3 Random-effects meta-analysis with estimation of study × treatment interaction effects
8.4 A random-effect interaction between two fixed-effect terms
8.5 Meta-analysis of individual-subject data using R
8.6 Meta-analysis of individual-subject data using SAS
8.7 Meta-analysis when only summary data are available
8.8 The multiple testing problem: Shrinkage of BLUPs as a defence against the Winner's Curse
8.9 Fitting of multiple models using R
8.10 Fitting of multiple models using SAS
8.11 Summary
8.12 Exercises
References

Chapter 9: The use of mixed models for the analysis of unbalanced experimental designs

9.1 A balanced incomplete block design
9.2 Imbalance due to a missing block: Mixed-model analysis of the incomplete block design
9.3 Use of R to analyse the incomplete block design
9.4 Use of SAS to analyse the incomplete block design
9.5 Relaxation of the requirement for balance: Alpha designs
9.6 Approximate balance in two directions: The alphalpha design
9.7 Use of R to analyse the alphalpha design
9.8 Use of SAS to analyse the alphalpha design
9.9 Summary
9.10 Exercises
References

Chapter 10: Beyond mixed modelling

10.1 Review of the uses of mixed models
10.2 The generalized linear mixed model (GLMM): Fitting a logistic (sigmoidal) curve to proportions of observations
10.3 Use of R to fit the logistic curve
10.4 Use of SAS to fit the logistic curve
10.5 Fitting a GLMM to a contingency table: Trouble-shooting when the mixed modelling process fails
10.6 The hierarchical generalized linear model (HGLM)
10.7 Use of R to fit a GLMM and a HGLM to a contingency table
10.8 Use of SAS to fit a GLMM to a contingency table
10.9 The role of the covariance matrix in the specification of a mixed model
10.10 A more general pattern in the covariance matrix: Analysis of pedigrees and genetic data
10.11 Estimation of parameters in the covariance matrix: Analysis of temporal and spatial variation
10.12 Use of R to model spatial variation
10.13 Use of SAS to model spatial variation
10.14 Summary
10.15 Exercises
References

Chapter 11: Why is the criterion for fitting mixed models called REsidual Maximum Likelihood?

11.1 Maximum likelihood and residual maximum likelihood
11.2 Estimation of the variance σ² from a single observation using the maximum-likelihood criterion
11.3 Estimation of σ² from more than one observation
11.4 The μ-effect axis as a dimension within the sample space
11.5 Simultaneous estimation of μ and σ² using the maximum-likelihood criterion
11.6 An alternative estimate of σ² using the REML criterion
11.7 Bayesian justification of the REML criterion
11.8 Extension to the general linear model: The fixed-effect axes as a sub-space of the sample space
11.9 Application of the REML criterion to the general linear model
11.10 Extension to models with more than one random-effect term
11.11 Summary
11.12 Exercises
References

Index

End User License Agreement

List of Illustrations

Figure 1.1

Figure 1.2

Figure 1.3

Figure 1.4

Figure 1.5

Figure 1.6

Figure 1.7

Figure 1.8

Figure 1.9

Figure 1.10

Figure 1.11

Figure 1.12

Figure 1.13

Figure 1.14

Figure 1.15

Figure 1.16

Figure 1.17

Figure 1.18

Figure 2.1

Figure 2.2

Figure 2.3

Figure 3.1

Figure 3.2

Figure 3.3

Figure 3.4

Figure 3.5

Figure 3.6

Figure 4.1

Figure 4.2

Figure 4.3

Figure 4.4

Figure 4.5

Figure 5.1

Figure 5.2

Figure 5.3

Figure 5.4.

Figure 5.5.

Figure 6.1

Figure 6.2

Figure 6.3

Figure 7.1

Figure 7.2

Figure 7.3

Figure 7.4

Figure 7.5

Figure 7.6

Figure 7.7

Figure 8.1

Figure 8.2

Figure 8.3

Figure 8.4

Figure 8.5

Figure 8.6

Figure 9.1

Figure 10.1

Figure 10.2

Figure 10.3

Figure 10.4

Figure 10.5

Figure 10.6

Figure 10.7

Figure 10.8

Figure 10.9

Figure 10.10

Figure 10.11

Figure 10.12

Figure 11.1

Figure 11.2

Figure 11.3

Figure 11.4

Figure 11.5

Figure 11.6

Figure 11.7

Figure 11.8

Figure 11.9

Figure 11.10

Figure 11.11

Figure 11.12

Figure 11.13

List of Tables

Table 1.1

Table 1.2

Table 1.3

Table 1.4

Table 1.5

Table 1.6

Table 1.7

Table 1.8

Table 1.9

Table 2.1

Table 2.2

Table 2.3

Table 2.4

Table 2.5

Table 3.1

Table 3.2

Table 3.3

Table 3.4

Table 3.5

Table 3.6

Table 3.7

Table 3.8

Table 3.9

Table 3.10

Table 3.11

Table 3.12

Table 3.13

Table 4.1

Table 4.2

Table 4.3

Table 4.4

Table 4.5

Table 4.6

Table 4.7

Table 5.1

Table 5.2

Table 6.1

Table 6.2

Table 6.3

Table 6.4

Table 6.5

Table 6.6

Table 6.7

Table 6.8

Table 6.9.

Table 7.1

Table 7.2

Table 7.3

Table 7.4

Table 7.5

Table 7.6

Table 7.7

Table 7.8

Table 7.9

Table 7.10

Table 7.11

Table 7.12

Table 7.13

Table 7.14

Table 7.15

Table 8.1

Table 8.2

Table 8.3

Table 8.4

Table 8.5

Table 8.6

Table 8.7

Table 8.8

Table 9.1

Table 9.2

Table 9.3

Table 9.4

Table 9.5

Table 10.1

Table 10.2

Table 10.3

Table 10.4

Table 10.5

Table 10.6

Table 10.7

Table 10.8

Table 10.9

Table 10.10

Table 10.11

Table 10.12

Table 11.1

Table 11.2

Table 11.3

Table 11.4

Table 11.5

Table 11.6

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom

For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com.

The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. It is sold on the understanding that the publisher is not engaged in rendering professional services and neither the publisher nor the author shall be liable for damages arising herefrom. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Introduction to mixed modelling : beyond regression and analysis of variance / N. W. Galwey. – Second edition.

1. Multilevel models (Statistics) 2. Experimental design. 3. Regression analysis. 4. Analysis of variance. I. Title.

Preface

This book is intended for research workers and students who have made some use of the statistical techniques of regression analysis and analysis of variance (anova), but who are unfamiliar with mixed models and the criterion for fitting them called REsidual Maximum Likelihood (REML, also known as REstricted Maximum Likelihood). Such readers will know that, broadly speaking, regression analysis seeks to account for the variation in a response variable by relating it to one or more explanatory variables, whereas anova seeks to detect variation among the mean values of groups of observations. In regression analysis, the statistical significance of each explanatory variable is tested using the same estimate of residual variance, namely the residual mean square, and this estimate is also used to calculate the standard error of the effect of each explanatory variable. However, this choice is not always appropriate. Sometimes, one or more of the terms in the regression model (in addition to the residual term) represents random variation, and such a term will contribute to the observed variation in other terms. It should therefore contribute to the significance tests and standard errors of these terms: but in an ordinary regression analysis, it does not do so. Anova, on the other hand, does allow the construction of models with additional random-effect terms, known as block terms. However, it does so only in the limited context of balanced experimental designs.

The capabilities of regression analysis can be combined with those of anova by fitting to the data a mixed model, so called because it contains both fixed-effect and random-effect terms. A mixed model allows the presence of additional random-effects terms to be recognized in the full range of regression models, not only in balanced designs. Any statistical analysis that can be specified by a general linear model (the broadest form of linear regression model) or by anova can also be specified by a mixed model. However, the specification of a mixed model requires an additional step. The researcher must decide, for each term in the model, whether effects of that term (e.g. the deviations of group means from the grand mean) can be regarded as values of a random variable—usually taken to mean that they are a random sample from some much larger population—or whether they are a fixed set. In some cases, this decision is straightforward: in others, the distinction is subtle and the decision difficult. However, provided that an appropriate decision is made (see Section 6.3), the mixed model specifies a statistical analysis which is of broader validity than regression analysis or anova, and which is nearly or fully equivalent to those methods in the special cases where they are applicable.

It is fairly straightforward to specify the calculations required for regression analysis and anova, and this is done in many standard textbooks. For example, Draper and Smith (1998) give a clear, thorough and extensive account of the methods of regression analysis, and Mead (1988) does the same for the analysis of variance. To solve the equations that specify a mixed model is much less straightforward. The model is fitted—that is, the best estimates of its parameters are obtained—using the REML criterion, but the fitting process requires recursive numerical methods. It is largely because of this burden of calculation that mixed models are less familiar than regression analysis and anova: it is only in about the past three decades that the development of computer power and user-friendly statistical software has allowed them to be used routinely in research. This book aims to provide a guide to the use of mixed models that is accessible to the broad community of research scientists. It focuses not on the details of calculation, but on the specification of mixed models and the interpretation of the results.

The numerical examples in this book are presented and analysed using three statistical software systems, namely

GenStat, distributed by VSN International Ltd, Hemel Hempstead, via the website https://www.vsni.co.uk/.
R, from The R Project for Statistical Computing. This software can be downloaded free of charge from the website http://www.r-project.org/.
SAS, available from the SAS Institute via the website http://www.sas.com/technologies/analytics/statistics/stat/.

GenStat is a natural choice of software to illustrate the concepts and methods employed in mixed modelling because its facilities for this purpose are straightforward to use, extensive and well integrated with the rest of the system and because their output is clearly laid out and easy to interpret. Above all, the recognition of random terms in statistical models lies at the heart of GenStat. GenStat's method of specifying anova models requires the distinction of random-effect (block) and fixed-effect (treatment) terms, which makes the interpretation of designed experiments uniquely reliable and straightforward. This approach extends naturally to mixed models and provides a firm framework within which the researcher can think and plan. Despite these merits, GenStat is not among the most widely used statistical software systems, and the numerical examples are therefore also analysed using the increasingly popular software R, as well as SAS, which is long-established and widely used in the clinical and agricultural research communities.

The book's website, http://www.wiley.com/go/beyond_regression, provides solutions to the end-of-chapter exercises, as well as data files, and programs in GenStat, R and SAS, for many of the examples in this book.

This second edition incorporates many additions and changes, as well as some corrections. The most substantial are

the addition of SAS to the software systems used;
a new chapter on meta-analysis and the multiple testing problem;
recognition of situations in which it is appropriate to specify the interaction between two factors as a random-effect term, even though both of the corresponding main effects are fixed-effect terms;
an account of the Bayesian interpretation of mixed models, an alternative to the random-sample (frequentist) interpretation mentioned above;
a fuller account of the ‘great mixed model muddle’;
the random coefficient regression model.

I am grateful to the following individuals for their valuable comments and suggestions on the manuscript of this book, and/or for introducing me to mixed-modelling concepts and techniques in the three software systems: David Balding, Aruna Bansal, Caroline Galwey, Toby Johnson, Peter Lane, Roger Payne, James Roger and David Willé. I am also grateful to the participants in the GenStat Discussion List for their helpful responses to many enquiries. (Access to this lively forum can be obtained via the website https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=GENSTAT.) Any errors or omissions of fact or interpretation that remain are my sole responsibility. I would also like to express my gratitude to the many individuals and organizations who have given permission for the reproduction of data in the numerical examples presented. They are acknowledged individually in their respective places, but the high level of support that they have given me deserves to be recognized here.

References

Draper, N.R. and Smith, H. (1998) Applied Regression Analysis, 3rd edn, John Wiley & Sons, Inc., New York, 706 pp.
Mead, R. (1988) The Design of Experiments: Statistical Principles for Practical Application, Cambridge University Press, Cambridge, 620 pp.

Introduction to Mixed Modelling

Beyond Regression and Analysis of Variance

Preface

References