Cover
Dedication
Preface
Acknowledgements
About the Companion Website
Part One: Theories
1. Chapter 1: Robust Statistics and its Application in Linear Regression
  1. 1.1 Robust Aspects of Data
  2. 1.2 Robust Statistics and the Mechanism for Producing Outliers
  3. 1.3 Location and Scale Parameters
  4. 1.4 Redescending M‐estimates
  5. 1.5 Breakdown Point
  6. 1.6 Linear Regression
  7. 1.7 The Robust Approach in Linear Regression
  8. 1.8 S‐estimator
  9. 1.9 Least Absolute and Quantile Esimates
  10. 1.10 Outlier Detection in Linear Regression
2. Chapter 2: Nonlinear Models: Concepts and Parameter Estimation
  1. 2.1 Introduction
  2. 2.2 Basic Concepts
  3. 2.3 Parameter Estimations
  4. 2.4 A Nonlinear Model Example
3. Chapter 3: Robust Estimators in Nonlinear Regression
  1. 3.1 Outliers in Nonlinear Regression
  2. 3.2 Breakdown Point in Nonlinear Regression
  3. 3.3 Parameter Estimation
  4. 3.4 Least Absolute and Quantile Estimates
  5. 3.5 Quantile Regression
  6. 3.6 Least Median of Squares
  7. 3.7 Least Trimmed Squares
  8. 3.8 Least Trimmed Differences
  9. 3.9 S‐estimator
  10. 3.10 ‐estimator
  11. 3.11 MM‐estimate
  12. 3.12 Environmental Data Examples
  13. 3.13 Nonlinear Models
  14. 3.14 Carbon Dioxide Data
  15. 3.15 Conclusion
4. Chapter 4: Heteroscedastic Variance
  1. 4.1 Definitions and Notations
  2. 4.2 Weighted Regression for the Nonparametric Variance Model
  3. 4.3 Maximum Likelihood Estimates
  4. 4.4 Variance Modeling and Estimation
  5. 4.5 Robust Multistage Estimate
  6. 4.6 Least Squares Estimate of Variance Parameters
  7. 4.7 Robust Least Squares Estimate of the Structural Variance Parameter
  8. 4.8 Weighted M‐estimate
  9. 4.9 Chicken‐growth Data Example
  10. 4.10 Toxicology Data Example
  11. 4.11 Evaluation and Comparison of Methods
5. Chapter 5: Autocorrelated Errors
  1. 5.1 Introduction
  2. 5.2 Nonlinear Autocorrelated Model
  3. 5.3 The Classic Two‐stage Estimator
  4. 5.4 Robust Two‐stage Estimator
  5. 5.5 Economic Data
  6. 5.6 ARIMA(1,0,1)(0,0,1)7 Autocorrelation Function
6. Chapter 6: Outlier Detection in Nonlinear Regression
  1. 6.1 Introduction
  2. 6.2 Estimation Methods
  3. 6.3 Point Influences
  4. 6.4 Outlier Detection Measures
  5. 6.5 Simulation Study
  6. 6.6 Numerical Example
  7. 6.7 Variance Heteroscedasticity
  8. 6.8 Conclusion
Part Two: Computations
1. Chapter 7: Optimization
  1. 7.1 Optimization Overview
  2. 7.2 Iterative Methods
  3. 7.3 Wolfe Condition
  4. 7.4 Convergence Criteria
  5. 7.5 Mixed Algorithm
  6. 7.6 Robust M‐estimator
  7. 7.7 The Generalized M‐estimator
  8. 7.8 Some Mathematical Notation
  9. 7.9 Genetic Algorithm
2. Chapter 8: nlr Package
  1. 8.1 Overview
  2. 8.2 nl.form Object
  3. 8.3 Model Fit by nlr
  4. 8.4 nlr.control
  5. 8.5 Fault Object
  6. 8.6 Ordinary Least Squares
  7. 8.7 Robust Estimators
  8. 8.8 Heteroscedastic Variance Case
  9. 8.9 Autocorrelated Errors
  10. 8.10 Outlier Detection
  11. 8.11 Initial Values and Self‐start
3. Chapter 9: Robust Nonlinear Regression in R
  1. 9.1 Lakes Data Examples
  2. 9.2 Simulated Data Examples
Appendix A: nlr Database
1. A.1 Data Set used in the Book
2. A.2 Nonlinear Regression Models
3. A.3 Robust Loss Functions Data Bases
4. A.4 Heterogeneous Variance Models
References
Index
End User License Agreement

List of Tables

Chapter 1
1. Table A.1 Chicken‐growth data (g).
2. Table A.2 Methane data.
3. Table A.3 Carbon dioxide data.
4. Table A.4 Lakes data.
5. Table A.5 Net money data.
6. Table A.6 Iran trademark data.
7. Table A.7 ntp data.
8. Table A.8 Cow milk production data for a single cow in a year.
9. Table A.9 Three cases of simulated outliers from logistic model.
10. Table A.10 Artificially contaminated data.
11. Table A.11 Robust rho functions.
12. Table A.12 Variance model functions.
Chapter 1
1. Table 1.1 Linear regression formulas.
Chapter 3
1. Table 3.1 The robust MM‐ and classical OLS estimates for the scaled exponential model, CH₄ data.
2. Table 3.2 The robust MM‐ and classical OLS estimates for the exponential convex model, CH₄ data.
3. Table 3.3 The robust MM‐ and classical OLS estimates for the power model, CH₄ data.
4. Table 3.4 The robust MM‐ and classical OLS estimates for the exponential with intercept model, CH₄ data
5. Table 3.5 The robust MM‐ and classical OLS estimates for the scaled exponential model, CO₂ data.
6. Table 3.6 The robust MM‐ and classical OLS estimates for the scaled exponential convex model, CO₂ data
7. Table 3.7 The robust MM‐ and classical OLS estimates for the power model, CO₂ data
8. Table 3.8 The robust MM‐ and classical OLS estimates for the exponential with intercept model, CO₂ data
Chapter 4
1. 1 Parameter estimates for chicken‐growth data using a logistic model with a power heteroscedastic function.
2. Table 4.2 Parameter estimates of the Hill model for chromium concentration in the kidney of mouse (values in parentheses show estimated errors).
Chapter 5
1. Table 5.1 The robust MM‐estimates of fitted four models for the Iran trademark data.
2. Table 5.2 The RTS estimates of the fitted exponential with intercept models for the Iran trademark data, with ARIMA(1,0,1)(0,0,1)7 autocorrelated structure.
Chapter 6
1. Table 6.1 Measures based on OLS estimates for a data set with one outlier (Case1): mixture of OLS with tangential leverage and Jacobian leverage.
2. Table 6.2 Measures based on MM‐estimates for a data set with one outlier (Case1): mixture of MM with tangential leverage, Jacobian leverage, and robust Jacobian leverage.
3. Table 6.3 Measures based on OLS estimates for a data set with three outliers (Cases 6, 7, and 8): mixture of OLS with tangential leverage and Jacobian leverage.
4. Table 6.4 Measures based on MM‐estimates for a data set with three outliers (Cases 6, 7, and 8): mixture of MM with tangential leverage, Jacobian leverage and robust Jacobian leverage.
5. Table 6.5 Measures based on OLS estimates for a data set with six outliers (last six observations in Cases 6, 7, and 8): mixture of OLS with tangential leverage and Jacobian leverage.
6. Table 6.6 Measures based on MM‐estimates for a data set with six outliers (last six observations in Cases 6, 7, and 8): mixture of MM with tngential leverage, Jacobian leverage and robust Jacobian leverage.
7. Table 6.7 Outlier measures for lakes data, computed using the hat matrix and Jacobian leverage, and the MM‐estimator.
Chapter 9
1. Table 9.1 Parameter estimates for lakes data, using the nlr, nlrq, nlrob, and nls functions.
2. Table 9.2 Parameter estimates for artificially contaminated data, using nlr, nlrq, nlrob, and nls functions, and parameter biases.

List of Illustrations

Chapter 1
1. Figure 1.1 Contaminated normal densities: (a) mixture of two normal distributions with different means; (b) mixture of two normal distributions with different variances.
2. Figure 1.2 (a) functions; (b) functions.
Chapter 2
1. Figure 2.1 Fitted Wood model for a single cow's milk production data for a year.
Chapter 3
1. Figure 3.1 Simulated logistic model, artificially contaminated. (a) Logistic model: solid line, least squares estimate; dashed line, robust MM‐estimate. (b) Computed residuals. Upper chart: least squares estimate; lower chart: MM‐estimate.
2. Figure 3.2 Simulated linear model, artificially contaminated. (a) Least squares fit: solid line: least squares estimate; dashed line: robust MM‐estimate. (b) Computed residuals: top, least squares estimate; bottom, MM‐estimate.
3. Figure 3.3 Simulated nonlinear exponential model, artificially contaminated. (a) Exponential model: solid line, estimates using OLS; dotted line, LMS using Nelder–Mead optimization; dashed line, LMS using a genetic algorithm. (b) Median of error squares: objective function LMS for different parameter values.
4. Figure 3.4 Four models fitted to methane data using the robust MM‐estimator.
5. Figure 3.5 Four models fitted to carbon dioxide data using the robust MM‐estimator.
Chapter 4
1. Figure 4.1 Chicken‐growth data estimates using the classical multistage estimator.
2. Figure 4.2 Plot of the standardized residual and its square against the predictor and response to assess the adequacy of the weighted least squares of the model. Source: Riazoshams and Midi (2009). Reproduced with permission of European Journal of Science.
3. Figure 4.3 Graphs for choosing the model variance: (a) exponential variance model assessment; (b) power variance model assessment; (c) adequacy of estimated variance model; (d) assessment of the equality of standard deviation and estimate thereof.
4. Figure 4.4 Toxicology data, fitted by robust and classical methods: (a) classical estimates, (b) robust estimates.
Chapter 5
1. Figure 5.1 Fitted exponential with intercept model for the Iran trademark data using OLS and MM‐estimates.
2. Figure 5.2 Fitted exponential with intercept model for the Iran net money data using OLS and MM‐estimates.
3. Figure 5.3 ACF, PACF, and robust ACF for OLS and MM‐estimates. (a) MM‐estimate residuals, (b) MM‐estimate residuals, (c) OLS‐estimate residuals, (d) OLS‐estimate residuals, (e) MM‐estimate residuals, (f) OLS‐estimate residuals.
4. Figure 5.4 ACF, PACF, and robust ACF for OLS and MM‐estimates. The residual of the outlier was deleted from the computed residuals from model fits. (a) MM‐estimate, (b) MM‐estimate, (c) OLS‐estimate, (d) OLS‐estimate, (e) robust ACF MM‐estimate, (f) robust ACF OLS estimate.
5. Figure 5.5 ACF, PACF, and robust ACF for OLS and MM‐estimates. The fits were performed after the outlier had been deleted and the residuals were computed from the fitted model without the outlying point: (a) MM‐estimate, (b) MM‐estimate, (c) OLS‐estimate, (d) OLS‐estimate, (e) robust ACF, (f) robust ACF.
6. Figure 5.6 Model fit using OLS and MM‐estimators when the response of the outlier point was replaced by its prediction.
7. Figure 5.7 ACF, PACF, and robust ACF for OLS and MM‐estimates. Residuals were computed from the fitted model when the response of the outlier point was replaced by its prediction. (a) MM‐estimate, (b) MM‐estimate, (c) OLS‐estimate, (d) OLS‐estimate, (e) robust ACF, (f) robust ACF.
Chapter 6
1. Figure 6.1 Simulated outliers. Case 6.1: one outlier; Case 6.2: three outliers; Case 6.3: six high leverage points.
2. Figure 6.2 Artificial data: (a) without outliers, (b) contaminated data with outlier data points but variances are not outliers, (c) contaminated data with extremely small variance but data points are not outliers.
3. Figure 6.3 Artificial data, nonreplicated data.
4. Figure 6.4 Artificial data, replicated data.
5. Figure 6.5 Artificial data, replicated data: (a) without outlier, (b) with data outliers in the middle but their variance is not outlier.
6. Figure 6.6 Artificial replicated and nonreplicated data: (a) replicated data with small variance outlier at the end of the data series, (b) nonreplicated data with a single outlier in the middle.
Chapter 8
1. Figure 8.1 Plot of predicted values for t he scaled exponential model, computed using initial values from the selfStart slot for carbon dioxide data.
2. Figure 8.2 Output objects, inheritance hierarchy.
3. Figure 8.3 Fitted scale exponential convex model for the carbon data, using the history=T command.
4. Figure 8.4 Fitted scale exponential convex model for the carbon data, using Nelder–Mead algorithm: (a) using the default ; (b) using , and both using selfStart as initial values.
5. Figure 8.5 Fitted scale exponential convex model for methane gas data using least mean squares.
6. Figure 8.6 Fitted logistic model with the power variance function model for chicken‐growth data.
7. Figure 8.7 Fitted Hill model and Hill variance function model for mouse kidney data. WM Weighted M-estimate, RME Robust Multi Stage Estimate.
8. Figure 8.8 Outlier detection measures for lakes data: studentized residual, Cook distance, and .
9. Figure 8.9 Tangential plane leverage, robust Jacobian leverage and their differences for lakes data.
10. Figure 8.10 Fitted exponential with intercept for Iran net money data using: (a) LMS and (b) selfStart as the initial value.
11. Figure 8.11 Fitted power model for Iran net money data.
Chapter 9
1. Figure 9.1 Outlier detection measures based on manual initial values (=1, =1).
2. Figure 9.2 Artificially contaminated data fitted with six methods from three packages.

Copyright

This edition first published 2019

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

The right of Hossein Riazoshams, Habshah Midi and Gebrenegus Ghilagaber to be identified as the authors of this work has been asserted in accordance with law.

Registered Offices

John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

Editorial Office

9600 Garsington Road, Oxford, OX4 2DQ, UK

For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears in standard print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of Warranty

While the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

Library of Congress Cataloging-in-Publication Data

Names: Riazoshams, Hossein, 1971-- author. | Midi, Habshah, author. | Ghilagaber, Gebrenegus, author.

Title: Robust nonlinear regression: with applications using R / Hossein Riazoshams, Habshah Midi, Gebrenegus Ghilagaber.

Description: Hoboken, NJ : John Wiley & Sons, 2018. | Includes bibliographical references and index. |

Identifiers: LCCN 2017057347 (print) | LCCN 2018005931 (ebook) | ISBN 9781119010456 (pdf) | ISBN 9781119010449 (epub) | ISBN 9781118738061 (cloth)

Subjects: LCSH: Regression analysis. | Nonlinear theories. | R (Computer program language)

Classification: LCC QA278.2 (ebook) | LCC QA278.2 .R48 2018 (print) | DDC 519.5/36-dc23

LC record available at https://lccn.loc.gov/2017057347

Cover Design: Wiley

Cover Image: © Wavebreakmedia Ltd/Getty Images; © Courtesy of Hossein Riazoshams

Preface

This book is the result of the first author's research, between 2004 and 2016, in the robust nonlinear regression area, when he was affiliated with the institutions listed. The lack of computer programs together with mathematical development in this area encouraged us to write this book and provide an R‐package called nlr for which a guide is provided in this book. The book concentrates more on applications and thus practical examples are presented.

Robust statistics describes the methods used when the classical assumptions of statistics do not hold. It is mostly applied when a data set includes outliers that lead to violation of the classical assumptions.

The book is divided into two parts. In Part 1, the mathematical theories of robust nonlinear regression are discussed and parameter estimation for heteroscedastic error variances, autocorrelated errors, and several methods for outlier detection are presented. Part 2 presents numerical methods and R‐tools for nonlinear regression using robust methods.

In Chapter 1, the basic theories of robust statistics are discussed. Robust approaches to linear regression and outlier detection are presented. These mathematical concepts of robust statistics and linear regression are then extended to nonlinear regression in the rest of the book. Since the book is about nonlinear regression, the proofs of theorems related to robust linear regression are omitted.

Chapter 2 presents the concepts of nonlinear regression and discusses the theory behind several methods of parameter estimation in this area. The robust forms of these methods are outlined in Chapter 3. Chapter 2 presents the generalized least square estimate, which will be used for non‐classical situations.

Chapter 3 discusses the concepts of robust statistics, such as robustness and breakdown points, in the context of nonlinear regression. It also presents several robust parameter estimation techniques.

Chapter 4 develops the robust methods for a null condition when the error variances are not homogeneous. Different kinds of outlier are defined and their effects are discussed. Parameter estimation for nonlinear function models and variance function models are presented.

Another null condition, when the errors are autocorrelated, is discussed in Chapter 5. Robust and classical methods for estimating the nonlinear function model and the autocorrelation structure of the error are presented. The effect of different kinds of outlier are explained, and appropriate methods for identifying the correlation structure of errors in the presence of outliers are studied.

Chapter 6 explains the methods for identifying atypical points. The outlier detection methods that are developed in this chapter are based mainly on statistical measures that use robust estimators of the parameters of the nonlinear function model.

In Chapter 7, optimization methods are discussed. These techniques are then modified to solve the minimization problems found in robust nonlinear regressions. They will then used to solve the mathematical problems discussed in Part 1 of the book and their implementation in a new R package called nlr is then covered in Chapter 8.

Chapter 8 is a guide to the R package implemented for this book. It covers object definition for a nonlinear function model, parameter estimation, and outlier detection for several model assumption situations discussed in the Part 1. This chapter shows how to fit nonlinear models to real‐life and simulated data.

In Chapter 9, another R packages for robust nonlinear regression are presented and compared to nlr. Appendix A presents and describes the databases embedded in nlr, and the nonlinear models and functions available.

At the time of writing, the nlr package is complete, and is available at The Comprehensive R Archive Network (CRAN‐project) at https://cran.r‐project.org/package=nlr.

Because of the large number of figures and programs involved, there are many examples that could not be included in the book. Materials, programs, further examples, and a forum to share and discuss program bugs are all provided at the author's website at http://www.riazoshams.com/nlr and at the book's page on the Wiley website.

Response Manager, Shabdiz Music School of Iran,

Full time faculty member of Islamic Azad University of Lamerd, Iran,

Department of Statistics, Stockholm University, Sweden,

Institute for Mathematical, Research University of Putra, Malaysia

November 2017

Hossein Riazoshams

Robust Nonlinear Regression: with Applications using R

Copyright

Dedication

Preface

Acknowledgements

About the Companion Website

Part One
Theories