Contents
Preface
Acknowledgments
Introduction
Performance versus Scalability
Part 1 The Basics
1 Hardware Platform
1.1 Turing Machine
1.2 von Neumann Machine
1.3 Zuse Machine
1.4 Intel Machine
1.5 Sun Machine
1.6 System Under Test
1.7 Odds Against Turing
1.8 Sizing Hardware
1.9 Summary
Recommended Reading
Exercises
2 Software Platform
2.1 Software Stack
2.2 APIs
2.3 Multithreading
2.4 Categorizing Software
2.5 Enterprise Computing
2.6 Summary
Recommended Reading
Exercises
3. Testing Software Performance and Scalability
3.1 Scope of Software Performance and Scalability Testing
3.2 Software Development Process
3.3 Defining Software Performance
3.4 Stochastic Nature of Software Performance Measurements
3.5 Amdahl’s Law
3.6 Software Performance and Scalability Factors
3.7 System Performance Counters
3.8 Software Performance Data Principles
3.9 Summary
Recommended Reading
Exercises
Part 2 Applying Queuing Theory
4 Introduction to Queuing Theory
4.1 Queuing Concepts and Metrics
4.2 Introduction to Probability Theory
4.3 Applying Probability Theory to Queuing Systems
4.4 Queuing Models for Networked Queuing Systems
4.5 Summary
Recommended Reading
Exercises
5 Case Study I: Queuing Theory Applied to SOA
5.1 Introduction to SOA
5.2 XML Web Services
5.3 The Analytical Model
5.4 Service Demand
5.5 MedRec Application
5.6 MedRec Deployment and Test Scenario
5.7 Test Results
5.8 Comparing the Model with the Measurements
5.9 Validity of the SOA Performance Model
5.10 Summary
Recommended Reading
Exercises
6 Case Study II: Queuing Theory Applied to Optimizing and Tuning Software Performance and Scalability
6.1 Analyzing Software Performance and Scalability
6.2 Effective Optimization and Tuning Techniques
6.3 Balanced Queuing System
6.4 Summary
Recommended Reading
Exercises
Part 3 Applying API Profiling
7 Defining API Profiling Framework
7.1 Defense Lines Against Software Performance and Scalability Defects
7.2 Software Program Execution Stack
7.3 The PerfBasic API Profiling Framework
7.4 Summary
Exercises
8 Enabling API Profiling Framework
8.1 Overall Structure
8.2 Global Parameters
8.3 Main Logic
8.4 Processing Files
8.5 Enabling Profiling
8.6 Processing Inner Classes
8.7 Processing Comments
8.8 Processing Method Begin
8.9 Processing Return Statements
8.10 Processing Method End
8.11 Processing Main Method
8.12 Test Program
8.13 Summary
Recommended Reading
Exercises
9 Implementing API Profiling Framework
9.1 Graphics Tool—dot
9.2 Graphics Tool—ILOG
9.3 Graphics Resolution
9.4 Implementation
9.5 Summary
Exercises
10 Case Study: Applying API Profiling to Solving Software Performance and Scalability Challenges
10.1 Enabling API Profiling
10.2 API Profiling with Standard Logs
10.3 API Profiling with Custom Logs
10.4 API Profiling with Combo Logs
10.5 Applying API Profiling to Solving Performance and Scalability Problems
10.6 Summary
Exercises
Appendix A Stochastic Equilibrium and Ergodicity
A.1 Basic Concepts
A.2 Classification of Random Processes
A.3 Discrete-Time Markov Chains
A.4 Continuous-Time Markov Chains
A.5 Stochastic Equilibrium and Ergodicity
A.6 Birth–Death Chains
Appendix B Memoryless Property of the Exponential Distribution
Appendix C M/M/1 Queues at Steady State
C.1 Review of Birth–Death Chains
C.2 Utilization and Throughput
C.3 Average Queue Length in the System
C.4 Average System Time
C.5 Average Wait Time
Index
Copyright © 2009 by IEEE Computer Society. All rights reserved.
Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.
Library of Congress Cataloging-in-Publication Data:
Liu, Henry H.
Software performance and scalability : a quantitative approach/Henry H. Liu
p. cm.
Includes bibliographical references and index.
ISBN 978-0-470-46253-9 (cloth)
1. Computer software—Development. 2. System design. I. Title.
QA76.76.D47L577 2009
005.1—dc22
2009005654
To my family
Preface
Software platforms are a written product of the mind.
—D. S. Evans, A. Hagiu, and R. Schmalensee
WHY THIS BOOK
Few people would disagree with the fact that building a large-scale, high-performance, and scalable software system is a complex task. This is evidenced by the magnitude of required up-front and ongoing financial costs and personnel commonly seen at every large software development organization. Seeking effective, efficient, and economical approaches to developing large-scale software is of interest to the entire software community.
Regardless of its complexity and scope, every software development project is driven by a few common factors:
While management is responsible for providing sufficient budget to cover personnel, development, test infrastructure, and so on, we, the technical staff (developers, quality assurance engineers, and performance engineers), are accountable for delivering the software product under development on schedule and within budget while meeting high standards on performance and scalability.
However, it’s not uncommon to see that performance and scalability are pushed aside by the following higher priority activities:
Performance and scalability are often an afterthought during the last minute of product release. And even worse, performance and scalability issues might actually be raised by unsatisfied customers as soon as the product is rushed to the market. Under such circumstances, intense pressure builds up internally, and panic and fire-fighting-like chaos ensues.
On the other hand, software performance and scalability are indeed very challenging technical issues. Precautions must be taken with every major effort to improve the performance and scalability of a software product. A few industrial pioneers have issued warnings:
So, how can we implement an effective, efficient, and economical approach to building performance and scalability into software? Establishing a very capable performance and scalability test team would certainly help. However, it is my observation that this approach is insufficient for guaranteeing that performance and scalability issues are dealt with properly, as it may easily exclude software developers from taking performance and scalability concerns into account in the first place. It’s a reactive and less efficient approach to let the performance and scalability test engineers find the performance and scalability defects and then fix them with the developers. It’s a lot more costly to fix software performance and scalability defects without having the developers take care of them in the first place.
That’s the motivation behind this book, which promotes a proactive approach of letting the software developers build the performance and scalability into the product and letting the performance and scalability test engineers concentrate on the performance and scalability verification tests with a larger volume of data, more representative workloads, and more powerful hardware. This approach requires a mindset shift for the software developers that their job is just to make the software work and the performance and scalability problems can be fixed outside their job scope. Software developers should think consciously from performance and scalability perspectives whenever they make a design or an implementation decision.
Software developers already possess strong, valuable, and hard-to-obtain software design and implementation skills. Regardless of their experience, they can complement their existing coding skills by acquiring from this book knowledge about designing and implementing performance and scalability into their products in the first place during the various life cycles of development.
Of course, it’s impractical to have only the software developers take care of all the performance and scalability challenges. Building a software system that performs and scales is a cross-team effort. This book provides a common knowledge platform for all stakeholders to work together to tame the challenging performance and scalability issues so that the product they are all responsible for is built to perform and scale.
WHO THIS BOOK IS FOR
If you are reading this book, probably you are interested in learning how you can help design and build performance and scalability into your software for which you are one of the stakeholders, either from the technical or management perspective. No matter what your roles are, I am very confident that you will learn something from this book that can help you become more knowledgeable, more productive, and more efficient in solving your performance and scalability issues.
I wrote this book with some specific groups of readers in my mind. In deciding what material to include in this book and how to write it, I tried my best to make this book pertinent and useful for the following groups of readers:
Many books available today on the subject of software performance and scalability do not provide the same level of quantitativeness, which is one of the most distinctive merits of this book. In my opinion, quantitativeness is a requirement for dealing with software performance and scalability issues, as performance and scalability are quantitatively measurable attributes of a software system.
I hope that the quantitative approach and the real-world quantitative case studies presented throughout this book can help you learn about software performance and scalability faster and more effectively. And more importantly, I am confident that by applying everything you learn from this book to your product, you can make a huge difference in improving the performance and scalability of your product to the satisfaction of your company and customers.
HOW THIS BOOK IS ORGANIZED
Software Performance and Scalability: A Quantitative Approach is the first book to focus on software performance and scalability in a quantitative approach. It introduces the basic concepts and principles behind the physics of software performance and scalability from a practical point of view. It demonstrates how the performance and scalability of your products can be optimized and tuned using both proven theories and quantitative, real-world examples presented as case studies in each chapter. These case studies can easily be applied to your software projects so that you can realize immediate, measurable improvements on the performance and scalability of your products.
As illustrated in Figure A, this book elaborates on three levels of skill sets for coping with software performance and scalability problems.
Figure A Three levels of skill sets for solving software performance and scalability challenges.
Figure A Three levels of skill sets for solving software performance and scalability challenges.
Specifically, this book consists of the following three parts:
In order to make this book more suitable as a textbook for an upper division undergraduate or graduate level course for computer and software engineering students, exercises have been provided at the end of each chapter. In most cases, the exercises have been designed to encourage the reader to conduct his/her own research and come up with the quantitative solutions to the exercises. In addition, the reader is encouraged to think and practice, rather than simply writing a program or filling in a formula with numbers. Dealing with software performance and scalability problems is more challenging than simply coding, and oftentimes, it’s really passion and discipline that can make a difference.
I have made every effort to make this book concise, practical, interesting, and useful for helping you solve your software performance and scalability problems. I hope you’ll enjoy reading this book, apply what you learn from this book to your work, and see immediate positive results. In addition, be conscious that by developing high-performance and scalable software that consumes less electric power to run,you are not only contributing to the success of your company and your customers, but also helping reduce global warming effects, for which we are all responsible.
HOW TO REACH THE AUTHOR
All mistakes and errors, if any, in the text are my responsibility. You are more than welcome to email me your comments about the contents of this book, or errors found therein, at henry@perfmath.com For any downloads and updated information about the book, visit the book’s website at http://www.perfmath.com.
HENRY H. LIU, PHD
Folsom, California
September 2008
Acknowledgments
First, I would like to thank all of the colleagues I had worked with in the field of physics research. Some of the greatest physicists that I was so lucky to have had a chance to work for and with include: Professor S. Liu, Professor J. Xie, Dr. J. Le Duff, Professor Dr. A. Richter, Dr. J. Bisognano, Dr. G. Neil, and Dr. F. Dylla. Because of them, my previous physics research career had been so enjoyable and fruitful. I’d like to mention that my current career as a software performance engineer has benefited tremendously from my previous career as a physicist. Although I left physics research and jumped to computers and software about a decade ago, the spirit of pursuing a subject in a rigorous, quantitative, and objective manner cultivated through my earlier physics research career has never left me. I have had this hopelessly immutable habitude of trying to deal with every software performance issue as quantitatively as possible as if it were a physics research subject. I have been totally soaked with that spirit, which gives me the power and energy for pursuing every challenging software performance and scalability problem quantitatively, and thus this book—Software Performance and Scalability: A Quantitative Approach.
With my career as a software performance professional, I’d like to especially thank Pat Crain, who introduced me to applying queuing theory to solving software performance challenges. Pat also encouraged me to write my first research paper on software performance, which was presented and awarded the best paper award in the category of software performance at the 2004 CMG Conference held in Las Vegas. I owe a debt of gratitude to Keith Gordon, who was the VP of the software company I worked for. Keith had enthusiastically read draft versions of my papers prior to publication and had always encouraged me to publish and share my software performance experience with the greater software community. I also feel excited to mention one of my fellow software performance engineers, Mary Shun, who encouraged me to write a book on software performance someday. Many thanks and this is it, Mary!
Special thanks are also due to Engel Martin, one of the greatest software performance group managers I have ever worked for. While taking on a tremendous amount of managerial responsibilities, Engel has demonstrated an extremely sharp and accurate sense of software performance and scalability issues at high levels. The atmosphere Engel created within the group he manages has always made me feel comfortable to express my opinions freely and to make my own judgments objectively on technical issues, as I used to as a scientist, for which I am truly grateful.
I would like to take this opportunity to thank the nonanonymous yet never-met authors of some hundreds of books I bought in mathematics, physics, computers, and software. The books they wrote fed my knowledge-hungry mind at various stages of my careers.
I sincerely thank those anonymous referees who offered very useful opinions on how to make this book more valuable to the readers and more suitable as a textbook for an upper division undergraduate or graduate level course for students majoring in computers and software. Their very helpful suggestions have been incorporated in the various parts of this book.
I also owe thanks to Dr. Larry Bernstein, who kindly recommended my book proposal to Wiley. The structure of this book has been deeply influenced by his seminal works published by Wiley in the series of Quantitative Software Engineering.
Paul Petralia at Wiley-Interscience mentored me as a first-time book writer through each step of the entire process. It would have not been such a smooth process without Paul’s guidance and high professionalism. Michael Christian helped me achieve the next milestone for the book—to get it into production at Wiley—with his hard work and high efficiency. My production manager, Shirley Thomas, production editor, Christine Punzo, compositor (Techset Composition Ltd.), and illustration manager, Dean Gonzalez, at Wiley were the hard-working, efficient engines behind completing the last stage of publishing this book. Needless to say, without such a highly efficient and professional team at Wiley, my software performance experience accumulated over a decade would have still been scattered in various publications and work notes. So many thanks to everyone involved at Wiley.
I’d like to thank my wife, Sarah Chen, who sacrificed so much to take care of our newborn son, William, most of the time, in order to let me sit down and focus on writing this book using my weekends, nightly hours, and even vacation times.
You as a reader are greatly appreciated as well. Your interest in this book has shown your strong motivation to further the success of your company and also your willingness to help contain global warming by developing high-performance and highly scalable software that burns less electric power.
HENRY H. LIU
Introduction
All good things start with smart choices.
— Anonymous
PERFORMANCE VERSUS SCALABILITY
Before we start, I think I owe you an explanation about what the difference is between performance and scalability for a software system. In a word, performance and scalability are about the scalable performance for a software system.
You might find different explanations about performance versus scalability from other sources. In my opinion, performance and scalability for a software system differ from and correlate to each other as follows:
Unfortunately, there is no panacea for solving all software performance and scalability challenges. The best strategy is to start with the basics, being guided by queuing theory as well as by application programming interface (API) profiling when coping with software performance and scalability problems. This book teaches how one can make the most out of this strategy in a quantitative approach.
Let’s begin with the first part—the basics.
Part 1
The Basics
I went behind the scenes to look at the mechanism.
—Charles Babbage, 1791–1871, the father of computing
The factors that can critically impact the performance and scalability of a software system are abundant. The three factors that have the most impact on the performance and scalability of a software system are the raw capabilities of the underlying hardware platform, the maturity of the underlying software platform (mainly the operating system, various device interface drivers, the supporting virtual machine stack, the run-time environment, etc.), and its own design and implementation. If the software system is an application system built on some middleware systems such as various database servers, application servers, Web servers, and any other types of third-party components, then the performance and scalability of such middleware systems can directly affect the performance and scalability of the application system.
Understanding the performance and scalability of a software system qualitatively should begin with a solid understanding of all the performance bits built into the modern computer systems as well as all the performance and scalability implications associated with the various modern software platforms and architectures. Understanding the performance and scalability of a software system quantitatively calls for a test framework that can be depended upon to provide reliable information about the true performance and scalability of the software system in question. These ideas motivated me to select the following three chapters for this part:
The material presented in these three chapters is by no means the cliché you have heard again and again. I have filled in each chapter with real-world case studies so that you can actually feel the performance and scalability pitches associated with each case quantitatively.