Copyright © 2007 by John Wiley & Sons, Inc. All rights reserved.
Published by Jossey-Bass
A Wiley Imprint
989 Market Street, San Francisco, CA 94103-1741 www.josseybass.com
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-8600, or on the Web at www.copyright.com. Requests to the publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax 201-748-6008, or online at http://www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
Readers should be aware that Internet Web sites offered as citations and/or sources for further information may have changed or disappeared between the time this was written and when it is read.
Jossey-Bass books and products are available through most bookstores. To contact Jossey-Bass directly call our Customer Care Department within the U.S. at 800-956-7739, outside the U.S. at 317-572-3986, or fax 317-572-4002.
Jossey-Bass also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.
Library of Congress Cataloging-in-Publication Data
Miller, Barbara A., 1943-
Assessing organizational performance in higher education / Barbara A. Miller ; foreword by Suzanne Swope.
p. cm.
Includes bibliographical references and index.
ISBN-13: 978-0-7879-8640-7 (pbk.)
ISBN-10: 0-7879-8640-2 (pbk.)
1. Universities and colleges--United States--Evaluation.
2. Universities and colleges--United States--Administration. I. Title. LB2331.63.M55 2007
378.1'07--dc22
2006022688
TABLES
FIGURES
EXHIBITS
WORKSHEETS
Anyone interested in the survival of higher education realizes that the industry is going through a profound change. Just like manufacturing and health care before it, higher education must face the reality that costs, new technologies, and changing customer expectations create pressures on the industry. Anyone who works in colleges or has a stake in their success will find this book of great interest. Quality education in all its manifestations is crucial to the survival of democracy, as well as to the industry itself.
Peter Drucker, a longtime authority in management, proposed in Management Challenges for the 21st Century (1999) that we may need to stop thinking from a perspective of managing the work of people and begin managing for performance. To be effective, we must define customers’ values and their decision-making processes regarding their income distribution. Management must organize and evaluate the entire operational process, focusing on results and performance.
No one knows these principles better as they relate to higher education than Barbara A. Miller (formerly Lembcke). She has served as an administrative leader, teacher, researcher, and consultant in private and public universities. Her breadth of perspective and knowledge about systems—how they are defined, measured, evaluated, and changed—are extensive. Miller’s broadly based higher education background, combined with her teaching and administrative experience, makes her insights and analysis extremely valuable for those of us serving a variety of roles in the institution as well as those in evaluation positions as stakeholders outside the organization.
Assessing Organizational Performance in Higher Education embraces assessment at the organizational, program, and process levels and evaluates the work from a perspective rooted in systems thinking. Readers will be able to identify major work processes, the significance of these processes in producing quality outcomes, and the strategies necessary for continuous improvement. The book complements the body of literature on assessment, providing both an in-depth theoretical framework and techniques useful for implementation. The information in it is pertinent to everyone from the boardroom to the individual faculty or staff member and will serve as a set of tools to improve the work of the institution. Readers who fully understand the message Miller presents and who work through the exercises as they apply to the institution or program they are assessing will have done a great service to their constituencies—to the students whom they so gratefully serve and to others, both staff and faculty, who care about the quality of their work and the important role they play in this society.
Vice President for Enrollment and Student Affairs
Emerson College, Boston
I wrote this book to meet the needs of two important groups associated with assessment in higher education: assessors and assessment users. The first group, assessors, consists of persons engaged in day-to-day assessment work. They are faculty, staff, and administrators with part-time or full-time, temporary or permanent responsibilities for assessment. The second group, assessment users, are persons who evaluate or judge performance results measured and conveyed by assessors. I see assessment users as the end users or customers of assessment programs.
Assessors seek avenues for measuring performance required of assessment users; assessment users seek appropriate contexts for evaluating assessment findings measured and conveyed by assessors. Often assessors and assessment users are actually the same persons. However, I choose to differentiate the roles for purposes of discussion, assessor referring exclusively to persons exploring matters of measurement, and assessment user referring exclusively to persons engaged in evaluation. I describe various groups of external and internal assessment users and explain how each group uses assessment findings to support a wide range of decisions that have a potential impact on an organization’s capacity to perform.
My purpose in writing this book is to strengthen the knowledge, skills, and abilities of assessors and assessment users in higher education whether they are novices or experts. I define assessment as the measurement of organizational performance that assessment users evaluate in relation to reference points for the purpose of supporting their requirements and expectations.
The premise of this book is that assessors in higher education must go beyond assessment of student learning outcomes and institutional effectiveness and into assessment of performance of whole organizations, programs, and processes. This raises two questions: why? and how?
Why assess performance at the organization, program, and process levels? For a variety of reasons:
How is performance assessed at the organizational level?
The book’s focus on performance at the organization, program, and process levels complements and advances the many published works available today on assessment of student learning outcomes and institutional effectiveness. This focus helps readers understand the interdependence of organizations in higher education and complexities inherent in organizational performance. I believe that this understanding is fundamental to the practice and scholarship of assessment.
For assessors, the book offers a conceptual framework to guide the measurement of organizational performance in all seven areas of organizational performance. The conceptual framework applies to both academic and administrative units of analysis at any level within the hierarchical structure of educational institutions; it also applies to important programs and key work processes that operate within single organizations or across several organizations or functions within an institution.
What is most exciting about this book is its examination of assessment in several new and different areas of organizational performance—areas that include but go beyond institutional effectiveness, student learning outcomes, and input quality. The following are some of the new areas of performance that assessors can measure:
For external assessment users such as governing boards, governmental agencies, and organizations that affirm accreditation, classification, rank, and eligibility, the book is designed to expand knowledge of the nature and complexity of organizational performance in higher education—knowledge that will, ideally, enhance the ability to frame appropriate accountability questions of educational leaders.
For internal assessment users, such as senior leaders, administrators, and faculty and staff, the book is designed to expand knowledge of the internal workings and interdependence of organizations both inside and outside the institution, complexities inherent in organizational performance, and important links among organizational system elements, areas of organizational performance, and assessment. This knowledge will enhance their ability, as assessment users, to frame better performance questions that lead to better assessments of organizational performance.
Finally, the book offers educational leaders specific recommendations on how to build, deploy, and evaluate assessment programs in ways that provide the right information, at the right time, in the right format to meet ever-changing needs of important external and internal assessment users. The book presents many examples and worksheets to help assessors describe their unit’s organizational system elements and measure complex and interdependent areas of organizational performance using performance indicators and reference points appropriate to the organization’s mission, vision, strategic goals, and critical success factors.
The book is organized into six chapters. Chapter One describes external and internal assessment user groups in higher education. It explains what types of organizational performance results assessment users want to know, how they typically use assessment findings in their decision-making processes, and what is at stake for organizations whose performance is under review. A worksheet is provided to help assessors identify assessment information required of important external and internal assessment users groups.
Chapter Two introduces systems thinking and explains the benefits of viewing organizations as open, living, unique systems with a purpose. It begins with a discussion of interdependent system elements that make up organizations, programs, and processes in higher education and explains how each system element presents opportunities for assessment. Chapter Two describes five internal system elements: leadership systems, inputs, key work processes, outputs, and outcomes. Many examples are provided for academic and administrative organizations. Worksheets are also provided to help assessors identify and describe internal system elements of units whose performance they intend to measure.
Chapter Three continues the discussion of system elements and their link to assessment. It describes three external system elements: upstream systems, customers, and stakeholders. Again, many examples are provided for academic and administrative organizations. Worksheets are also provided to help assessors identify and describe external system elements of units whose performance they intend to measure.
Chapter Four is a discussion of how to assess organizational performance. It summarizes assessment methods and terminology. The chapter begins by differentiating the work of measurement from evaluation in assessment. It explains how to clarify units of analysis and the proper ways to use time frames, critical success factors, performance indicators, and reference points. It describes methods for collecting assessment data and disseminating performance results. Worksheets are provided to help assessors identify critical success factors and build an assessment report schedule for units whose performance they intend to measure.
Chapter Five is a discussion of what to assess in organizational performance. It covers the seven operational definitions of organizational performance noted earlier in this Preface: effectiveness, productivity, quality, customer and stakeholder satisfaction, efficiency, innovation, and financial durability. Many examples of performance indicators in each area are provided for academic and administrative organizations. Worksheets are provided to help assessors identify and describe performance indicators and reference points in all seven areas (including critical success factors) and link performance areas to specific assessment user needs and preferences.
Finally, Chapter Six is about how to build, deploy, and assess new or more formalized campuswide assessment programs. It offers suggestions about the importance of clarifying purpose, identifying important assessment user groups, and ensuring two-way, ongoing communication about assessment. It explains how to create and sustain a supportive organizational culture for assessment and how to build a leadership structure that ensures program success. It describes direct and indirect costs of assessment. It presents external and internal system elements of an assessment program as well as examples of indicators for measuring performance in areas deemed critical to program success. A worksheet is provided to help assessment leaders build an assessment communication plan.
This book reflects many years of work with friends and colleagues who helped me frame and apply this conceptual model for assessing performance of organizations in higher education. In particular, I would like to thank my husband and longtime friend and colleague, Louie Miller III, who not only served as my sounding board throughout the development of this book but also provided patient guidance and expertise resulting from his long and successful professional career as a tenured professor in sociology and senior executive in information services. I would also like to thank my friend and colleague Suzanne Swope, currently vice president for enrollment and student affairs at Emerson College, for her advice and collaboration over the many years we worked together at George Mason University. I would also like to thank my longtime friend Sandra Everett at Lorain County Community College for sharing her expertise in the area of quality management and helping me understand and apply those principles in the context of organizations in higher education. Finally, I would like to thank Scott Sink, Tom Tuttle, and Carl Thor, whose early works inspired the formation of this conceptual framework for assessing performance of organizations in higher education.
Greencastle, Indiana
June 2006
Barbara A. Miller (formerly Lembcke) is an experienced administrator in higher education and has served as a director of institutional planning and research, a senior planning and policy analyst, and an internal management consultant specializing in organizational development, and continuous quality improvement. She is also an experienced faculty member who has taught courses in management, leadership theory, organizational development, and communication. Her expertise in assessment results from thirty years of experience in large public research institutions, large and medium-sized two-year comprehensive community colleges, and small liberal arts institutions. She served for two years as an examiner for the Malcolm Baldrige National Quality Award Program and one year as an evaluator in the Baldrige pilot program in education, where documentation of performance results is critical.
Miller earned her bachelor of arts degree in sociology at the University of California, Berkeley; her master of arts degree in higher education and student personnel administration at Syracuse University; and her doctorate in higher education administration at the University of Florida in Gainesville. She has also taken M.B.A. courses at the University of North Florida, Jacksonville.
Miller lives in Greencastle, Indiana, where she serves as guest scholar at DePauw University and coordinates her consulting service.