THIRD EDITION
Copyright © 2018 by John Wiley & Sons, Inc. All rights reserved.
Portions of Chapter 15 were previously published by Routledge. Copyright © 2017 from Handbook on Measurement, Assessment, and Evaluation in Higher Education by Charles Secolsky and D. Brian Denison. Reproduced by permission of Taylor and Francis Group, LLC, a division of Informa plc.
Published by Jossey-Bass
A Wiley Brand
One Montgomery Street, Suite 1000, San Francisco, CA 94104-4594—www.josseybass.com
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-8600, or on the Web at www.copyright.com. Requests to the publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax 201-748-6008, or online at www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Readers should be aware that Internet Web sites offered as citations and/or sources for further information may have changed or disappeared between the time this was written and when it is read.
Jossey-Bass books and products are available through most bookstores. To contact Jossey-Bass directly call our Customer Care Department within the U.S. at 800-956-7739, outside the U.S. at 317-572-3986, or fax 317-572-4002.
Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.
Library of Congress Cataloging-in-Publication Data
Names: Suskie, Linda, author.
Title: Assessing student learning : a common sense guide / by Linda Suskie.
Description: Third edition. | San Francisco, CA : Jossey-Bass, [2018] | Includes bibliographical references and index.
Identifiers: LCCN 2017042567 | ISBN 9781119426868 (pdf) | ISBN 9781119426929 (epub) | ISBN 9781119426936 (pbk.)
Subjects: LCSH: College students—Rating of. | Educational tests and measurements.
Classification: LCC LB2336 .S87 2018 | DDC 378.1/662—dc23 LC record available at https://lccn.loc.gov/2017042567
Cover Design: Wiley
THIRD EDITION
To my husband Steve, for his unflagging support for everything I've done, including this book
To our children, Melissa and Michael
And to everyone in higher education who believes, as I do, that one of the answers to today's problems is for everyone to get the best possible education
When Jim Anker, the publisher of the first edition of this book, approached me about writing a second edition in 2008, I figured that I'd update the references and a few chapters and be done. The first edition was based, after all, on an enduring common sense approach to assessment that hadn't changed materially since the first edition was published in 2004. Ha! I ended up doing a complete reorganization and rewrite.
Fast-forward to 2017. With the second edition now eight years old, it was clearly time for an update. But the second edition had been very successful, so I again figured I'd update the references and a few chapters and be done. Ha! Once again, this is a complete reorganization and rewrite of the previous edition.
Why the rewrite? As I started work on this edition, I was immediately struck by how outdated the second edition had become in just a few short years. When I wrote the second edition, AAC&U's VALUE rubrics were largely untested, the National Institute for Learning Outcomes Assessment was just getting started, and the Degree Qualifications Profile didn't exist in the United States. Learning management systems and assessment information management systems were nowhere near as prevalent or sophisticated as they are today.
More broadly, the higher education community has largely moved from getting started with assessment to doing assessment. But a lot of assessment to date hasn't been done very well, so now we're starting to move from doing assessment to doing it meaningfully. Truly meaningful assessment remains a challenge, and this third edition aims to address that challenge in the following ways:
An increased emphasis on useful assessment. In earlier editions, I placed a chapter on using assessment results at the end, which made chronological sense. But many faculty and administrators still struggle to grasp that assessment is all about improving how we help students learn, not an end in itself, and that assessments should be planned with likely uses in mind. So I have added a second chapter on using assessment results to the beginning of the book. And throughout the book I talk not about “assessment results” but about “evidence of student learning,” which is what this is really all about.
Greater attention to building a culture in which assessment is useful and used. Getting colleagues on board remains a stubborn issue. Two chapters on this in the second edition have been expanded to six, including new chapters on guiding and coordinating assessment, helping everyone learn what to do, keeping assessment cost-effective, and making assessment collaborative.
An enhanced focus on the many settings of assessment, especially general education and co-curricula. Faculty and administrators are looking for more guidance on how to assess student learning in specific settings such as the classroom, general education curricula, undergraduate and graduate programs, and co-curricular experiences. The second edition provided little of this guidance and, indeed, did not draw many distinctions in assessment across these settings. A thorough treatment of assessment in each setting is beyond the scope of this book, of course. But this edition features a new chapter on the many settings of assessment, and several chapters now include discussions on applying the concepts in them to specific settings.
Call-out boxes to introduce assessment vocabulary. The jargon of assessment continues to put off faculty and staff as well as graduate students who use this as a textbook. I opened the second edition with a chapter that was essentially a glossary but overwhelmed graduate students. In my 2014 book, Five Dimensions of Quality: A Common Sense Guide to Accreditation and Accountability, I introduced higher education vocabulary with call-out boxes called Jargon Alerts. That feature was well received, so in this edition I've eliminated the glossary chapter and instead sprinkled Jargon Alert boxes throughout.
A new focus on synthesizing evidence of student learning into an overall picture of an integrated learning experience. Assessment committees and administrators today are often inundated with assessment reports from programs and units, and they struggle to integrate them into an overall picture of student learning, in part because learning itself is not yet an integrated experience. The idea that assessment should be part of an integrated learning experience is now a theme addressed throughout the book.
More immediate attention to learning goals. The second edition discussed learning goals about a third of the way through the book. Because learning goals are the drivers of meaningful assessment, the chapter on them now appears earlier in the book.
A new chapter on curriculum design. One of the major barriers to effective assessment and the use of student learning evidence is poor curriculum design, so I've added a whole new chapter on this.
More thorough information on planning assessment processes. There are now two chapters instead of one. The first one, on planning assessments of program learning goals, provides a framework, and the second one applies that framework to planning assessments in general education, co-curricula, and other settings.
New frameworks for rubric design and setting standards and targets. In 2016 I researched and wrote a chapter, “Rubric Development,” for the second edition of the Handbook on Assessment, Measurement, and Evaluation in Higher Education (Secolsky & Denison, 2017). My research changed my thinking on what an effective rubric looks like, how it should be developed, and how standards and targets should be set.
A new chapter on assessing the hard-to-assess. The former chapter on assessing attitudes and values is now two chapters – one on miscellaneous assessment tools and one on how to assess the hard-to-assess.
New resources. Many new assessment resources have emerged since the second edition was published, including books, models, published instruments, technologies, and research. Perhaps the most important new resources are the widely used VALUE rubrics published by the Association of American Colleges & Universities (www.aacu.org) and the many white papers published by the National Institute for Learning Outcomes Assessment (www.learningoutcomesassessment.org). This edition introduces readers to these and other valuable new resources. And, yes, I did update the references as I originally envisioned!
Interest in assessing student learning at colleges and universities – and the need to learn how to do it – skyrocketed in the last two decades of the twentieth century and continues to grow in the twenty-first century. The higher education community is increasingly committed to creating learning-centered environments in which faculty and staff work actively to help students learn, and the assessment of student learning is essential to understanding and gauging the success of these efforts. In the United States and elsewhere, accreditors and other quality assurance agencies require colleges and academic programs to assess how well students are achieving key learning goals. These trends have created a need for straightforward, sensible guidance on how to assess student learning.
Many years ago, someone commented on the value of my workshops to the “But how do we do it?” crowd. That phrase has stayed with me, and it is the root of this book. Yes, we in higher education are theorists and scholars, with an inherent interest in whys and wherefores, but there are times when all we need and want is simple, practical advice on how to do our jobs. Providing that advice is the purpose of this book.
Assessing Student Learning: A Common Sense Guide is designed to summarize current thinking on the practice of assessment in a comprehensive, accessible, and useful fashion for those without formal experience in assessing student learning. Short on background and theory and long on practical advice, this is a plainspoken, informally written book designed to provide sensible guidance on virtually all aspects of assessment to four audiences: Assessment newcomers, experienced assessment practitioners, faculty and administrators involved in student learning, and students in graduate courses on higher education assessment.
This book is called A Common Sense Guide because its premise is that effective assessment is based on simple, common sense principles. Because each college and learning experience is unique and therefore requires a somewhat unique approach to assessment, this book presents readers not with a prescriptive cookbook approach but with well-informed principles and options that they can select and adapt to their own circumstances.
This book is also based on common sense in that it recognizes that most faculty do not want to spend an excessive amount of time on assessment and are not interested in generating scholarly research from their assessment activities. The book therefore sets realistic rather than scholarly standards for good practice. It does not expect faculty to conduct extensive validation studies of the tests they write, for example, but it does expect faculty to take reasonable steps to ensure that their tests are of sufficient quality to generate fair and useful evidence of student learning, and it provides very practical suggestions on how to do that.
This book also minimizes the use of educational and psychometric jargon. For instance, while it discusses reliability and validity, it avoids using those terms as much as possible. Jargon Alert boxes are sprinkled throughout to help readers understand the vocabulary of higher education assessment.
This book is also unique in its comprehensive scope, although it is not (as my husband reminded me when I was in despair over ever finishing the first edition) an encyclopedia. If you'd like to learn more, every chapter cites additional resources to explore.
Assessment in higher education is still a nascent discipline. The science of educational testing and measurement is little more than a century old, and many of the ideas and concepts presented here have been developed only within the last few decades. Assessment scholars and practitioners still lack a common vocabulary or a widely accepted definition of what constitutes good or best assessment practices. As a result, a few may disagree with some of the ideas expressed here. As you hear conflicting ideas, use your own best judgment – your common sense, if you will – to decide what's best for your situation.
Assessment newcomers who want to gain a general understanding of all aspects of assessment will find that the book's five parts take them sequentially through the assessment process: Understanding assessment, planning the assessment process, getting everyone on board, choosing and developing appropriate tools, and understanding and using student learning evidence.
More experienced assessment practitioners will find the book a helpful reference guide. Plenty of headings, lists, tables, and cross-references, along with a thorough index, will help them find answers quickly to whatever questions they have.
Anyone involved in student learning, including faculty who simply want to improve assessments within their classes or student development staff who want to improve assessments in their co-curricular experiences, will find much of the book of interest. Table 2.1 in Chapter 2 suggests especially relevant chapters for various assessment settings.
Faculty and staff teaching professional development workshops and graduate courses in assessment will find this book a useful textbook or resource. Each chapter concludes with questions and exercises for thought, discussion, and practice. No answer key is provided, because these are mostly complex questions with no simple answers! Often the conversation leading to the answers will reinforce learning more than the answers themselves.
Some of the material in this book is adapted from my earlier book, Questionnaire Survey Research: What Works (Suskie, 1996), published by the Association for Institutional Research. I am grateful to the Association for permission to adapt this material. Some material in Chapter 15 is adapted from my book chapter “Rubric Development” in the Handbook on Measurement, Assessment, and Evaluation in Higher Education (Secolsky & Denison, 2017), by permission of Taylor and Francis Group, LLC, a division of Informa plc. I am grateful to Taylor and Francis for permission to adapt this material. I also thank my daughter Melissa for writing the deliberately less-than-sterling essay in Exhibit 15.3.
This book would not be in your hands without the work, support, and input of many people, including assessment practitioners and scholars, faculty and staff. Over the last 15 years, I have worked with literally thousands of faculty and staff at colleges and universities across the United States and throughout the world. Their questions, comments, thoughts, and ideas have pushed me to research and reflect on issues beyond those in the second edition, and this led to much of the new material in this edition. When I was contemplating this third edition, I emailed several hundred colleagues for their thoughts and ideas on a new edition, and dozens responded with extraordinarily thoughtful input. Assessment people are the nicest, friendliest, and most supportive people in the world! I particularly want to acknowledge, with deep gratitude, Elizabeth Barkley, Cynthia Howell, Claire Major, and Susan Wood, who reviewed drafts of the entire manuscript and offered wise counsel and suggestions.
Altogether I am incredibly grateful to these unsung colleagues for their willingness to share so much with me.
Linda Suskie is an internationally recognized consultant, writer, speaker, and educator on a broad variety of higher education assessment and accreditation topics. Her most recent book is Five Dimensions of Quality: A Common Sense Guide to Accreditation and Accountability (2014). Her experience includes serving as a Vice President at the Middle States Commission on Higher Education, Associate Vice President for Assessment & Institutional Research at Towson University, and Director of the American Association for Higher Education's Assessment Forum. Her more than 40 years of experience in higher education administration include work in accreditation, assessment, institutional research, strategic planning, and quality management, and she has been active in numerous professional organizations.
Linda has taught graduate courses in assessment and educational research methods, as well as undergraduate courses in writing, statistics, and developmental mathematics. She holds a bachelor's degree in quantitative studies from Johns Hopkins University and a master's in educational measurement and statistics from the University of Iowa.
While the term assessment can be used broadly – we can assess the achievement of any goal or outcome – in this book, the term generally refers to the assessment of student learning. Many assessment practitioners have put forth definitions of student learning assessment, but the best one I've heard is in the Jargon Alert box. It's from Dr. Jane Wolfson, a professor of biological sciences at Towson University (personal communication, n.d.). It suggests that student learning assessment has three fundamental traits.
Assessment is part of a four-step process of helping students learn (List 1.1). These four steps do not represent a one-and-done process but a continuous four-step cycle (Figure 1.1). In the fourth step, evidence of student learning is used to review and possibly revise approaches to the other three steps (see Jargon Alert on closing the loop), and the cycle begins anew.
Figure 1.1: Teaching, Learning, and Assessment as a Continuous Four-Step Cycle
If the cycle in Figure 1.1 looks familiar to you, it's the Plan-Do-Check-Act cycle of business quality improvement popularized by Deming (2000): Plan a process, do or carry out the process, check how well the process is working, and act on the information obtained during the Check step to decide on improvements to the process, as appropriate.
Faculty have been assessing student learning for centuries, often through written and oral examinations. How do today's approaches to assessment differ from traditional approaches? Table 1.1 summarizes some key differences between traditional and contemporary ways of thinking about assessment.
Table 1.1: Traditional Versus Contemporary Ways of Thinking About Assessment
| Traditional Approaches: Assessment is. . . | Contemporary Approaches: Assessment is. . . |
| Planned and implemented without consideration of learning goals, if any even exist | Carefully aligned with learning goals: The most important things we want students to learn (Chapter 4) |
| Often focused on memorized knowledge | Focused on thinking and performance skills (Chapter 4) |
| Often poor quality, simply because faculty and staff have had few formal opportunities to learn how to design and use effective assessment strategies and tools | Developed from research and best practices on teaching and assessment methodologies (Chapters 3 and 26) |
| Used only to assess and grade individual students, with decisions about changes to curricula and pedagogies often based on hunches and anecdotes rather than solid evidence | Used to improve teaching, learning, and student success as well as to assign grades and otherwise assess individual students (Chapters 6 and 26) |
| Used only in individual course sections; not connected to anything else | Viewed as part of an integrated, collaborative learning experience (Chapter 2) |
| Not used to tell the story of our successes; stories are told through anecdotes about star students rather than broader evidence from representative students | Used to tell our story: What makes our college or program distinctive and how successful we are in meeting societal and student needs (Chapter 25) |
Obviously there is a great deal of overlap between the tasks of grading and assessment, as both aim to identify what students have learned. There are two key differences, however. The first is that the grading process is usually isolated, involving only an individual faculty member and an individual student. Assessment, in contrast, focuses on entire cohorts of students, and it often considers how effectively many people, not just an individual faculty member, are collectively helping them learn.
The second difference between grading and assessment is that they have different purposes. The main purpose of grades is to give feedback to individual students, while assessment has three broader purposes discussed in Chapter 6: Ensuring and improving educational quality, stewardship, and accountability. Grades alone are usually insufficient to achieve these purposes for several reasons.
Grades alone do not usually provide meaningful information on exactly what students have and haven't learned. We can conclude from a grade of B in an organic chemistry course, for example, that the student has probably learned a good deal about organic chemistry. But that grade alone cannot tell us exactly what aspects of organic chemistry she has and has not mastered.
Grading and assessment criteria may differ. Some faculty base grades not only on evidence of what students have learned, such as tests, papers, presentations, and projects, but also on student behaviors that may or may not be related to course learning goals. Some faculty, for example, count class attendance toward a final course grade, even though students with poor attendance might nonetheless master course learning goals. Others count class participation toward the final grade, even though oral communication skills aren't a course learning goal. Some faculty downgrade assignments that are turned in late. Under these grading practices, students who do not achieve major learning goals might nonetheless earn a fairly high grade by playing by the rules and fulfilling other less-important grading criteria. Conversely, students who achieve a course's major learning goals might nonetheless earn a poor grade if they fail to do the other things expected of them. To better sync grading and assessment criteria, add a professionalism learning goal (Chapter 4) or develop a competency-based curriculum (Chapter 5).
Grading standards may be vague or inconsistent. While many faculty base assignment and course grades on carefully conceived learning goals and standards, others may base grades on inconsistent, imprecise, and idiosyncratic criteria. Faculty may say they want students to learn how to think critically, for example, but base grades largely on tests emphasizing factual recall. Faculty teaching sections of the same course may not agree on common standards and may therefore award different grades to similar student performance. Sometimes individual grading standards are so vague that a faculty member might, in theory, award an A to a student's work one day and a B to identical work a week later.
Grades do not reflect all learning experiences. Grades give us information on student performance in individual courses or course assignments (Association of American Colleges & Universities, 2002), but they do not provide information on how well students have learned key competencies, such as critical thinking or writing skills, over an entire program. Grades also do not tell us what students have learned from ungraded co-curricular experiences.
Do grades have a place in an assessment effort? Of course they do! Grades can be useful, albeit indirect (Chapter 3), and therefore insufficient evidence of student learning. Although grades are often too holistic to yield useful information on strengths and weaknesses in student learning, they can be a good starting point for identifying potential areas of concern. DFIW rates – the proportions of students earning a D, F, Incomplete, or Withdrawal in a course – can identify potential barriers to student success.
Grades can be especially useful if courses, assignments, and learning activities are purposefully designed to help students achieve key learning goals (Chapters 5 and 16) by using tools such as test blueprints (Chapter 17) or rubrics (Chapter 15).
Assessment, while a cousin of scholarly research, differs in its purpose and therefore in its nature (Upcraft & Schuh, 2002). Traditional scholarly research is commonly conducted to test theories, while assessment is a form of action research (see Jargon Alert) conducted to inform one's own practice – a craft-based rather than scientific approach (Ewell, 2002). The four-step teaching-learning-assessment cycle of establishing learning goals, providing learning opportunities, assessing student learning, and using evidence of student learning mirrors the four steps of action research: Plan, act, observe, and reflect.
Assessment, like any other form of action research, is disciplined and systematic and uses many of the methodologies of traditional research. But most faculty and staff lack the time and resources to design and conduct rigorous, replicable empirical research studies of student learning. They instead aim to keep the benefits of assessment in proportion to the time and resources devoted to them (Chapter 12). If you design your assessments reasonably well and collect corroborating evidence (Chapter 21), your evidence of student learning may be imperfect but will nonetheless give you information that you will be able to use with confidence to make decisions about teaching and learning.
Is assessment a synonym for evaluation? It depends on the definition of evaluation that is used.
Evaluation may be defined as using assessment information to make an informed judgment on matters such as whether students have achieved the learning goals we've established for them, the relative strengths and weaknesses of teaching strategies, or what changes in learning goals and teaching strategies are appropriate. Under this definition, evaluation is the last two steps of the teaching-learning-assessment process: Interpreting student learning evidence (part of Step 3) and using it (Step 4). This definition points out that student learning evidence alone only guides us; it does not dictate decisions to us. We use our best professional judgment to make appropriate decisions. This definition of evaluation thus reinforces the ownership that faculty and staff have over the assessment process.
Evaluation may be defined as determining the match between intended and actual outcomes. Under this definition, evaluation is virtually synonymous with the third step of the teaching-learning-assessment cycle.
Evaluation may be defined as investigating and judging the quality or worth of a program, project, or other endeavor. This defines evaluation more broadly than assessment. We might evaluate an employee safety program, an alumni program, or a civic project designed to reduce criminal recidivism. While assessment focuses on how well student learning goals are achieved, evaluation addresses how well all the major goals of a program are achieved. An anthropology program, for example, might have goals not only for student learning but also to conduct anthropological research, provide anthropological services to local museums, and conduct its affairs in a cost-effective manner. An evaluation of the program would consider not only student learning but also research activities, community service, and cost-effectiveness.
Just as assessment and evaluation of student learning are sometimes considered synonymous, so are assessment and measurement of student learning. But many people have a relatively narrow conception of measurement, thinking of it as placing something on a quantitative scale akin to a yardstick. This book avoids the term measurement, because assessment is much broader than this conception. Assessment may generate qualitative as well as quantitative evidence of student learning (Chapter 20); it may generate categorical evidence as well as evidence that can be placed on a scale (Chapter 23); and it does not have the precision that images like a yardstick imply (Chapter 24).