Cover Page

Using Data-Informed Decision Making to Improve Student Affairs Practice


Kathleen M. Goodman
Darnell Cole

EDITORS





Number 159 • Autumn 2017

Jossey-Bass

San Francisco

Editors’ Notes

Assessment in higher education, which was once considered optional, has become the norm. Departments and offices dedicated to student success are also becoming common on college campuses. Across universities, student affairs professionals and academic administrators are looking for ways to use data to improve the quality of students’ educational experience. Many campuses collect large amounts of survey data each year, relying on big and midsize research entities to collect and summarize their quantitative data. This volume focuses on using that data to make informed decisions about student affairs practice.

The National Survey of Student Engagement (NSSE) and the Cooperative Institutional Research Program (CIRP) Freshman Survey are probably the most widely used surveys on college campuses, but other studies are widespread as well. The Multi-Institutional Study of Leadership (MSL), the Campus Religious and Spiritual Climate Survey (CRSCS), and the Campus Pride Index (CPI) provide data to a significant number of campuses. Table EN1 briefly describes national surveys commonly used by college campuses that are referred to in one or more chapters in this volume.

Table EN1 Brief Descriptions of Common National Surveys Used by College Campuses

Survey Topics Organization
CIRP Freshman Survey Students’ background characteristics, high school experiences, attitudes, behaviors, and expectations for college Cooperative Institutional Research Program https://heri.ucla.edu/overview-of-surveys/
Your First College Year (YFCY) Students’ academic and personal development during the first year of college Cooperative Institutional Research Program https://heri.ucla.edu/overview-of-surveys/
Beginning College Survey of Student Engagement (BCSSE) High school academic and cocurricular experiences, as well as students’ expectations about the educational activities in which they will participate during their first year of college Center for Postsecondary Research http://cpr.indiana.edu/index.cfm
National Survey of Student Engagement (NSSE) Students’ academic experiences and engagement; includes questions such as the number of term papers they wrote for courses, the level of academic challenge they perceive in courses, and so on Center for Postsecondary Research http://cpr.indiana.edu/index.cfm
Multi-Institutional Study of Leadership (MLS) Focused on understanding the influences of higher education in shaping socially responsible leadership capacity and other leadership-related outcomes such as efficacy, cognitive skills, and resiliency Multi-Institutional Study of Leadership https://www.leadershipstudy.net/
Campus Pride Index (CPI) Academic experiences, quality of campus life, and policies related to LGBTQ students Campus Pride https://www.campusprideindex.org/
Interfaith Diversity Experiences and Attitudes Longitudinal Survey (IDEALS) Student experiences and attitudes about religious, nonreligious, and spiritual diversity while in college Interfaith Youth Core https://www.ifyc.org

Additional data often available on campuses come from surveys that focus on institutional type, such as those administered by the Council for Christian Colleges surveys on behalf of member institutions and the Student Experience in the Research University (SERU) survey, which provides data to a collaborative of top-ranked research universities. Add to the mix college outcomes surveys, such as ACT's College Assessment of Academic Proficiency (CAAP) Critical Thinking Test and the Collegiate Learning Assessment; functional area surveys for those who work in, for example, residence life, admissions, and student activities; and the information that institutions collect for their own assessments. It seems clear that most campuses are swimming (maybe even drowning!) in data. Surely, then, institutions must be in a constant state of improvement and increased student success, no matter how the individual campus defines it. Unfortunately, this is often not the case.

Although a great amount of data are available on college campuses, they are rarely used to the extent that they could be. A common trope paints research reports as binders sitting on the shelves in the office of institutional research collecting dust. A more fitting image for today may be research data floating on cyber clouds. There are many obstacles to putting all of these data to good use: overworked staff (and/or understaffed departments), silos that prevent looking across institutional boundaries, lack of research training, or simply being overwhelmed by the possibilities. This volume seeks to address some of these obstacles by providing pragmatic ideas for implementing data-informed decision making to student affairs practitioners who want to improve student learning, student engagement, or the campus climate for diversity.

Purpose and Themes

The purpose of this volume is twofold. First, we demonstrate that advanced research knowledge is not necessary to make meaning of survey findings by illustrating how to analyze quantitative data and read assessment reports. Second, we provide suggestions for utilizing findings from large data sets typically available on campus; the chapters provide practical guidance for making sense of and using quantitative data to inform practice.

A common theme among several chapters is that analyzing data does not have to be a complex process. Many individuals seem to avoid analyzing quantitative data because they fear they do not have the necessary statistical skills or they are overwhelmed by the sheer quantity of data. Yet as the chapters in this volume illustrate, focusing on one or two items closely related to the mission or purpose of your office can be sufficient, and the national surveys provide some basic analysis for you. Several chapters provide examples of how to make meaning from the reports you receive (and not a single one suggests that you must run a complex statistical analysis!).

A second common theme across several chapters is using data to understand the experiences of nondominant populations on campus, which is especially relevant given the diversity of today's college students. Several chapters speak directly to using data to understand traditionally marginalized groups based on race, religion, or sexual orientation, while others focus on using data to understand diversity experiences. Another chapter focuses on helping students develop socially responsible leadership values, which can help students learn to interact with individuals different from themselves and inspire social change. Other chapters mention the need to disaggregate data (look at the data broken out by groups such as race or religion) in order to highlight the experiences of marginalized populations, which otherwise are masked or hidden when only viewing aggregate data.

Content of the Sourcebook

The first two chapters of this volume provide general guidance for using data, whether they come from large data sets or elsewhere. In Chapter 1, Kathleen Goodman and Buffy Stoll Turton make the case that “good data” are any data that are used to improve practice, and one doesn't have to be a research expert to use data. In Chapter 2, Charles Blaich and Kathleen Wise suggest that the best way to understand data is to have “sensemaking” conversations with a broad range of constituents focused on small chunks of data.

The next four chapters provide an overview of commonly used data sets that are available on many campuses. In Chapter 3, Jillian Kinzie and Sarah Hurtado describe how to use the most current version of the NSSE to understand diversity on campus. Matthew Johnson and Gretta Mincer share examples of using the Multi-Institutional Study of Leadership (MSL) in Chapter 4. Using the Campus Pride Index (CPI) to create more inclusive environments for LGBT students is described in Chapter 5 by Jason Garvey, Susan Rankin, Genny Beemyn, and Shane Windmeyer. In Chapter 6, Rebecca Crandall, Benjamin Correia-Harker, Matthew Mayhew, and Alyssa Rockenbach provide examples of using the Campus Religious and Spiritual Climate Survey (CRSCS) and the Interfaith Diversity Experiences and Attitudes Longitudinal Survey (IDEALS) to understand and improve the campus climate for religious/worldview diversity.

The final two chapters focus on important aspects of the campus environment rather than specific surveys. Chapter 7, written by Melora Sundt, Sharla Berry, and Adam Ortiz, provides advice for using data to support online student communities. Chapter 8, written by Melora Sundt, Darnell Cole, and Marissiko Wheaton, provides suggestions for using data to understand students’ diversity experiences.

A Note About Terminology

This volume is focused on assessment, yet we have frequently used the word research, which some might find confusing. To us, the primary distinction is that assessment reflects data from one institution used to evaluate and improve practice, whereas quantitative research reflects data from multiple institutions used to generalize findings about practice or theory. Yet the data collection and analysis skills used in both are called research skills. And the national surveys discussed throughout this volume are typically administered by research organizations such as CIRP or the Center for Postsecondary Research. These organizations provide individualized institutional data to colleges and universities (some might call this assessment data) in reports that compare the data to other institutions (this is called benchmarking, which is a particular type of assessment), and they also craft research articles and reports based on data from multiple institutions.

Fundamentally, the line between assessment and research is more blurred than most textbooks or experts would have you believe, and the more important concern is what you do with your data. Thus, the astute reader will notice that we have cleverly avoided the words research or assessment in the title of this volume and have focused on what we believe is the utmost concern: Using Data-Informed Decision Making to Improve Student Affairs Practice.

Kathleen M. Goodman
Darnell Cole
Editors