Cover Page

ASHE Higher Education Report: Volume 43, Number 5

Learning Analytics in Higher Education


Jaime Lester,

Carrie Klein,

Huzefa Rangwala,

Aditya Johri

Advisory Board

images

The ASHE Higher Education Report Series is sponsored by the Association for the Study of Higher Education (ASHE), which provides an editorial advisory board of ASHE members.

Amy Bergerson

University of Utah

Bryan Brayboy

Arizona State University

Ryan Gildersleeve

University of Denver

Michael Harris

Southern Methodist University

Elizabeth Jones

Holy Family University

Adrianna Kezar

University of Southern California

Kevin Kinser

SUNY – Albany

Peter Magolda

Miami University of Ohio

Dina C. Maramba

SUNY – Binghamton

Susan Marine

Merrimack College

Christopher Morphew

University of Iowa

Robert Palmer

SUNY – Binghamton

Michael Paulsen

University of Iowa

Todd Ream

Taylor University

Barbara Tobolowsky

University of Texas at Arlington

Carolyn Thompson

University of Missouri, Kansas City

Diane Wright

Florida Atlantic University

Executive Summary

IN 2003, 5 BILLION gigabytes of data had been collected since the beginning of recorded history; today, 5 billion gigabytes of data can be collected in 10 seconds (Zwitter, 2014, p. 2). The Internet is full of facts on how much data are created daily and projections on how much will be collected in the future. Whether the numbers are entirely correct, industrialized countries are now in the era of big data, which are often defined by three characteristics: volume, variety, and velocity (Laney, 2001). Combined, these three characteristics indicate that big data have a high degree of volume (large in size), are real time or timely, and contain a variety of measures (DeMauro, Greco, & Grimaldi, 2016; Ylijoki & Porras, 2016). More succinctly stated, “Big data is a generic term that assumes that the information or database system(s) used as the main storage facility is capable of storing large quantities of data longitudinally and down to very specific transactions” (Picciano, 2012, p. 12). In the context of higher education, big data are ubiquitous in the form of student transcripts that contain course-level information, student college application data (in other words, SAT and ACT test scores, high school grade point averages and location), data on wireless Internet access, interactions with learning management systems (LMS), and, more recently, when students swipe their student identification cards for meals or access to buildings.

Learning analytics has evolved in education alongside the explosion of the big data revolution as a specific form of educational data mining. Although there is not a uniformly accepted definition of learning analytics, multiple sources tend to have the similar elements of statistical analysis, prediction, and requirements of large (commonly referred to as big) data. For the purposes of this monograph, we adopt the definition of the “measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Siemens & Gašević, 2012, p. 1). Essentially, learning analytics is the form of educational data mining that performs predictive analysis on big data with the intention of creating platforms for intervention. Learning analytics can also involve exploratory analysis that leads to the generation of new hypotheses associated with learning behaviors and habits.

Learning analytics, by their volume, timeliness, and composition, “expands the capacity and ability of organizations to make sense of complex environments” and promises to improve pedagogy, course design, student retention, and decision making by providing personalized feedback for users (Ali, Hatala, Gašević, & Jovanović, 2012; Macfadyen & Dawson, 2012; Norris & Baer, 2013, p. 13). This promise is alluring to higher education institutions, which are facing increasing pressure to provide evidence of student learning in an environment in which teaching pedagogical best practices are moving to an increasingly individualized and student-focused learning model and in which innovative technologies are allowing for greater mining of student data (Austin & Sorcinelli, 2013). Within this context, learning and advising management systems, based on educational big data, or learning analytics, are being developed to better measure, analyze, report, and predict data related to student learning, retention, and completion. These learning analytics-informed systems have the potential to generate new insight into courses and student learning by creating responsive feedback mechanisms that can shape data-informed decision making as it relates to teaching, learning, and advising.

Given the potential and increasing presence of learning analytics in higher education, it is important to understand what it is, what associated barriers and opportunities exist, and how it can be used to improve organizational and individual practices, including strategic planning, course development, teaching pedagogy, and student advising. The purpose of this monograph is to give readers a practical and theoretical foundation in learning analytics in higher education, including an understanding of the challenges and incentives that are present in the institution, in the individual, and in the technologies themselves.

Among questions that are explored and answered are:

  1. What are the current trends in higher education that are driving a need for learning analytics tools?

  2. What role do institutional context, technological capacity, and individual beliefs play in promoting or constraining adoption and integration of learning analytics technologies in higher education?

  3. What are the ethical considerations related to use of learning analytics or other predictive data and associated interventions?

  4. What are the practical implications and future research recommendations associated with learning analytics?

Organized into five chapters, this monograph is intended to serve as an introduction to learning analytics for those practitioners and researchers who are interested in learning more about the development, implementation, and promise of harnessing educational big data with predictive methods. We also complicate learning analytics in higher education by drawing attention to the complex ethical and privacy issues surrounding the collection and dissemination of such data. Although the issues are far from simple, there are considerations and questions that can guide development and practical use of learning analytics tools.

Learning analytics as a field is new and emerging. The major association Learning Analytics & Knowledge has existed for less than 10 years; theoretical frameworks and research literature are just now beginning to emerge in large quantities. As with all new fields, learning analytics has drawn from a number of multidisciplinary trends and literatures to examine different facets of use, design, and implementation but has yet to bring together the complexity of external and internal organizational factors; faculty, advisor, and student motivation to use learning analytics; and ethics and privacy concerns. This monograph draws from several areas of research—organizational theory, technology adoption, faculty beliefs and behaviors, and ethics and privacy—in a comprehensive model of learning analytics in higher education. Our model conceptualizes adoption of learning analytics in higher education as being done within the context of organizational factors (for example, infrastructure, change readiness, and so on) with ethics and privacy underlying all other areas; meaning, ethics, and privacy should be the guidepost for all decision making regarding learning analytics. The purpose of this model is to identify the complex issues surrounding adoption of learning analytics in higher education that is often noted as a challenge in the literature that takes into account the organizational, technological, individual, and ethics literature.

The first chapter provides an overview of the monograph and of the issues related to use of learning analytics in higher education, including information on what learning analytics is, the environmental context that has contributed to the emergence and evolution of the use of learning analytics in higher education, how analytics are currently being used in higher education, and some of the unique challenges and opportunities learning analytics systems face in higher education settings. In this chapter, we present the framework for learning analytics in higher education with a brief overview of each tenet of the model. Subsequent chapters provide extensive review of the literature and discussion of the model. The chapter concludes with an introduction to the structure and purpose of the remaining chapters.

The second chapter focuses on organizational aspects of the learning analytics in higher education model with a brief review of the literature on organizational change, institutional logics, and capacity and readiness related to learning analytics tools in higher education. We argue that organizational factors that create barriers and opportunities for learning analytics implementation and adoption in higher education are rooted in issues of institutional structures, commitment, resources, readiness, and capacity and a lack of incentives and rewards (Arnold, Lonn, & Pistilli, 2014; Austin, 2011; Bichsel, 2012; Kezar & Lester, 2009; Macfadyen & Dawson, 2012; Norris & Baer, 2013). Implementation of learning analytics also requires attention to a host of technological factors including provision of data, technical data analytics expertise, cross-organization collaboration, leadership, and attention to organizational climate (Arnold et al., 2014; Bichsel, 2012; Ferguson, 2012; Klein, Lester, Rangwala, & Johri, in press; Norris & Baer, 2013). The chapter concludes with an overview of technological aspects of learning analytics tools and individual decision making, including a review of innovation adoption and tool alignment.

In the third chapter, we focus on the aspects of individual decision making that exist within that context for faculty, advisors, and students who are increasingly interacting with learning analytics, whether or not they are aware of it. Data from learning management systems (LMS) (for example, Blackboard and Moodle) are being mined and incorporated into learning analytics algorithms that provide data visualizations and performance feedback related to teaching, advising, and course performance. LMS and learning analytics tools are examples of changing pedagogical innovations that have been deployed and leveraged as a way to improve institutional and individual decision making (Bichsel, 2012; Dahlstrom, Brooks, & Bichsel, 2014; Macfadyen & Dawson, 2012). However, these tools are useful to higher education only if individuals decide to adopt them. Engaging the theoretical models and research on faculty pedagogy change (Austin, 2011) and the work on learning analytics and student behaviors (Arnold & Pistilli, 2012), we argue that for faculty, advisors, and students the decision to engage in these tools is rooted in professional identity, beliefs, and behaviors and through learning analytics visualizations.

Emerging as a major consideration of learning analytics use in higher education are issues of ethics and privacy. The fourth chapter explores the challenges associated in creating analytics-based technologies as they relate to establishing an ethics of care and consent, respecting and maintaining privacy, and safeguarding against algorithmic bias and data insecurity, including an overview of ethical and privacy guidelines, laws, and policies; the choices related to including specific data points in learning analytics algorithms (especially demographic-based data); and the use of those data to predict student outcomes. Further, we review the privacy concerns related to collection, use, and ownership of student faculty and staff data and issues related to individual agency in an age of educational data mining.

In the final chapter of the monograph, we engage the framework proposed in the first chapter to look at how some of the issues associated with learning analytics in higher education can be mitigated and to consider the directions in which learning analytics needs to move in order for it to be transformational. The solutions, we believe, lie in thinking through the complexities of individual decision making, pedagogical change, organizational policies and practices, and data access, ethics, and privacy. Before any of these issues are a consideration, data must be available and of a high quality to build the tools. Simply, learning analytics tools cannot exist without data. And, like other issues, data come with their own set of complexities. We explore issues and provide specific recommendations on data and data use in learning analytics, such as data use and availability; importance of design thinking; and personalization in data visualization. The chapter concludes with suggestions for future research.

Acknowledgments

WE WOULD LIKE to thank Lisa Wolf-Wendel and Kelly Ward for the opportunity to contribute to the ASHE Higher Education Report series through the work of this monograph. Our research that is referenced in this monograph is supported in part by a grant from the National Science Foundation under grant IIS-1447489.

Foreword

BIG DATA ARE big news. One need not look far to see news reports on how institutions around the globe are mining data to help improve institutional functions. Higher education too is jumping on the “big data” bandwagon, actively working on how to make use of faculty and student data to improve outcomes. The topic is important and timely. As such, it is with great pleasure that I present this monograph on Learning Analytics in Higher Education by Jaime Lester, Carrie Klein, Huzefa Rangwala, and Aditya Johri as part of the ASHE Higher Education Report series.

Learning analytics (LA)—the use of educational “big data” to analyze and predict student learning and success—holds great promise for higher education. This promise, however, has yet to be fully realized because we haven't fully tapped into its potential and figured out how to harness it to truly help students. The present monograph explores these and related issues—explaining what LA are, how they work, the associated barriers and opportunities that LA provides, and how it can be harnessed to improve student learning. The monograph offers practical and theoretical understanding of learning analytics, building on the small but growing empirical literature that is available on the subject.

This monograph is sure to be of interest to those who study topics related to student outcomes, assessment, institutional research, and institutional effectiveness. This monograph will be of interest to institutional researchers, student affairs administrators, provosts, deans, and others with responsibilities related to the assessment of student outcomes. Researchers in the field, both senior level and graduate students, are also bound to learn a lot from this monograph that will be of use. Most important, the monograph is geared toward faculty members and advisors who find themselves on the frontlines of implementing, adopting, and integrating LA into their work with students.

As the monograph explains, “Learning analytics provide personalized, real-time, actionable feedback through mining and analysis of large data sets, which can illuminate trends and predict future outcomes that may not be visible via smaller data sets.” The authors caution, however, that the adoption of learning analytics tools is expensive and fraught with challenges—including how to make the data meaningful to those who need to use it most (that is, decision makers, faculty members, advisors, and students). The purpose of this monograph is to delve into the research, literature, and issues associated with learning analytics implementation, adoption, and use by individuals within higher education institutions. Through the use of vignettes and a summary of relevant research and theory, the authors clearly outline what is happening with regard to LA in institutions of higher education, its future potential in the field, along with an important consideration of ethical and privacy concepts and concerns. This is a “must read” for everyone in the field. Big data are here to stay—so we had best figure out how to use them in a thoughtful manner or they will do more harm than good. This monograph helps readers work their way through the complexities of the issues and figure out practical next and future steps.