Cover Page

Using Qualitative Research to Promote Organizational Intelligence

Ezekiel Kimball

Karla I. Loya

EDITORS





Number 174

Jossey-Bass

San Francisco






This issue is dedicated to the memory of NDIR Editor, Dr. John Ryan.









THE ASSOCIATION FOR INSTITUTIONAL RESEARCH (AIR) is the world's largest professional association for institutional researchers. The organization provides educational resources, best practices, and professional development opportunities for more than 4,000 members. Its primary purpose is to support members in the process of collecting, analyzing, and converting data into information that supports decision‐making in higher education.

Editors' Notes

In 2016, the Association for Institutional Research (AIR) released its Statement of Aspirational Practice for Institutional Research (Swing & Ross, 2016a, 2016b). The Statement represents the efforts of the institution (AIR) to respond to the changing needs of postsecondary institutions. Its creation involved more than a year-long process and included an open call for ideas, crafting by six “subject matter experts” (2016b, p. 7), pilot testing at 10 institutions who vetted the statement, and feedback from more than 260 researchers (2016b). The statement release emphasized that some of its elements already exist at many institutions of higher education, but also recognized that much of what is included in the statement remains aspirational for most institutions (2016b).

The Statement suggests a new model of institutional research that moves it away from being the “one source of truth” (2016a, p. 2) and instead incorporates other institutional agents (i.e., students, faculty, and staff) not just as consumers or potential data sources, but also as institutional stewards and decision-makers in their own right. This is an important shift, with institutional research no longer confined to a service unit or office, but rather coaching and interacting with multiple decision makers. Importantly, the Statement proposes a student-focused paradigm, that is, one aimed at improving the student experience by “intentionally grounding institutional research initiatives and reports in a student-focused perspective” (2016b, p. 6, bold in original).

In this volume, we propose that one way to move institutional research (and researchers) toward the aspirational practices presented in the Statement is by more systematically incorporating qualitative research. We posit that qualitative research techniques can add depth to current institutional research practices and promote a holistic understanding of student experiences. Our work is anchored by Terenzini's (1993) definition of institutional research as a form of organizational intelligence and discussion of the various awarenesses required to cultivate organizational intelligence. Terenzini (1993) described organizational intelligence as arising from technical/analytical awareness, issues awareness, and contextual awareness. The technical/analytical domain of organizational intelligence consists of having the requisite methodological skills to construct a systematic research design and execute it with rigor and probity. Issues awareness is based on the ability of the institutional researcher to identify pressing organizational problems and to generate information that can help facilitate decision making related to them. Finally, knowledge of context focuses on the way in which these individual problems fit into the broader complex systems in the institution and the broader ecologies of which the institution is a part.

Individually and collectively the chapters in this volume present the argument that qualitative research can serve as a mechanism to produce organizational intelligence. They also hint at how very difficult it may be to achieve true organizational intelligence without at least some use of qualitative research. Although institutional researchers often employ informal qualitative methods (their experiences and anecdotal observations) to anchor their understandings of issues and context, we argue that the creation of true organizational intelligence is facilitated by formalizing the qualitative data collection and analysis process in institutional research. To that end, we present a systematic approach to qualitative research in institutional research that connects technical and analytical skills with issues awareness and knowledge of context. As a result, we organized the volume following Terenzini's (1993) definition of organizational intelligence. Chapters 1 and 2 describe technical/analytical awareness practices, Chapters 3 and 4 are dedicated to issues awareness, and Chapters 5 and 6 present practices that achieve contextual awareness.

In Chapter 1 (“A Qualitative Toolkit for Institutional Research”), Chrystal A. George Mwangi and Genia M. Bettencourt provide tools, resources, and examples for effectively grounding and conducting qualitative inquiry as a part of institutional research and assessment. The authors provide a review of key qualitative skills and knowledge areas, focusing on research paradigms, specific methodologies and methods, and data analysis. This chapter presents concrete ways for institutional researchers to build qualitative research competencies and to support the development of organizational intelligence in institutional research and assessment through the use of qualitative inquiry among its professionals. In Chapter 2 (“Integrity Is More Than Validity: Seeking Credible, Usable, and Ethical Research”), Sharon F. Rallis and Rachael B. Lawrence present a use-centered understanding of validity that is anchored in practical and responsive use of data. Together, these chapters illustrate practical ways to use qualitative research to enrich common technical and analytical awareness practices in institutional research through the use of alternative epistemological, methodological, and analytic frameworks.

In Chapter 3 (“Using Mixed Methods to Assess Initiatives with Broad-Based Goals”), Karen Kurotsuchi Inkelas describes an innovative, mixed-methods assessment design that can be used to provide evidence of an initiative's effectiveness in changing behaviors, and to inform decision-making for future programming needs. She presents a teaching and pedagogy workshop to exemplify how this type of assessment can yield not only summative but also formative data to inform decision-making. Using examples drawn from a qualitative study of belonging among students with disabilities, Rachel E. Friedensen, Byron P. McCrae, and Ezekiel Kimball demonstrate in Chapter 4 (“Using Qualitative Research to Document Variations in Student Experience”) how qualitative evidence can add depth to quantitative information about what is generally true of all students while also complicating that messaging by describing differences in experiences for a specific group of students. These two chapters exemplify issues-awareness types of organizational intelligence through the use of qualitative data to include and inform multiple institutional decision-makers.

Chapters 5 and 6 provide examples of qualitative practices that reach contextual awareness in institutional research projects that use a qualitative research approach (the case study). Elizabeth A. Williams and Martha L.A. Stassen present a case study describing their three-pronged approach integrating information derived from qualitative inquiry into planning and assessment cycles typically dominated by quantitative information in Chapter 5 (“Context Matters: Using Qualitative Inquiry to Inform Departmental Effectiveness and Student Success”). Using their qualitative inquiry cycle as a guide, the authors describe the creation of contextual awareness as an integral part of effective institutional research work. In Chapter 6 (“Cooperative Attention: Using Qualitative Case Studies to Study Peer Institutions”), Bethany Lisi uses the metaphor of the divided brain to describe the difficult work of promoting contextual awareness as a part of organizational intelligence and to demonstrate the importance of case studies in its cultivation. Lisi uses the analogy of cooperative attention of the two sides of a brain that must work cooperatively to describe both the interplay between technical, issues, and contextual intelligence, as well as the need for institutional researchers to activate both levels of attention—narrow focus on institutional issues and broad awareness of contextual patterns—to maintain organizational intelligence.

In the concluding Chapter 7 (“Using Qualitative Inquiry to Promote Organizational Intelligence”), we (the editors) integrate the arguments and examples deployed throughout the volume into a coherent call for action for institutional and educational researchers to rethink their work and practice to respond to the changing needs of higher education institutions. This chapter illustrates the importance of qualitative research in institutional research practices that respond to the aspirational practices captured in the Statement. This is possible for at least two reasons: First, qualitative research can help institutional researchers capture, include, and communicate the voices, perspectives, and needs of different institutional agents in colleges and universities. Second, qualitative research, framed under Terenzini's (1993) view of institutional research as organizational intelligence, can support the efforts of institutional researchers to gain the knowledge and skills required to push beyond technical/analytical intelligence (first tier) and toward issues (second tier) and contextual (third tier) intelligences.



Karla I. Loya
Ezekiel Kimball
Editors

References

  1. Swing, R. L., & Ross, L. E. (2016a). A new vision for institutional research. Change: The Magazine of Higher Learning, 48(2), 6–13.
  2. Swing, R. L., & Ross, L. E. (2016b). Statement of aspirational practice for institutional research (pp. 1–11). Tallahassee, FL: Association for Institutional Research. Retrieved from https://www.airweb.org/aspirationalstatement
  3. Terenzini, P. T. (1993). On the nature of institutional research and skills if requires. Research in Higher Education, 34(1), 1–10.