Tables

I.1 Disciplines Covered by Social Sciences

I.2 Overview of the Contents of the Book

2.1 Professional Skills That Research Managers Need When Conducting an Applied Social Research Study

2.2 PMI's Four Project Stages

2.3 The Three Stages of Managing an Applied Social Research Study

3.1 Template: Stakeholders to Consider Early in the Planning Stage

3.2 Questions to Help Identify a Research Problem

3.3 Template: Research Problem Statement

3.4 Template: Research Objective and Hypothesis

3.5 A Portion of a Two-Axis Matrix Used to Organize Research Material

3.6 The Sections Commonly Found in a Research Proposal for a Dissertation

4.1 Template: Planning the Sample Acquisition

4.2 Template: Planning the Data Analysis

5.1 Template: Planning Common Major Milestones in an Applied Social Research Study

5.2 Activity List Showing Key Tasks, Considerations, and Timeline Implications for Jared's Study

5.3 Template: Planning a Research Study Schedule

6.1 Template: Listing of Stakeholders for a Research Study

6.2 Examples of a Matrix of Stakeholders and Their Roles

6.3 Template: Matrix of Stakeholders and Their Roles

6.4 Listing of Tasks for Applied Social Research Studies

6.5 Matrix of Tasks and Project Staff

6.6 Example Job Description for Senior Institutional Research Analyst at a University

6.7 Position Description for a Short-Term Project

6.8 Developmental Plan for Faculty and Instructors

6.9 Example of a Portion of a Communication Plan for a Research Study

6.10 Template: Communication Plan for a Research Study

7.1 Three Examples of Personnel Budgets for a Research Project

7.2 Budget Template for a Multiyear Research Project

7.3 Example of a Budget Justification

7.4 Template: Budget Justification

8.1 Responses to Potential Risks to a Research Study

9.1 Template: Monitoring the Schedule

9.2 Template: Monitoring Work Effort

9.3 Template: Monitoring Expenses

9.4 Template: Monitoring Communications

10.1 Example Agenda for First Team Meeting

10.2 Template: Agenda for First Team Meeting

10.3 Example Agenda for an Ad-Hoc Meeting

10.4 Template: Agenda for an Ad-Hoc Meeting

10.5 Example of a Team-Building Exercise

12.1 Example of Reviewer Comments

12.2 Example of Reviewer Comments and Author Responses

12.3 Plotting a Research Paper

14.1 Questions for After-Action Review (AAR)

Figures

1.1 Stages in Designing and Conducting a Research Study

1.2 The Three Stages in an Applied Social Research Study

1.3 The Five Factors Embedded within Each Phase of Managing an Applied Social Research Study

2.1 The Relationship Between a Portfolio, Two Programs, and Three Projects

5.1 A WBS for the Project Plan the Meeting

5.2 Steps 1–3 of the WBS for Our Example Research Study That Has Two Research Questions

5.3 A Portion of the Outline for a WBS Dictionary for an Applied Social Research Study

5.4 An Example of a Criterion That Was Used for the Work Package Permission Obtained for the High-Level Deliverable Instruments

5.5 Path Diagram Showing the Major Research Milestones of an Example Research Study

6.1 Laura's Organizational Chart for Projects A, B, and C

8.1 Template: Risk Matrix

Darlene Russ-Eft, PhD, is professor and discipline liaison of adult education and higher education in the College of Education at Oregon State University, where she teaches master's and doctoral courses in research, program evaluation, and ethics. Her most recent books are Building Evaluation Capacity: Activities for Teaching and Training (2nd ed., 2015, Sage), Evaluator Competency Fieldbook (2014, Information Age), A Practical Guide to Needs Assessment (3rd ed., Pfeiffer), Instructional Designer Competencies: The Standards (4th ed., Information Age), Evaluation in Organizations: A Systematic Approach to Enhancing Learning, Performance, and Change (2009, Basic Books), and Evaluator Competencies: Standards for the Practice of Evaluation in Organizations (2008, Jossey-Bass). Dr. Russ-Eft is a past president of the Academy of Human Resource Development (AHRD) and is a past director of the International Board of Standards for Training, Performance, and Instruction (www.ibstpi.org). She is a past board member of the American Evaluation Association (AEA), a past chair of the Research Committee of the American Society for Training and Development (ASTD, now Association for Talent Development, ATD), and a past editor of the research journal Human Resource Development Quarterly (HRDQ). She received the 1996 Editor of the Year Award from the Times Mirror, the Outstanding Scholar Award from AHRD, the Outstanding Research Article Award from ASTD, and the Distinguished Service Award from AHRD. Prior to joining Oregon State University in 2002, she was director of research at AchieveGlobal and division director of research for Zenger-Miller. She also held the position of senior research scientist with the American Institutes for Research. Her BA in psychology is from the College of Wooster, and her MA and PhD in psychology are from the University of Michigan, Ann Arbor.

Catherine (Cathy) M. Sleezer, PhD, now works with Employee Training and Performance Improvement Specialists to consult on human resources. Prior to this, she served as a resident consultant for the University of Tulsa's Center for Professional and Executive Development to help organizations identify performance barriers and align training with goals. She taught strategy, performance, leadership, coaching, engagement, and more in the business world and as a university professor. Cathy also held roles in managing human resource projects for an international Fortune 500 organization (Baker Hughes) and in academia (professor of HRD at Oklahoma State University, assistant professor in Human Resource Development and director of the Institute for Research in Training and Development at Penn State, and visiting professor at the University of Tulsa's Collins College of Business). She has served as president of the International Board of Standards for Training, Performance, and Instruction (ibstpi), board member of the Academy of Human Resource Development, and member of the ASTD Research Committee. Her coauthored books include A Practical Guide to Needs Assessment (now in its third edition) (Wiley/ASTD), Human Resources Development Review: Research and Implications (Sage), Human Resource Development and Information Technology: Making Global Connections (Kluwer), Improving Human Resource Development through Measurement (ASTD), and Fieldbook of ibstpi Evaluator Competencies (Information Age). She also coauthored the ASTD (now ATD) course Measuring Training and Performance: A Course in Human Performance Improvement.

Gregory Sampson, PhD, is an educational researcher with a wide variety of professional experiences. He has worked on several large-scale assessment and evaluation projects with the aim of improving learning outcomes. His roles have ranged from research assistant to principal investigator. As a research methodologist, Dr. Sampson has been responsible for the design and execution of numerous high stakes grants and contracts. In addition to working as a professional researcher, he has also taught at several universities, primarily in the area of learning theory and research methodology.

Laura Leviton, PhD, is senior adviser for evaluation at the Robert Wood Johnson Foundation in Princeton, New Jersey, a position the foundation created for her to improve evaluations and disseminate their findings to the health field. In that role, she has overseen more than 120 national, state, and local evaluations and a wide variety of other grants and contracts for social research related to health. Formerly a professor at two schools of public health, Laura collaborated on several large randomized experiments, including the first study of effectiveness of HIV prevention and two large studies on improving medical care quality. For her work on HIV and occupational health issues, Laura received an award from the American Psychological Association for Distinguished Contributions to Psychology in the Public Interest (1993). She served on three National Academy of Medicine committees evaluating preparedness for terrorist attack, federally supported hearing loss research, and childhood obesity prevention. Laura was president of the American Evaluation Association (2000), was appointed by the U.S. Surgeon General to the National Advisory Committee on HIV and STD Prevention, and has served on other national and international committees devoted to improved methods in health and social research. Laura has coauthored two other books: Foundations of Program Evaluation (Sage, 1991) and Confronting Public Health Risks (Sage, 1997).

I am honored to write the foreword for this important book, Managing Applied Social Research. While reading it, I certainly found myself wishing I had such a guide when I was designing and managing research projects early in my career. I now realize I had benefits of guidance that many researchers may not have depending on their situation. As a graduate student in the Industrial and Organizational Psychology program at the Pennsylvania State University, I participated in our practicum course with other graduate students. Project teams included both first-year graduate students and more seasoned and experienced students. By working with these students on our applied research projects, I was learning from their experience about how to plan, execute, and conclude a successful project. When I was the more senior student on a project team, it was my responsibility to ensure that other students learned the important steps of conducting a rigorous applied research study.

In my first year of working at the Center for Creative Leadership, I was assigned a complex project to lead. My manager at the time was herself very organized, and I was able to model my project management approaches after her successful approaches. Years later, she shared with me that I was her favorite project manager because I structured and managed the complex, multiyear project so well. Managing that project taught me many of the lessons presented in this book. I learned the importance of planning, understanding, and designing for stakeholder expectations, communication, and follow-through.

But, oh! How I wish I had had a resource like this book all of these years! The guidance in this book is superb. It makes you feel like you've got a highly experienced coach sitting with you, asking you questions to prompt your own thinking.

A researcher can't ask for better coaches than the authors of this book. A review of their biographies is impressive and somewhat intimidating in and of itself. I have had the benefit of working directly with Darlene Russ-Eft and can testify to the depth of experience she has in managing projects successfully. Her approach always results in a study that is planned and designed to draw the intended learning from the results; she stays connected with stakeholders throughout the process and engages them for their input along the way, as appropriate; and she leads projects to completion in a way that energizes and satisfies both team members and other stakeholders. Although I haven't had the opportunity to work with Cathy Sleezer, Greg Sampson, and Laura Leviton directly, I know and respect them by reputation and through the interactions we've had over the years. All four of these authors are leaders in their fields and respected for their methodological expertise. When preparing a research proposal that one or more of them will review, one knows to be thorough in every aspect of the proposal.

These four highly respected professionals' willingness to put their time and energy into writing a book of this nature reflects the passion they feel for the topic. Indeed, they tell us clearly in the Postscript that they were tired of encountering unnecessarily challenging situations caused by the simple lack of knowledge and ability to effectively manage research projects and teams. As they point out, although research methods are commonly taught in graduate school, research management is not. It is something that must be learned through experience, yet it can be significantly improved by having written guidance providing advice on all of the nuances required for successful research management.

Readers will appreciate the clear organization of the book, divided into the major phases of planning, execution, and closing out the project. Within each section, the chapters each address a specific aspect of work in that phase of research management. The depth of guidance and resources in each chapter is impressive. Unlike some other books, the authors not only tell us that, for example, digital resources are helpful for supporting multiple aspects of a research project, but they also give the reader specific platforms or software recommendations based in their experience (Chapter 3). And they base their recommendations not only on their own experience but on the very solid practices of organizations such as the Project Management Institute (PMI) (Chapter 2). I have to admit, I was not aware of PMI for the first decade or so of my research management practice. Even when I did become aware of it, I wasn't so sure that it was what I needed for research management—it seemed better suited to disciplines such as information technology project management. Yet the authors are absolutely accurate in stating that—even though any given research project may be unique in some way(s)—most projects still should follow the same basic steps and phases.

The ideal audience for this book is fairly large. Certainly students and early career researchers will benefit from the guidance in this resource. I would recommend it as a must-have for students and early career scientists practicing in the social research fields. I would also recommend it for researchers managing large, complex, and/or longitudinal studies as a thorough and handy resource when making decisions about the project itself. Those who are the lead investigators should also share the highlights of the book, at a minimum, with others on their team. Certainly other team members will learn from the way they see the research manager leading the research team, yet knowing more about the “magic” behind their actions is invaluable. In fact, I'd actually recommend that anyone playing a role on a research project have access to the book. I will certainly recommend it to each of the research faculty members in my organization.

In closing, I am beyond impressed by the skill and expertise demonstrated in the chapters of this excellent book. It is a thorough and critical resource for anyone responsible for social research project management. It is clearly written and structured and will be helpful to research managers at any stage of their project. Whether one is at the beginning of a project or far enough along to be nearing completion, the guidance in this book can strengthen a research project and thereby increase the likelihood that it will have the intended impact.

Jennifer W. Martineau, PhD
Senior Vice President, Research, Evaluation, and Societal Advancement
Center for Creative Leadership
Greensboro, NC

Applied social research studies are complex because of the topics they examine and because they are conducted in the real-world environments that are messy, chaotic, and political. Many books and articles describe how to successfully design such studies and how to collect and analyze data for them. But, in practice, even well-designed applied social research studies that use best-in-practice data collection and analysis techniques face challenges that can cause the study to fall behind schedule, acquire unanticipated and unwanted changes in focus, exceed budgeted funds, and become chaotic and frustrating to complete. Researchers who manage their applied social research studies can avoid and minimize these challenges.

This book applies the relevant thinking from the project management field to the management of applied social research studies. It views an applied social research study as a project that has a life cycle with predictable challenges. It offers an easy-to-use framework plus tools and templates for managing applied social research studies. Thus, this book augments, rather than repeats, what other authors have written about applied social research.

The authors of this book have a variety of different experiences in managing applied social research studies. Given these experiences, and given that each had to be a self-learner in managing research projects, they are passionate about bringing some of this “hidden” knowledge to light.

The book was originally conceived by Laura Leviton, based on her experiences with hundreds of evaluation projects and other social and health research undertaken through grants and contracts supported by the Robert Wood Johnson Foundation and by other foundations. While most of these ambitious, large research projects were successful, an unacceptable number ended up being terribly behind schedule, impractical to complete as proposed, and disappointing in terms of information value for the field of health. As the steward of philanthropic funds, Laura worried that these disappointments would reflect on the usefulness of other commissioned research, or even social research more generally. In any case, the grant money was wasted—money that could otherwise have supported food banks and health clinics for the poor. Her unit commissioned a report to understand the extent of the problem, revealing many of the challenges in planning, execution, and reporting that are seen in this volume.

On reflection, however, Laura realized that nobody had ever trained these researchers in project management. All their coursework was about best practices for statistics and measurement, but their management of research was all a matter of on-the-job experience. Lucky researchers would be “apprenticed” to mentors who could coach them. This practice is common in good contract research firms, as seen in portions of the interview with Randall Brown that are highlighted later. Yet clearly, many researchers had very flawed management practices, and without the proper experience, better practices were never learned. Where was the advice on best practices? There was next to nothing written about this issue for researchers at any level, whether novices or veterans—just a few tips and tricks for survey research and a standout chapter on managing randomized experiments (Boruch, 1997) but that was all.

Laura recalled her own experience when, as a junior faculty member in a public health school, she was suddenly awarded over $1 million in research grants and contracts. Nobody had ever coached her in managing people or resources at this scale! She was fortunate to hire her own mentor, Russell Schuh, another of our interviewees. That experience paid off immediately, in terms of surviving and completing these complex projects. Later, it paid off for Laura with widely respected evaluation projects in the areas of HIV/AIDS prevention, quality improvement in medicine, and childhood obesity prevention. It allowed her to anticipate problems as she reviewed grant proposals for the National Institutes of Health and National Institute of Mental Health. Once she joined RWJF Laura was able to anticipate and prevent problems for the researchers and evaluators that RWJF funded.

Laura commissioned a follow-up report aimed at researchers who sought grants from RWJF. Its aim was to help them to plan, execute, and report findings with fewer delays and to develop more realistic proposals. Junior researchers in several grant programs liked it, and their mentors liked it even more.

Laura then asked Darlene Russ-Eft to work with her on a book on such research management issues, given Darlene's experiences in managing research projects while at the American Institutes for Research, later with some for-profit training companies (Zenger-Miller and AchieveGlobal, Inc.), and currently as a faculty member at Oregon State University. For Darlene, the research projects have ranged from large-scale, multiyear efforts to those requiring only a few months. But, in each project, different challenges emerged. Indeed, writing this book represented a project, and as a research manager, Darlene recognized that such an effort really required multiple authors, who would be able to bring their own experiences and backgrounds. So, she invited Catherine (Cathy) Sleezer and Greg Sampson to contribute their own ideas and experiences to the text.

Cathy Sleezer's extensive experiences in applied social research include working as a member of research teams at the University of Minnesota's Training and Development Research Center while a graduate student and conducting an award-winning doctoral dissertation. Later, she served on the faculty at Penn State where she taught graduate courses and supervised graduate research. She also directed the Institute for Research in Training and Development where she established contracts; supervised the work of graduate students, other faculty, and support staff in conducting applied social research studies; and developed and managed budgets, schedules, resources, and so forth. At Oklahoma State University, Cathy worked as a professor and continued to conduct and supervise local, national, and international applied social research studies. Cathy then applied her research-based learning by working for an international Fortune 500 firm in various management roles including organization development, training, management development, and employee engagement.

Greg Sampson joined this writing project with the intent of sharing some of his early career lessons with future researchers who are interested in moving toward applied social science careers. He finished his PhD in 2008 and has worked on numerous applied social science projects that inform educational policy as well as teaching and learning practices. His background includes the areas of educational assessment, psychometrics, and applied evaluation. Dr. Sampson's projects have been funded by governmental agencies, foundations, and corporations.

We want to thank our interviewees for their reflections and their time, specifically, Martha Bleeker, Randall Brown, Anita Chandra, Christopher Gallagher, Roger Levine, Russell Schuh, and Lauress Wise. We also want to thank Jane Ostrander for allowing us to use some of her work, and Mary Nakashian for the ideas that originated from her interviews with foundation grantees and their challenges. Frank Chaloupka and Lloyd Johnston were generous in allowing us to reproduce Appendix B from their ground-breaking effort, Bridging the Gap (http://www.bridgingthegapresearch.org/). Mary Dixon-Woods was equally generous in letting us adapt her valuable slide show on effective writing for research papers, as seen in Table 12.3. We want to acknowledge our colleagues and students who have worked on applied social research studies—and learned research management with us. Finally, we want to thank our families for supporting us during the writing process.