DevOps For Dummies®
Published by: John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030-5774, www.wiley.com
Copyright © 2019 by John Wiley & Sons, Inc., Hoboken, New Jersey
Media and software compilation copyright © 2017 by John Wiley & Sons, Inc. All rights reserved.
Published simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without the prior written permission of the Publisher. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions.
Trademarks: Wiley, For Dummies, the Dummies Man logo, Dummies.com, Making Everything Easier, and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.
LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY: THE PUBLISHER AND THE AUTHOR MAKE NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESS OF THE CONTENTS OF THIS WORK AND SPECIFICALLY DISCLAIM ALL WARRANTIES, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE. NO WARRANTY MAY BE CREATED OR EXTENDED BY SALES OR PROMOTIONAL MATERIALS. THE ADVICE AND STRATEGIES CONTAINED HEREIN MAY NOT BE SUITABLE FOR EVERY SITUATION. THIS WORK IS SOLD WITH THE UNDERSTANDING THAT THE PUBLISHER IS NOT ENGAGED IN RENDERING LEGAL, ACCOUNTING, OR OTHER PROFESSIONAL SERVICES. IF PROFESSIONAL ASSISTANCE IS REQUIRED, THE SERVICES OF A COMPETENT PROFESSIONAL PERSON SHOULD BE SOUGHT. NEITHER THE PUBLISHER NOR THE AUTHOR SHALL BE LIABLE FOR DAMAGES ARISING HEREFROM. THE FACT THAT AN ORGANIZATION OR WEBSITE IS REFERRED TO IN THIS WORK AS A CITATION AND/OR A POTENTIAL SOURCE OF FURTHER INFORMATION DOES NOT MEAN THAT THE AUTHOR OR THE PUBLISHER ENDORSES THE INFORMATION THE ORGANIZATION OR WEBSITE MAY PROVIDE OR RECOMMENDATIONS IT MAY MAKE. FURTHER, READERS SHOULD BE AWARE THAT INTERNET WEBSITES LISTED IN THIS WORK MAY HAVE CHANGED OR DISAPPEARED BETWEEN WHEN THIS WORK WAS WRITTEN AND WHEN IT IS READ.
For general information on our other products and services, please contact our Customer Care Department within the U.S. at 877-762-2974, outside the U.S. at 317-572-3993, or fax 317-572-4002. For technical support, please visit https://hub.wiley.com/community/support/dummies.
Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.
Library of Congress Control Number: 2019945514
ISBN: 978-1-119-55222-2
ISBN 978-1-119-55223-9 (ebk); ISBN 978-1-119-55224-6 (ebk)
What is DevOps?
This question is one of the most common questions I get in my work — and I hear it from experts and novices alike. I’ve worked in technology for almost two decades and have been a DevOps researcher, strategist, and expert who has guided hundreds of technology leaders and engineers to make their software better, allowing them to deliver value to their customers faster and safer. Yet, many of us in this field still hear this question.
Unfortunately, a single, universally accepted definition of DevOps doesn’t exist — much to everyone’s chagrin. But if we’re honest, it probably doesn’t matter; after all, having one clear definition codified in the Agile Manifesto didn’t help the Agile community much, either.
With DevOps For Dummies, Emily Freeman has written a book about DevOps that you can read from cover to cover as a book, or use as a reference, or jump around in for a choose-your-own adventure exploration through DevOps concepts. It’s a brilliant way to structure the content because DevOps covers some development, some operations, and a whole lot of culture — plus a whole lot more. In this book, Emily has done a great job of digging into important concepts for teams implementing this new way of work, whether you’re greenfield or brownfield or have no idea of what the words greenfield or brownfield mean.
Emily brings clear eyes and a fresh voice to the topic, crafting insightful narratives and breaking down concepts into clear writing. By coming into technology mid-career, she brings an understanding and comprehension that others who were “born” into the tech sometimes take for granted. Her writing communicates these (sometimes hidden) details effortlessly, walking the reader through the landscape with ease and wit.
Some of my favorite sections are those on developing code so that it’s clear and maintainable (check out Chapter 9, especially the part about peer review) and empowering teams to help you scale. I have too many favorite chapters and sections to list, so I urge you to find your own favorites and not to run out of sticky notes and highlighters!
I wish you the best of luck on your DevOps journey. Remember that where you start doesn’t matter; what matters is that you keep going and keep improving.
— Nicole Forsgren, Research & Strategy at Google Cloud and co-founder and CEO of DevOps Research and Assessment (DORA)
I believe that the greatest challenges facing the tech industry aren’t technical; they’re human. Think about it: Hardware and computing are more powerful than they ever have been. Automated tools remove the drudgery of rote work. Robust frameworks and libraries build shortcuts and functionality into applications for you. You can do more, faster, than ever before.
The problem that organizations face now is that of the social dynamics of humans on engineering teams. Specifically, those dynamics are the natural friction that arises from misaligned incentives and poorly communicated goals; the frustration in attempting to explain a concept or approach to someone who has a different expertise than you; and the fear people feel when they think about looking stupid in front of their colleagues or being automated out of a job.
DevOps addresses all these issues, and this book explains how.
I’ve designed this book to be a resource that someone who has never heard of DevOps — or doesn’t know what it actually means — can walk through to gain a broad understanding of DevOps and how it fits into the software development life cycle (SDLC) — that is, the entire process of how software is developed and released. Although SDLC has traditionally used the word development, I prefer delivery because it removes the possible elevation of developers over other disciplines.
I’ve ordered the information in this book to be both approachable chronologically as well as selectively. You can read it straight through and allow each section to build on the last or you can hop around to your heart’s content.
I use the following conventions throughout the book:
monofont. If you're reading a digital version of this book on a device connected to the Internet, you can click or tap the web address to visit that website, like this: www.dummies.com.When I first started writing this book, I struggled to identify my main audience. Developers? Operations people? Executives? It was important to me to make DevOps approachable and real. Too often, people talk about DevOps only in the context of greenfield projects and companies with nearly endless resources. I wanted to pull back the shine and get to the substance of DevOps as a discipline — one that helps engineers do their job better and faster. I also wanted to ensure that I met the reader where they were and spoke to people who had never heard of DevOps as much as I spoke to the engineers who are well on their way to advanced DevOps practices. In the end, I focused on anyone who needs to deliver business priorities as much as retain engineering talent. They need realistic solutions to real-world problems. They see the benefits of automation but also need to ensure security and compliance with regulatory bodies.
Regardless of whether you fit that exact profile, I hope that you can glean what you need from this book and that it may play a small part in your success as you evolve and improve your engineering practice.
The Tip icon marks tips (duh!) and shortcuts that you can use to make implementing DevOps practices easier.
Remember icons mark the information that’s especially important to know. To siphon off the most important information in each chapter, just skim through these icons.
The Technical Stuff icon marks information of a highly technical nature that you can normally skip over.
The Warning icon tells you to watch out! It marks important information that may save you headaches and unnecessary conflict.
This section describes where readers can find book content that exists outside the book itself. A For Dummies technical book may include the following, although only rarely does a book include all these items:
www.dummies.com.www.dummies.com.I’ve written this book in such a way that you can skip around if you so choose. If you’re a developer, you may find that you know most of the information in Part 2, which covers the development pipeline (though I suggest skimming it to catch a few new ideas!). If you’re an operations engineer, you may feel more confident in some of the more infrastructure-focused chapters.
A DevOps transformation is no small feat, nor is it an overnight process. It will take hours of planning, honest conversations, brainstorming, reflection, and technical changes. Don’t rush through the process. Simply learning and thinking about your everyday work from a different perspective is a healthy way of waking up your mind. The journey is just as valuable as the outcome.
Part 1
IN THIS PART …
Understand DevOps values and priorities, which focus on people, process, and technology.
Design your organization’s culture with DevOps in mind through encouraging teamwork, reducing silos, and embracing failure.
Identify waste and locate bottlenecks along your software development life cycle to locate the easiest (and most immediate) wins for your DevOps transformation.
Persuade your colleagues, from executives to engineers, of the benefits of DevOps for an engineering organization.
Measure your work and track your DevOps successes, allowing everyone to see the incremental improvements.
Chapter 1
IN THIS CHAPTER
Surveying the main tenets of DevOps
Understanding DevOps values
Seeing how your organization benefits
DevOps has transformed the way engineering teams collaborate to create and ship software. It’s a broad and encompassing philosophy that inspires diverse implementations across the industry.
I define DevOps as an engineering culture of collaboration, ownership, and learning with the purpose of accelerating the software development life cycle from ideation to production. DevOps can enable you to reduce interpersonal friction, eliminate bottlenecks, improve collaboration, increase job satisfaction through engineer empowerment, and accelerate team productivity. DevOps is no silver bullet, but it can have massive impact on your organization and your products.
In this chapter, I emphasize the importance of culture over process and tooling, discuss the principles and values of DevOps, and dive into how your organization will benefit from a DevOps approach.
This book has no exact DevOps prescription for you — because none exists. DevOps is a philosophy, one that that prioritizes people over process and process over tooling. DevOps builds a culture of trust, collaboration, and continuous improvement. As a culture, it views the development process in a holistic way, taking into account everyone involved: developers, testers, operations folks, security, and infrastructure engineers. DevOps doesn’t put any one of these groups above the others, nor does it rank the importance of their work. Instead, a DevOps company treats the entire team of engineers as critical to ensuring that the customer has the best experience possible. (You can find more about company culture in Chapter 2.)
In 2001, 17 software engineers met and published the “Manifesto for Agile Software Development,” which spelled out the 12 principles of Agile project management (see the sidebar “The origins of Agile” in Chapter 7 for more details). This new workflow was a response to the frustration and inflexibility of teams working in a waterfall (linear) process. Working within Agile principles, engineers aren’t required to adhere to original requirements or follow a linear development workflow in which each team hands off work to the next. Instead, they’re capable of adapting to the ever-changing needs of the business or the market, and sometimes even the changing technology and tools.
Although Agile revolutionized software development in many ways, it failed to address the conflict between developers and operations specialists. Silos still developed around technical skill sets and specialties, and developers still handed off code to operations folks to deploy and support.
In 2008, Andrew Clay Shafer talked to Patrick Debois about his frustrations with the constant conflict between developers and operations folks. Together, they launched the first DevOpsDays event in Belgium to create a better — and more agile — way of approaching software development. This evolution of Agile took hold, and DevOps has since enabled companies around the globe to produce better software faster (and usually cheaper). DevOps is not a fad. It’s a widely accepted engineering philosophy.
Anyone who says that DevOps is all about tooling wants to sell you something. Above all else, DevOps is a philosophy that focuses on engineers and how they can better work together to produce great software. You could spend millions on every DevOps tool in the world and still be no closer to DevOps nirvana. Instead, focus on your most important engineering asset: engineers. Happy engineers make great software. How do you make happy engineers? Well, you create a collaborative work environment in which mutual respect, shared knowledge, and acknowledgement of hard work can thrive. See Chapters 2 and 15 for more about how to create teams of happy, empowered engineers who embody a growth mindset and take pride in their work.
Your company has a culture, even if it has been left to develop through inertia. That culture has more influence on your job satisfaction, productivity, and team velocity than you probably realize.
Company culture is best described as the unspoken expectations, behavior, and values of an organization. Culture is what tells your employees whether company leadership is open to new ideas. It’s what informs an employee’s decision as to whether to come forward with a problem or to sweep it under the rug.
Culture is something to be designed and refined, not something to leave to chance. Though the actual definition varies from company to company and person to person, DevOps is a cultural approach to engineering at its core.
A toxic company culture will kill your DevOps journey before it even starts. Even if your engineering team adopts a DevOps mindset, the attitudes and challenges of the larger company will bleed into your environment.
With DevOps, you avoid blame, grow trust, and focus on the customer. You give your engineers autonomy and empower them to do what they do best: engineer solutions. As you begin to implement DevOps, you give your engineers the time and space to adjust to it, allowing them the opportunities to get to know each other better and build rapport with engineers with different specialties. Also, you measure progress and reward achievements. Never blame individuals for failures. Instead, the team should continuously improve together, and achievements should be celebrated and rewarded.
Observing your workflow without expectation is a powerful technique to use to see the successes and challenges of your workflow realistically. This observation is the only way to find the correct solution to the areas and issues that create bottlenecks in your processes. Just as with software, slapping some Kubernetes (or other new tool) on a problem doesn’t necessarily fix it. You have to know where the problems are before you go about fixing them. As you continue, you collect data — not to measure success or failure but to track the team’s performance. You determine what works, what doesn’t work, and what to try next time. In Chapter 3, you learn how to identify bottlenecks in your development process.
Selling the idea of DevOps to your leaders, peers, and employees isn’t easy. The process isn’t always intuitive to engineers, either. Shouldn’t a great idea simply sell itself? If only it were that easy. However, a key concept to always keep in mind as you implement DevOps is that it emphasizes people. The so-called “soft skills” of communication and collaboration are central to your DevOps transformation. Persuading other folks on your team and within your company to adopt DevOps requires practicing good communication skills. Early conversations that you have with colleagues about DevOps can set you up for success down the road — especially when you hit an unexpected speed bump.
The aspect of DevOps that emphasizes making changes in small, incremental ways has its roots in lean manufacturing, which embraces accelerated feedback, continuous improvement, and swifter time to market. When I talk about DevOps transformations, I like to use water as a metaphor. Water is one of the world’s most powerful elements. Unless people are watching the flood waters rise in front of them, they think of it as relatively harmless. The Colorado River carved the Grand Canyon. Slowly, over millions of years, water cut through stone to expose nearly two billion years of soil and rock.
You can be like water. Be the slow, relentless change in your organization. Here’s that famous quote from a Bruce Lee interview to inspire you (https://www.youtube.com/watch?v=cJMwBwFj5nQ):
Be formless, shapeless, like water. Now you put water into a cup, it becomes the cup. You put water into a bottle, it becomes the bottle. You put it in a teapot, it becomes the teapot. Now, water can flow or it can crash. Be water, my friend.
Making incremental changes means, for example, that you find a problem and you fix that problem. Then you fix the next one. You don’t take on too much too fast and you don’t pick every battle to fight. You understand that some fights aren’t worth the energy or social capital that they can cost you.
This entire book dives into how you and your team can benefit from implementing DevOps in your organization. Beyond the human component, which enables faster delivery, improved functionality, and fearless innovation, DevOps has technical benefits.
Continuous integration and continuous delivery (CI/CD) are closely aligned with DevOps. Continuous software delivery removes many of the bottlenecks often seen in teams that deploy infrequently. If you create automated pipelines that pass new code through a robust test suite, you can feel more confident in your deployments. (I talk more about CI/CD in Chapter 11.)
DevOps also enables faster recovery from incidents. You will inevitably experience a customer-impacting service disruption at some point, no matter how well tested your code is. But teams who work in a DevOps methodology find resolutions faster through better coordination, more open accessibility, shared learning, and better performance monitoring.
Engineering is not the only side of your organization that benefits from DevOps. The business side of your organization will see fewer customer complaints, faster delivery of new features, and improved reliability of existing services.
DevOps enables you to do more with the resources you already have. It accepts the reality of constraints and shows you how to succeed within your unique environment.
As you begin to get familiar with DevOps, you’ll likely come across a model called CALMS. It stands for culture, automation, lean, measurement, and sharing, and it’s a helpful framework through which to understand the DevOps principles and evaluate your DevOps success as you apply those principles throughout your organization.
Your culture needs to be collaborative and customer centered, which means your engineers understand that the purpose of technology is to make your customers’ lives easier. If customers don’t find value in the product, the product will fail. Technology is secondary to this goal. The best DevOps cultures are extremely collaborative and cross-functional, with people from different teams and varying skill sets working together to engineer a better product. Listening is a major component of communication, and an easy litmus test of culture is to listen to conversations. Are people constantly talking over each other? If so, I bet you have opportunities for major cultural improvements ahead.
Rote tasks are an engineer’s worst nightmare, not only because they’re, well, boring, but because they’re inefficient. Engineers speak computer so that they can make computers do the jobs that people don’t want to do. Usually the lowest-hanging fruit for improvements in automation are code builds, automated testing, deployments, and infrastructure provisioning. I dig deeper into identifying low-hanging fruit in Chapter 3.
Lean doesn’t refer just to lean manufacturing. It applies more widely to the nature of DevOps teams, which are agile and scrappy. Lean teams eschew low-impact activity because it doesn’t provide value to the customer. Another aspect of lean is how it keeps to the goal of continuous improvement. Everyone embraces a growth mindset and earnestly wants to improve.
Data is critical to DevOps. Measuring progress through data will inform nearly every aspect of your organization’s transformation. Keep in mind, though, that progress should never be tied to individual performance. Think of it as tracking your progress along an endless marathon rather than as a way of knowing when you’re “done.” You’re never done. No one is.
Instead of regarding the data you collect as a measure of how poorly you’re doing, think of it as gauging your improvement. Celebrate the wins. That approach bolsters the entire team and keeps your engineers happy, motivated, and productive. I guarantee you’re doing some things well, and highlighting the good is important. I talk about what you can measure in Chapter 5.
DevOps was founded because operations and development had some conflict. They lacked common ground and were incentivized based on different standards. Operations folks are typically measured on the reliability and availability of an application, whereas developers are, more often than not, incentivized to create new features for the application. (I talk more about how operations and development are measured in the next section.) You know what the biggest threat to uptime is? Deployments. Developers initiate deployments with new code releases. Thus, operations folks hate developers. Now, it’s not usually that bleak, but there’s a seed of truth in that. The friction makes solving problems nearly impossible and turns everything into a blame game. DevOps seeks to change that atmosphere completely and create an environment in which both teams teach each other and feel empowered — thus building a single team through which everyone contributes.
On traditional engineering teams, developers (those who write the code) and operations engineers (those who deploy systems and maintain infrastructure) are on opposing sides of a never-ending war. Okay, that’s not exactly accurate. But they don’t get along, and that’s because they’re measured by different criteria.
Developers are typically measured by the number of features they release or the number of bugs they fix. (Evaluating developers by lines of code written is a terrible idea. Many times, the best developers delete more lines of code than they add.)
Unfortunately, code quality and reliability aren’t typically measured. As a result, developers naturally prioritize the work that will make them look productive. They don’t spend time refactoring code to make it more readable or paying off technical debt accrued from the last big product push.
In contrast to how developers are measured, operations teams are typically measured by site reliability and uptime. You’ve likely heard of the five 9s: 99.999 percent availability. The five 9s means that your site can be down for only five minutes per year. Five minutes … per year. That’s a lot to ask. It’s also expensive to maintain because of the number of storage and compute resources you must have at your disposal, not to mention the personal impact it has on the operations individuals tasked to keep availability at that level. Those people are often asked to take on heroic efforts and respond to problems regardless of the day, time, existing workloads, or personal obligations.
To make the conflict clear: In traditional engineering organizations, developers must deploy new code to release new features. But deploys are the most common action that initiates service disruptions and site outages.
Two problems come from this situation:
Do you see why you hear audible sighs when developers and operations teams interact? DevOps seeks to eliminate both the challenges created by siloed responsibility and opposing goals. By aligning incentives, sharing knowledge, removing barriers, and respecting different roles, DevOps can dramatically improve the interpersonal communication and cooperation on your team.