A healthy DevOps culture affects everyone in an organization. When a company’s concept of DevOps is that it is a team, or that only certain groups are affected by DevOps, they have actually become an anti-pattern for DevOps.
Cloud generation companies have adopted all the DevOps principles without a hitch, and it helps them be successful in a modern application environment.
Some successful BDE [1] companies have looked into this phenomenon, and seeing some of the advantages that modern companies enjoy, have decided to test the waters to see if they can enjoy the same benefits.
Low hanging fruit
Because it is hard to turn a large boat around, BDE companies start by adopting some of the easiest and most cost-effective pieces of DevOps, like moving to a modern versioning system, building some pipelines (CI), or adopting a limited infrastructure-as-code strategy for some of their newer projects.
But in order to fully benefit from DevOps, the main issue needs to be addressed: a culture of Continuous Change.
Continuous Change
By now, you have probably already heard of CI and CD and might be thinking that this is what Continuous Change refers to. Close, but not exactly. Let me explain.
CI
Continuous Integration is the discipline that forces all of the developers [2] to integrate their changes with everybody else’s changes constantly, and by constantly, we mean many times a day. This is a behavioral change that is very difficult for teams that were trained BDE. Do not confuse this with agile methodologies. Agile breaks the efforts into smaller chunks (two weeks), but even using agile, developers can get their assigned task and work on it until they feel like it is ready. CI in DevOps dictates that changes should be integrated several times a day, regardless of their status. This might seem like a dangerous practice, and BDE developers shy away from the most radical forms of it, but, once getting used to the discipline and with the help of a mature pipeline, it has many benefits. We should probably follow up with a full article on CI practices.
CD
Continuous Delivery or Continuous Deployment is the principle that states that every time a successful artifact is created (CI), is also delivered to the customer. Depending on the nature of the application, that delivery can be a new setup.exe file Delivered to a website or a new version of the application Deployed to a live server. Well, this sounds like a very dangerous practice, doesn’t it? A developer changes some logic in the code and, next thing you know, it is running on the production server. It reminds me of “I never test my code, but when I do it, I do it in production.” If you are starting to panic, don’t feel alone. That was exactly the reaction of my team a decade ago when I proposed we start doing just that, but if you take the DevOps principles as a whole and not piecemeal them to your convenience, you’ll realize that this is not a dangerous practice. Again, we should probably follow up with a full article on CD, too.
So, what is Continuous Change and how does it differ from CI/CD?
Continuous Change is a cultural change and is required for a successful DevOps transformation, but it requires that all the principles of DevOps are adopted.
This is the area where the major cultural shift needs to take place, the area where it might seem like DevOps has the potential to wreak havoc on our stable systems and, therefore, companies try to avoid it at all costs.
ITIL
ITIL and traditional change management have done a good job of protecting the stability of our applications for decades and have evolved to specify roles, guidelines, and best practices to protect our applications.
It needs to be said that both ITIL and DevOps have the same goal in this respect: to protect the stability of our applications. They just go about it in what might seem to be opposite ways of accomplishing it.
Many articles have been written about how as much as 80% of IT-related incidents are change-related, and although some in the DevOps camp will question the number, it is a very high number regardless.
ITIL protects the running applications by ensuring that if a change is absolutely necessary, it is done with careful planning, adequate testing, rehearsal, rollback strategies, timing, and resource considerations.
You can probably sense by now, that, even though traditional change management is created so that change is done safely, it is perceived as “do not touch it”. Again, the purpose is that the running systems should not be changed without careful planning, but, the effect is that because of the perception and the overhead for change, change is infrequent.
The stability of the system is effectively protected by discouraging change.
DevOps
DevOps philosophy basically says, let’s create the right culture and tooling so that change is so frequent that we become masters of change, disruption is minimized by the size of the change, and all the readiness is verified by automation.
In other words, DevOps challenges the notion that change itself is the culprit for service disruption by touching on three major areas of change: frequency, size, and readiness.
Frequency – The first suggested optimization is related to frequency. In a DevOps environment, by the time the application is made available to users, it has been deployed hundreds of times to production-like [3] environments. Any issues related to the deployment (change) have been exercised and progressively perfected. Highly related to frequency is timing. When the change is immediately deployed to production, it is very fresh in the minds of everyone involved, as opposed to looking at a failure today related to a change in code or configuration that happened several weeks ago. Continuous Integration forces the developers, or application maintainers, to merge their changes as soon as possible, making the merge smoother and less likely to run into a conflict, and the same is true for delivery and deployment.
Size – DevOps also maintains that the magnitude of the impact on IT incidents is directly proportional to the size of the change. Let’s be honest, as much as we would like, there is no perfect code. Under that assumption, it is easy to infer that delivering smaller packages will cause less disruption. Accumulating a larger number of changes 1. increases the complexity (potential of conflicts) and 2. the affected user count, which causes an original disruption. But, it also aggravates the disruption of the rollback, since 3. all the features (even those that were not broken) have to be rolled back together, and the 4. MTTR (Mean Time To Repair) is larger.
Readiness – But, DevOps would be a moot point if it couldn’t improve on the readiness aspect of the change. DevOps agrees 100% with the common wisdom that a great majority of IT incidents caused by the change are due to deficient preparation and readiness verification, but in this sense, a DevOps approach can be seen as an enforcement of the Change Management checklist via automated processes. Automation can not only fulfill the readiness requirements but even take them to a higher level of compliance because all the processes and checkpoints are digital by nature. DevOps savvy reviewers (SDLC and business process, networking, security, etc.) can read the pipeline and decide early on if it is providing enough and appropriate gates for the application to be deployed in production. In a full DevOps environment, the pipeline is the runbook[4] and the Readiness Assessment all in one.
Areas that will need attention
Tooling
Most BDE companies have invested heavily in tools for change management from the ITIL perspective, most of those tools are evolving to include DevOps modules. But, since DevOps can mean different things to different people or institutions, companies should give careful consideration of the tooling in coordination with the DevOps CoE (Center or Excellence), or the team/people driving the DevOps transformation so that the tool(s) identified are in line with the initiatives.
In some cases, it might be a good option to keep both the traditional change management and DevOps change tracking on the same tool (assuming a DevOps module or option), and in some other cases, it might be better to keep them totally separate. The special circumstances of each company, the maturity of the change management process, and the existing tools make it very difficult to give a one-size-fits-all answer to the tooling question.
Culture
It is very unlikely that everyone in a BDE organization is trained and ready to start making a difference using DevOps methodologies, and it is likely to cause some fear of change (pun intended). But, the DevOps culture change is not mostly about new tools or technologies, instead, it is about changing the way we do things, and most people are capable of making the switch given the right training. In other words, a successful DevOps transformation should change the culture and practices of the people on the things they do today, rather than bringing new tools or even people.
Metrics
If you can’t measure it, it didn’t happen, or so they say. Start by creating criteria for success. How will you determine if you are being successful in the transition? If the traditional change management has not been providing stability, it might be easy to determine a successful conversion, but it will get trickier if the system is already stable, the determination will need to move into other areas like MTTR, customer satisfaction, etc.
Conclusion
DevOps is not perfect, and during the adoption stage, there will be delays and frustrations. Yet, very large and successful companies are using its principles to keep some of the largest and most complex applications in the world running stable and without problems. To emulate their success, a company needs to be willing to make this profound, cultural “change management” change.
[2]: Developers in a DevOps world are not just the people writing logical pieces of code in Java or PHP, but every participant in the development of the application, which includes ISO writing security policies, networking, setting up an IP address, or opening a firewall port, and every person that participates in shaping the application’s capabilities and behavior.