software development | devops | product development
Welcome to Create Impact, a new series from Aviture focused on the topics that inspire our engineers to innovate. In each article, an Aviture team member will take you on a deep dive into a subject they’re passionate about, showing you the thinking behind cutting-edge engineering advances, the latest UX trends, development theories, and other unique topics that enable Aviturians to embrace the Art of the Possible for our clients.
In this post, Senior Solution Architect Joel Elixson explores how continuous delivery can revolutionize the software development process. He defines continuous delivery, explores how to overcome barriers to widespread adoption, and shows how the right tools and culture are crucial for spurring innovation.
Many software companies release product updates at the end of a sprint. That’s been the traditional model for a long time.
That equates to a release about every two weeks (the typical length of a sprint), sometimes longer. In an enterprise setting, we often plan releases that require coordination between multiple teams, product owners, and, occasionally, company leadership. This lengthy process of approval-gathering and release planning may produce release schedules with infrequent (and very large) releases. Monthly or quarterly releases aren’t uncommon in this setting.
Software deployments more or less follow the same deployment patterns. You run some tests against a release candidate, perform any static analysis you need to, deploy your release artifacts to a test environment, run some features through quality assurance, and gather approvals before finally deploying to production. We return to our lives in our safe, non-production sandbox, and we wait for support requests to roll in.
When we release large, infrequent updates, we’re relying on a fair bit of luck. If we’re lucky, we won’t receive many support requests. If we’re very lucky, the support requests we do receive will be easy to diagnose and hotfix. Most of us spend a lot of time on both activities.
How do you solve for this? One answer is continuous delivery.
Continuous Delivery: Defined
Continuous delivery is all about adding a lot of value, cumulatively, in very small chunks, where each chunk is the product of a tight feedback loop.
What really makes continuous delivery different from a more traditional delivery model is the rapidity of product releases. “Rapidity” here refers both to how often we deliver features and how quickly we collect feedback on what’s delivered so we can course-correct as soon as possible.
When we talk about rapid delivery, we're talking about a cadence that’s less than weekly and typically not more than a couple days. A lot of the continuous delivery evangelists deliver product updates into production multiple times per day.
The Benefits of Continuous Delivery
The interesting thing about continuous delivery is - conceptually - you still deliver the same number of changes over the same period of time as traditional development models. But in reality, the aggregate value of those changes will be larger. That’s because continuous delivery imparts a few key benefits.
In a continuous delivery model, the responsibility of collecting feedback on new features is shifted away from focus groups and onto a larger, more representative sample of users. Our feedback loops become more accurate and, because we collect feedback from our users every time any new feature is completed or an old feature is improved, our feedback loops also become timelier!
In practice, all changes begin to look very similar to traditional hotfixes. And because we’re highly sensitive to user feedback, we’re also highly responsive, which translates into the perception – among our users – that we’re more engaged in improving our products. Tight feedback loops are, in my mind, the most valuable feature of continuous delivery.
One of the other things I really love about continuous delivery is the ability to quickly isolate and fix the root cause of a new defect. This is really an emergent property of the continuous delivery process. When a feature breaks (and I do mean when), we don’t have to comb through a month’s worth of changelogs and commit history to determine which change to which library is at the center of our current drama. That’s the worst case, but it's not an uncommon one.
Under continuous delivery, the blast radius of any specific change is often small, so we’re afforded more opportunities to “fix forward” rather than rolling back our product releases. At the end of the day, fixing forward feeds back into the user’s perception of being highly engaged in the maintenance of our products. And if we must roll back a release, the impact on the user’s experience is relatively minor.
So while conceivably, yes, you could deliver the same amount of value over the same period of time with both continuous and traditional delivery models, odds are you won’t.
The benefits I’ve explored are user-facing, but a well-designed continuous delivery process also strongly correlates with net-positive impacts to team satisfaction, software architecture, and organizational culture.
Accelerate: Building and Scaling High Performing Technology Organizations is a very approachable book on the topic of continuous delivery, and I highly recommend it.
The Right Tools for Continuous Delivery
This is a big topic and, I think, one better suited for a follow-on blog. Still, I can give the broad strokes of a tooling solution for continuous delivery; I’ll ignore the huge, architectural elephant in the room for now.
Let’s say you’re committed to at least exploring continuous delivery. What then?
Focus on a fully featured DevOps platform. Azure, Amazon, and Google all provide excellent, cloud-based DevOps solutions. You may decide on Jenkins or CircleCI as your solution. That’s fine. It simply matters that the platform you choose can accommodate your build and release processes, which means you also need a tool for modeling them. I don’t have a personal recommendation here; whatever tool you currently use to model your service architecture should be sufficient for this purpose.
Your remaining challenges are procedural and architectural. How do you manage quality checks? How do you assess the fitness of code being merged? What architectural strategies do you use to ensure services and systems are loosely coupled? How do you promote team ownership of every feature and defect? How do you promote trust in your release processes?
The right tools will start you down the path to success, but process and advocates will take you the rest of the way.
The People Hurdle
In my experience, the biggest obstacle to pursuing continuous delivery isn’t technology. DevOps technology is already very mature, and someone, somewhere has published a blog on how to implement your exact delivery strategy. Rather, it’s considerably more important to create a team willing to embrace the principles of continuous delivery. This is The Big Pitch.
Continuous delivery is quite a change from the more traditional approaches to software delivery. This one’s a little more difficult to address, because preparing developers to operate in this brand-new way is something of a trial by fire.
Developers have traditionally relied on a promotional strategy, gradually deploying from lower to higher environments. As our code graduates from one environment to the next, we become more confident in its correctness. Under continuous delivery, we require acceptance testing to occur much earlier, which has the effect of requiring frequent communication between quality engineers and developers, eventually leading developers to better understand the importance of the quality criteria.
Beyond improving and automating our acceptance testing, we’re asking developers to become comfortable operating very close to our production environment. This is a psychological barrier, and overcoming it requires us to address deficiencies immediately so we can continue to build confidence in our deliveries.
If a feature breaks in production, the team stops what they’re working on to write acceptance tests or unit tests for it. If a poor process brings down our application, the team stops what they’re working on, rolls back (if necessary), and addresses the root cause or automates the process if it’s a manual one. Success in this environment very much depends on communicating the expectation that every problem is a team problem.
We must create a culture of psychological safety to be truly successful with continuous delivery and, for many organizations, that’s far more difficult than the technical or engineering components.
To Be Continuous
Continuous delivery is something that’s really close to my heart right now. On our engagement with Carson, we’ve sped up our releases quite a lot. We’re still shy of the daily releases I’d love to see, but continuous delivery is a continuous process. We’re dedicated to delivering the most value we can, and continuous delivery is a huge part of that strategy.
Interested in finding out how continuous delivery can be used to spark innovation at your company? Contact Aviture.
