Day 2 DevOps: Getting more out of your software delivery pipeline

Many teams’ DevOps origins started in one of two ways. In the first scenario, a product lead or manager returns from a conference and makes the sudden claim, “We need DevOps!” with an “automate everything” mantra. Or, in the other scenario, a rogue developer becomes tired of repeating a manual installation and decides to automate his woes away. Regardless of the approach, after a few months these development teams are operating day-to-day on haphazard automation across multiple tools, teams, and dependencies. Furthermore, only a subset of the team understands how to run the deployment tasks, since you need knowledge of the login credentials, application, running process, and the ability to interpret the successful or – even worse – failing output logs.

Unless you are using these tools every day, it can be impossible to quickly answer the simple question: “What version is in production?” In today’s CI/CD world, developers are turning towards the pipeline to provide structure to their inconsistently automated world and help answer these simple questions. However, as you dig deeper into a pipeline, that answer is only a fraction of the bigger picture.

At its simplest form, a pipeline is the intersection of an application distributable and the environment that is the intended host. The presented inventory moves from left to right, through the pipeline, from environment to environment, as new versions are built and deployed. However, thanks to multiple daily CI builds, we are now incapable of telling the difference between builds that happen hours apart. Multiply cryptic versions with automatic deployments and now there are meaningless versions everywhere. Therefore, you need a more advanced pipeline that can help answer deeper questions such as: “What changes are in the production?” and “How were the changes version deployed into production?” With that knowledge in hand, you can quickly pinpoint problematic changes when they occur and halt problematic versions before deployment. Fortunately, HCL Accelerate was built to provide the deeper visibility and tight governance across your entire pipeline. Let me show you how!

Building Your Pipeline

To get started, we are going to take a step back from the code and identify candidate applications and their stakeholders. We will want to include any individuals who assist in the movement of the built artifacts starting from the CI server and finally into the customer’s hands. These stakeholders may include: development, operations, test, or performance engineers, release managers, and technical support. Once identified, gather these resources for approximately a two-hour session with the following two part agenda.

First, generate the following three lists to identify all known pipeline components:

1. What environments are upgraded and maintained – from a dev, test, production, or release perspective – that support your application’s deployments? (Note: These environments might not always host your application. Frequently, secondary instances are required to adequately test, such as for performance or integration.)
2. What manual or automated processes are used to update these environments and where are they initiated from?
3. What release requirements, milestones, or sign offs must every version obtain before release?

Second, with this information in hand, use typical value stream mapping practices to construct a flowchart of your application’s pipeline where you will follow a single product build and end with its release. While whiteboarding is most common, any online entity relationship diagram tool works just fine. In this model, the environments (#1), processes (#2), and requirements (#3) are represented by the bubbles, inputs, and outputs respectively. Once all stakeholders are satisfied with the placement of each component, you can disperse from the meeting with a clear action plan to transpose cleanly into HCL Accelerate’s pipeline.

As an aside, patterns and groupings of various entities were likely recognized as a side effect of the pipeline mapping process. For example, instead of sequential deployments, perhaps multiple environments can be deployed simultaneously. Or, instead of one automation script per environment, a single script can be repurposed across multiple environments. Or, can you shift any release requirements to the left and prevent any potentially wasteful testing? While these ideas are important to optimize your CI/CD pipeline and reduce technical debt, your pipeline will continue to evolve as each list changes. Regardless, the adoption of a simple pipeline is a huge step towards improving development empowerment and you should expect the flexibility from your pipeline.

The provided pipeline strategy is narrow in focus to keep implementation easy. Once completed, teams can alert their managers’ stating they have finally gained control over the “automate everything” DevOps mantra and can confidently answer the question “What version is in production?” However, that is only Day 1 of their larger DevOps journey. How can the pipeline help solve Day 2 challenges where any team member can accurately and quickly identify deeper problems in their pipeline?

Fortunately, HCL Accelerate is ready to answer the challenging Day 2 questions postulated earlier. While I have only discussed CI/CD tools thus far, teams can connect the dots between versions, planned work, and unplanned work through utilizing our free Source Control and Issue Tracking plugins. This relationship allows teams to pointedly answer the question “What changes are in the production?” Furthermore, our pipeline’s deployment plans can explain “How were the changes version deployed into production?” through audit and traceability. Then, when it is time for tighter or looser control, utilizing our robust gating system to govern the versions flowing through your pipeline. Day 2 challenges can be solved iteratively through HCL Accelerate’s free Community Edition. Click here to download it today!