Data-Driven DevOps Part 3: Track and Plan with Data

Click here to read part 1 of the series

Click here to read part 2 of the series

A challenge that comes with having a responsive, highly agile development organization is constantly juggling the numerous requests from the business, the customers, and our own teams. This leaves us with more work that needs to be discussed and prioritized than can realistically be accomplished within a given sprint. Over the years, teams have used a number of methodologies to try to get better at tracking and planning – from Scrum, Kanban, and Extreme Programming to Adaptive, Dynamic, and Lean Software Development. Each approach takes a different twist on the same challenge – being flexible, while still being predictable.

Data can play an integral role in a development teams’ ability to track and plan. In the last post, we discussed how data can improve the collaboration and communication across a development team. More specifically, we see that as a team analyzes the data it creates on a per-sprint basis, it fosters conversations around process improvement to identify what is working and what is not working. Beyond culture, data analytics has several advantages in an organization’s ability to track and plan.

Improve planning by identifying team velocities

For a Scrum team, Velocity is a measure of the amount of work a team can tackle during a single sprint and is calculated by totaling the number of story points delivered in a sprint. Not only does knowing a team’s velocity help us plan for a given sprint or release, it also helps us improve communication throughout our organization and to our customers.

Understanding a team’s velocity can help set meaningful expectations. How many times have you found out a few weeks before a planned release that a handful of items expected to be delivered won’t make it out the door? It’s always a letdown. Setting meaningful, realistic expectations with stakeholders and customers is necessary to create transparency and reduce confusion. In turn, this same transparency increases the chances of individual contributors being successful in achieving the goals set for them.

Lead time, cycle time, throughput, and more

Velocity isn’t the only metric that can greatly improve an organization’s ability to forecast accurately. Lead time and cycle time are the most common statistics development teams, product managers, and release engineers track to get an understanding of the time it takes to deliver software through the organization. The difference between these two metrics is subtle.

• Lead time is the period of time between a new task’s appearance in your backlog and its final state of “done”. Many teams will use an alternate way of defining lead time, which is to measure the time between when the business commits to a specific request/task to when it is completed. Otherwise, it is possible that new tasks could sit in the backlog for months before being prioritized, and that can inflate the lead time.
• Cycle time begins the moment someone starts working on a request or task, and spans until the work is completed.

Both these metrics are powerful. Aside from helping us better understand capacity, they provide valuable information about a team’s current working process and indications of bottlenecks in the current workflow.

However, lead time and cycle time are not enough. When interpreted alone, they still leave many questions about the types of work a team is able to complete in a given sprint or release. To better understand the answers to these questions, we look to different metrics that can be found within the data, including throughput, distribution, and contributors.

• Throughput is the number of tasks or work items that were completed for a given period of time. This is different than velocity, which takes into account story points.
• Distribution is a detailed breakdown of the types of work completed for a sprint, be it a task, a defect, a feature, etc.
• Contributors is simply the number of developers participating in a given sprint. A team of ten developers doesn’t always have ten people contributing. With vacations, illness, and many types of unplanned work, this number can be fairly inconsistent.

It is important to consider all these metrics when trying to accurately forecast deliverables.  Knowing a development team’s lead time, cycle time, velocity, throughput, average number of contributors, and typical distribution gives us a much more accurate picture when work should be completed and the types of work that will most likely be done.

Identifying Risk

Obviously, every team will occasionally experience some sort of delay in delivery. In these scenarios it is important that an organization can make calculated and informed decisions on what should or should not be removed from the scope of a release. One of the key benefits of analyzing the data from development teams is that we can start to link specific tasks to business value. In development terms, this means linking epics (business value) to work items (tasks).

By creating this linkage, we can start to understand what percentage of a certain business value is complete. This information is priceless in our ability to identify and communicate risk to stakeholders and customers in order to set meaningful expectations. Anytime a deliverable slips, you can guarantee there are going to be a few questions, like, “Why were we not able to complete this work?” and “What was the team focused on besides this deliverable?” The answer to these questions can be found in the data. It could be that there was too much work in progress, or that the team got sidetracked by support issues and pulled away from their planned work. Whatever the answer might be, analyzing the data allows the development team to have a more meaningful retrospective, and adjust their processes to prevent the same mistakes from happening in the future.

Data-driven DevOps is extremely powerful. So far, we have discussed how it can improve culture by improving communication and collaboration, and we highlighted some of the key ways data can hone our abilities to be more effective at tracking and planning our business value. Please stay tuned for our next installment in this series, as we discuss how data can improve an organization’s business agility – the capability of a business to rapidly respond to a change by adapting to maintain stability.

Click here to read part 4

Get all parts and more in the Data-Driven DevOps eBook