Accelerate Chapter 2 Discussion Points

Moving on to chapter 2 of Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations, here's a list of bullet points for discussion:

• We wanted to discover what works and what doesn't in a scientific way, starting with a definition of what "good" means in this context
• Measuring performance in the domain of software is hard
• The flaws in previous attempts to measure performance
• Many other measures in general suffer from two drawbacks:
• They focus on outputs rather than outcomes
• They focus on individual or local measurements rather than team or global ones
• Three examples:
• Lines of code
• Rewarding developers for writing lines of code leads to:
• Bloated software
• higher maintenance costs
• higher cost of change
• Minimizing lines of code isn't ideal, either
• In the extreme, leads cryptic code that would be clearer if written with more lines
• Velocity
• Velocity is designed to be used as a capacity planning tool
• However, some managers have also used it as a way to measure team productivity, or even compare teams, which has several flaws:
• Velocity is a relative and team dependent metric, making them incomparable
• When used as a productivity measure, team inevitable game their velocity
• This can lead to inflated estimates and being uncooperative with other teams
• Utilization
• High utilization is only good up to a point
• Once utilization gets above a certain level, there is no spare capacity (or "slack") for unplanned work, changes to the work, or improvement work
• This results in longer lead times to complete work
• Measuring software delivery performance
• A successful measure of performance should:
• Focus on a global outcome to ensure that teams aren't pitted against each other
• Focus on outcomes, not output (shouldn't reward people for large amounts of busywork that doesn't achieve organizational goals)
• Four measures of delivery performance:
• Delivery lead time
• Time it takes to go from a customer making a request to the request being satisfied
• When we need to satisfy multiple customers in potentially unanticipated ways, the lead time has two parts:
• The time it takes to design and validate a product or feature
• high variability ("fuzzy front end")
• The time to deliver the feature to customers
• implemented, tested, and delivered
• easier to measure and lower variability
• Shorter product delivery lead times:
• enable faster feedback
• allow us to course correct more rapidly
• allow better responsiveness to defects or outages
• Measured as time to go from code committed to code successfully running in production
• Point to consider: How does code deployed to production behind a feature toggle count? Does the toggle need to be turned on for it to count?
• Deployment frequency
• Closely tied to batch size, but batch size is difficult to measure, and deployment frequency is easy to measure
• Reducing batch sizes:
• Reduces cycle times and variability in flow
• Accelerates feedback
• Reduces risk and overhead
• Improves efficiency
• Increases motivation and urgency
• Reduces costs and schedule growth
• Measured as software deployment to production or to an app store
• Time to restore service
• It is important that as performance improves, that it doesn't come at the expense of stability
• Traditionally reliability is measured as time between failures, but with complex software systems, failure is inevitable
• So the question then becomes: How quickly can service be restored?
• Change fail rate
• What percentage of changes for the primary application or service they work on:
• Result in degraded service, or
• Subsequently require remediation
• Lead to service impair or outage
• Require a hotfix, a rollback, a fix-forward, or a patch
• The research shows that high performers do well on all four points, and low performers do poorly on all four points
• The next question is: Does software delivery performance matter?
• The impact of delivery performance on organizational performance
• The research shows that high performance organizations were twice as likely to exceed goals in profitability, market share, and productivity than low performers