Agile coaching is what we love to do. There is nothing better than seeing a workplace comprised of people enjoying what they do while providing high-quality work that customers love. Many times you can feel the impact that a coach has on a team, enterprise or organization, but we like to also quantify that transformation impact with metrics. The goal of metrics is to provide all decision makers (including team members) with shared context to make good decisions. By tracking continuously, the feedback cycles will be shorter and avoid overcorrection as much as possible. Our recommendation is to look at a couple of metrics together to get real insight.
Context
The 2018 State of Agile reports that 97% of respondents’ organizations practice Agile development methods and more than 50% of respondents stated that over half of teams in their organizations are using Agile practices. Agile portfolio planning techniques went from 25% to 35%, indicating that more organizations are scaling their Agile across teams. This blog is not a how-to on using SAFe, LeSS, DAD, NEXUS or your own internal method; it describes different metrics an organization can use to track the impact of their scaling effort.
An organization is producing some value, but presumably wants to increase production and reduce elements that negatively impact how much value the organization can create. This applies to for-profit and non-profit organizations, and government agencies. Important metrics an organization can target include:
- Overall business value improvement (e.g. If my goal was to add more customers, am I seeing that needle move?)
- Happiness or joy of their workforce
- Customer retention, or ‘net promoter score’
Reach Your Full Value
These are some metrics an organization can use to impact possible root causes of not reaching your full potential value. Not all of the metrics will be exact; some will be gut-feel metrics. By asking input from multiple people in your organization you can lower the impact of bias on the metric. It is also an option to run small experiments occasionally to validate the groups gut-feel.
Total Number of Tickets/Stories
A gut-feel metric where you can optionally get an exact number. If an organization has a lot of tickets/stories the chance that they do just-in-time (JIT) analysis is slim.
Cycle Time
The period required to complete one cycle of an operation or to complete a function, job, or task from start to finish. Cycle time is used in differentiating total duration of a process from its run time and measured by the amount of time per unit (minutes/customer, hours/part, etc.).
Lead Time
Lead time is the latency between the initiation and execution of a process and its completion, and is measured by elapsed time (minutes, hours, etc.).
Value-Stream Map
A value-stream map is a map of the series of events that take a product or service from its beginning through to the customer. Each event will have a touch (value adding) time for completing each event plus the wait (non-value adding) time between events. This metric is related to the lead and cycle time metric.
Process Efficiency
From the value-stream map, add up all the touch time and divide it by the total calendar time from the beginning through to the customer. Each transformational improvement should result in an increase in process efficiency.
Can This Team Operate Without a Scrum Master?
A binary metric (true/false); a mature team has members that simulate and drive each other, and doesn’t need a Scrum Master. The organization abides by the agreed methodology rules (e.g. Scrum, Kanban), lessening the need for a team protector.
Average Team Size
While the ideal team size depends on a number of factors, Scrum guidelines dictate 3 to 9 people per team. If the organization has very large teams, this is a place to start making changes. Why? Check out ‘Brooke’s Law.’
Number of Dependencies Between Teams
A gut-feel metric, and optional to get an exact number. A high number of dependencies will result in more meetings and more defects, but a low number of dependencies will result in more independent teams and delivery.
Focus Factor of People
A gut-feel metric with the option to get an exact number. Someone on a high number of teams/projects will deal with more meetings and wasted time due to context switching. On the other hand, people on fewer teams/projects will contribute to an independent team environment and faster delivery overall. The technique to use includes defining containers, significant differences and transforming exchanges of a team or organization.
Rate of New Habits Adopted
A gut-feel metric with the option to get an exact number. Track how long it takes for a team to adopt a new habit they decided to implement after a retrospective.
Safety Check (Team/Program/Organization) at the Beginning and End
Out of the effective team key dynamics that the researchers identified, psychological safety was by far the most important. Researchers found that individuals on teams with higher psychological safety are less likely to leave their company, more likely to harness the power of diverse ideas from their teammates, bring in more revenue and receive high effectiveness ratings from executives. To measure a team’s level of psychological safety, researchers asked team members how strongly they agreed or disagreed with these statements:
- If you make a mistake on this team, it is often held against you
- Members of this team are able to bring up problems and tough issues
- People on this team sometimes reject others for being different
- It is safe to take a risk on this team
- It is difficult to ask other members of this team for help
- No one on this team would deliberately act in a way that undermines my efforts
- Working with members of this team, my unique skills and talents are valued and utilized
Happiness Index
The goal of this metric is to have a graphic representation of team member emotions during iterations. This kind of information helps the team identify what helps, or hurts, its performance during the iterations. Whatever problem the team goes through, this exercise helps them to reveal team emotions immediately.
Net Promoter Score
Net Promoter, or Net Promoter Score (NPS), is a tool that can be used to gauge the loyalty of a firm or team’s (customer) relationships. It serves as an alternative to traditional (customer) satisfaction research and claims to be correlated with revenue growth and team effectiveness.
Average Age of a Branch
This metric will give you insight into the level of continuous integration. The older the age of a branch, the harder to merge it back to the master, and the bigger the chance the changes in that branch will have an unforeseen impact on development in another branch.
Escaped Defects
Escaped defects are the number of defects that go live when a product is released. This is an indicator of the quality of a delivered product. The effort to change a defect in the product is higher.
Velocity
The metric is calculated by reviewing work the team successfully completed during previous sprints; for example, if the team completed 10 stories during a two-week sprint and each story was worth 3 story points, then the team’s velocity is 30 story points per sprint. Do not use this metric to manage a team/organization. It is very easy to mock this metric by just sizing stories larger. You can’t compare velocity between teams, since the team might work on more complex solutions, newer technologies, receive more unplanned work (e.g. production support, management reports), be a newer team or have less experience.
Story Point Cost
This metric is used to measure and track the cost of user stories and consequently the cost of Agile projects. Story Point Cost = (FTE of team members * Average hourly rate * average hours worked * Sprint length) / average velocity. Do not use this metric to manage a team/organization.
Increase of Business Value Per Story Point
The Product Owner gives each user story a value that represents its impact on the stakeholders. At the end of each sprint, together with story point cost, you’ll have a number that can tell you how much value you’ve delivered to your customers through the eyes of the Product Owner.
This metric does not measure performance; instead, it measures impact. Ideally, your Product Owner will prioritize higher value items towards the top of the backlog and thus each sprint will deliver the maximum value possible. If you’re working on a finite project with a definite end in sight, your sprints will start out very high value and gradually trend towards delivering less and less value as you get deeper into the backlog.
Conclusion
By quantifying the impact of Agile coaching, not only are you proving your business with invaluable feedback, but you are increasing the significance of the work you are delivering on your company’s behalf to its clients. Through continuing these tracking cycles and paying attention to the myriad metrics listed above, your team is ensuring shorter feedback cycles and fewer overcorrections.
Still interested in other ways that you can make your Agile adoption work for you? Listen to our webinar, “Making Your Agile Adoption Work for You: A Deep Dive” and get specific examples for teams just like yours!