Toggle Menu

Insights / Tech Tips / Transformation Chaos: Quantify Your Company’s Impact

March 26, 2019

Transformation Chaos: Quantify Your Company’s Impact

8 mins read

Written by

Jaap Dekkinga

Sr. Enterprise Agile Transformation Lead

Agile coaching is what we love to do. There is nothing better than seeing a workplace comprised of people enjoying what they do while providing high-quality work that customers love. Many times you can feel the impact that a coach has on a team, enterprise or organization, but we like to also quantify that transformation impact with metrics. The goal of metrics is to provide all decision makers (including team members) with shared context to make good decisions. By tracking continuously, the feedback cycles will be shorter and avoid overcorrection as much as possible. Our recommendation is to look at a couple of metrics together to get real insight.

Context

The 2018 State of Agile reports that 97% of respondents’ organizations practice Agile development methods and more than 50% of respondents stated that over half of teams in their organizations are using Agile practices. Agile portfolio planning techniques went from 25% to 35%, indicating that more organizations are scaling their Agile across teams. This blog is not a how-to on using SAFe, LeSS, DAD, NEXUS or your own internal method; it describes different metrics an organization can use to track the impact of their scaling effort.

An organization is producing some value, but presumably wants to increase production and reduce elements that negatively impact how much value the organization can create. This applies to for-profit and non-profit organizations, and government agencies. Important metrics an organization can target include:

  1. Overall business value improvement (e.g. If my goal was to add more customers, am I seeing that needle move?)
  2. Happiness or joy of their workforce
  3. Customer retention, or ‘net promoter score’

Reach Your Full Value

These are some metrics an organization can use to impact possible root causes of not reaching your full potential value. Not all of the metrics will be exact; some will be gut-feel metrics. By asking input from multiple people in your organization you can lower the impact of bias on the metric. It is also an option to run small experiments occasionally to validate the groups gut-feel.

Total Number of Tickets/Stories

A gut-feel metric where you can optionally get an exact number. If an organization has a lot of tickets/stories the chance that they do just-in-time (JIT) analysis is slim.

Cycle Time

The period required to complete one cycle of an operation or to complete a function, job, or task from start to finish. Cycle time is used in differentiating total duration of a process from its run time and measured by the amount of time per unit (minutes/customer, hours/part, etc.).

Lead Time

Lead time is the latency between the initiation and execution of a process and its completion, and is measured by elapsed time (minutes, hours, etc.).

Value-Stream Map

A value-stream map is a map of the series of events that take a product or service from its beginning through to the customer. Each event will have a touch (value adding) time for completing each event plus the wait (non-value adding) time between events. This metric is related to the lead and cycle time metric.

Process Efficiency

From the value-stream map, add up all the touch time and divide it by the total calendar time from the beginning through to the customer. Each transformational improvement should result in an increase in process efficiency.

Can This Team Operate Without a Scrum Master?

A binary metric (true/false); a mature team has members that simulate and drive each other, and doesn’t need a Scrum Master. The organization abides by the agreed methodology rules (e.g. Scrum, Kanban), lessening the need for a team protector.

Average Team Size

While the ideal team size depends on a number of factors, Scrum guidelines dictate 3 to 9 people per team. If the organization has very large teams, this is a place to start making changes. Why? Check out ‘Brooke’s Law.’

Number of Dependencies Between Teams

A gut-feel metric, and optional to get an exact number. A high number of dependencies will result in more meetings and more defects, but a low number of dependencies will result in more independent teams and delivery.

Focus Factor of People

A gut-feel metric with the option to get an exact number. Someone on a high number of teams/projects will deal with more meetings and wasted time due to context switching. On the other hand, people on fewer teams/projects will contribute to an independent team environment and faster delivery overall. The technique to use includes defining containers, significant differences and transforming exchanges of a team or organization.

Rate of New Habits Adopted

A gut-feel metric with the option to get an exact number. Track how long it takes for a team to adopt a new habit they decided to implement after a retrospective.

Safety Check (Team/Program/Organization) at the Beginning and End

Out of the effective team key dynamics that the researchers identified, psychological safety was by far the most important. Researchers found that individuals on teams with higher psychological safety are less likely to leave their company, more likely to harness the power of diverse ideas from their teammates, bring in more revenue and receive high effectiveness ratings from executives. To measure a team’s level of psychological safety, researchers asked team members how strongly they agreed or disagreed with these statements:

  1. If you make a mistake on this team, it is often held against you
  2. Members of this team are able to bring up problems and tough issues
  3. People on this team sometimes reject others for being different
  4. It is safe to take a risk on this team
  5. It is difficult to ask other members of this team for help
  6. No one on this team would deliberately act in a way that undermines my efforts
  7. Working with members of this team, my unique skills and talents are valued and utilized

Happiness Index

The goal of this metric is to have a graphic representation of team member emotions during iterations. This kind of information helps the team identify what helps, or hurts, its performance during the iterations. Whatever problem the team goes through, this exercise helps them to reveal team emotions immediately.

Net Promoter Score

Net Promoter, or Net Promoter Score (NPS), is a tool that can be used to gauge the loyalty of a firm or team’s (customer) relationships. It serves as an alternative to traditional (customer) satisfaction research and claims to be correlated with revenue growth and team effectiveness.

Average Age of a Branch

This metric will give you insight into the level of continuous integration. The older the age of a branch, the harder to merge it back to the master, and the bigger the chance the changes in that branch will have an unforeseen impact on development in another branch.

Escaped Defects

Escaped defects are the number of defects that go live when a product is released. This is an indicator of the quality of a delivered product. The effort to change a defect in the product is higher.

Velocity

The metric is calculated by reviewing work the team successfully completed during previous sprints; for example, if the team completed 10 stories during a two-week sprint and each story was worth 3 story points, then the team’s velocity is 30 story points per sprint. Do not use this metric to manage a team/organization. It is very easy to mock this metric by just sizing stories larger. You can’t compare velocity between teams, since the team might work on more complex solutions, newer technologies, receive more unplanned work (e.g. production support, management reports), be a newer team or have less experience.

Story Point Cost

This metric is used to measure and track the cost of user stories and consequently the cost of Agile projects.  Story Point Cost = (FTE of team members * Average hourly rate * average hours worked * Sprint length) / average velocity. Do not use this metric to manage a team/organization.

Increase of Business Value Per Story Point

The Product Owner gives each user story a value that represents its impact on the stakeholders. At the end of each sprint, together with story point cost, you’ll have a number that can tell you how much value you’ve delivered to your customers through the eyes of the Product Owner.

This metric does not measure performance; instead, it measures impact. Ideally, your Product Owner will prioritize higher value items towards the top of the backlog and thus each sprint will deliver the maximum value possible. If you’re working on a finite project with a definite end in sight, your sprints will start out very high value and gradually trend towards delivering less and less value as you get deeper into the backlog.

Conclusion

By quantifying the impact of Agile coaching, not only are you proving your business with invaluable feedback, but you are increasing the significance of the work you are delivering on your company’s behalf to its clients. Through continuing these tracking cycles and paying attention to the myriad metrics listed above, your team is ensuring shorter feedback cycles and fewer overcorrections.

Still interested in other ways that you can make your Agile adoption work for you? Listen to our webinar, “Making Your Agile Adoption Work for You: A Deep Dive” and get specific examples for teams just like yours!

Jaap Dekkinga

Sr. Enterprise Agile Transformation Lead

You Might Also Like

Resources

Overcoming Obstacles to Continuous Improvement in Your Organization

Does driving change in your organization sometimes feel like an uphill climb? You’ve tried implementing...

Resources

Responsible AI for Federal Programs

Excella AI Engineer, Melisa Bardhi, join host John Gilroy of Federal Tech Podcast to examine how artificial intelligence...