Figure 1 – Defining Your Metrics Framework
Setting up the metrics that helps Scrum Team can indeed be a very difficult job and, in this article, I will explore some things to keep in mind while you are setting up your Metrics
- Metrics Are Not About Keeping Things Green – In my experience I have seen quite a lot of Scrum Teams where the focus of metrics reporting is to show how what we are doing is working or in other words we are all green- this usually stems from metrics that are defined in agile contracts and are tied with payment for services rendered. This makes the Scrum Team focus on ensuring that there always seen green. In such a case the focus cannot be on what we need to do in order to improve. I usually guide the teams not to have these service level agreements in their Agile Contracts – I rather focus on having metrics that are measure the right things like value of work delivered.
- Focus on End-to-End Metrics – It is usually a better idea to have metrics which measure the value end to end across the value chain rather than measuring value at interim points. To give an example, it is better to have metrics that capture into and time required for a feature rather than tracking time for a specific task like coding or testing. Another example can be, number of zero defects features delivered is a better Metrics than number of defects identified during testing phase. When defined in this manners Metrics encourage Scrum Team to work as a unit. It becomes a challenge to define such metrics in a multi-vendor scenario.
- Measure Only What You Want to Improve – Metrics Should NOT be cast in stone – Often, I see very complicated and detailed metric structures that texts up a lot of time and energy for the Scrum Team to create and maintain the data. These metrics frameworks often are collecting data for it sake without a concrete plan of how the data will be utilised in order to help the Scrum Team to be more effective and efficient. Such metric frameworks make it difficult for the Scrum Teams to derive meaningful in inferences out of huge amount of data being collected. I have seen that it instead works better to have a smaller number of metrics that the Scrum Team has to track. This way the team can actually use the data being captured. Some of the most effective organisation and Scrum Teams are known to choose their focus for the upcoming period of 6 months to an year and then decide the metrics that will help them achieve their goals for the said period. This also aligns with the Scrum Value of Focus
- Avoid metrics that Compare one team or one person against another – Agile believes into collaboration. For a Scrum Team working collaboratively is the best way to ensure frequent delivery of valuable software. If the metrics are structured to measure an individual performance or productivity it may sometimes lead to people working against each other to protect their own Metrics level. It is a better idea to measure value delivered as the whole team. The same principal can be applied to creating a competition between the teams- for large products requiring many teams it can hamper the collaboration amongst Scrum Teams. It works better if the teams are measured against their own last performance and encourage to have a continuous improvement in their way of working rather than comparing against other teams.
- Trends Need To Be Monitored – NOT individual numbers – Sometimes a particular metric may show a bad value. This can happen because of multiple transient reasons and it may not be worth it to really worry about an isolated value or reading in a metric. It is much more useful to monitor the trends and analyze if the Scrum Team is going in the right direction.