Management
Let’s talk about Team productivity through the prism of metrics

Published on
May 17, 2016
by Artem Borodin

Understanding the productivity of a team working with scrum processes is key to planning the number of story points that the team can and should perform during sprint.

It is important to do 3 main things:

  1. Always evaluate the stories being chosen by the team to be put into the sprint or product backlogs.
  2. During the sprint take into account all the completed story points in order to adequately assess the status of accomplishing all the iterations of sprint.
  3. Use metrics as a param to plan the next iterations with the purpose of increasing the productivity of the team.

If this is your first time on sprint don’t worry we will lead you through it.

How do we evaluate user stories?

First, remind your development team that the assessment of “Points” (‘parrots’, ‘story points’) are comparative not quantitative (we are not looking for something like time or number of words).  Such as an apple is comparative to another apple but a pineapple is not comparative to an apple.  The ‘size’ is in the story point (apple) not in the time it takes to complete (grow) the story point (apple).

Second, conduct a calibration.  Is the point ‘size’ an apple, desk, or jet plane?  With the team determine the ‘size’ (Work x Difficulty x Hesitation) of what the story points are.  For this you’ll need to take a query of the work your team has already conducted for the last 3 to 4 weeks by (if you don’t have any previous story points I’ll explain what you should do later on):

  1. I suggest that you print out on separate cards or write down a code (ID) on separate pieces of paper a short name of the story so everyone knows what story is being talked about.  
  2. Next, with the team calculate the ‘size’ of the ID cards. ‘Size’ = Work x Difficulty x Hesitation.  For tips in how to calculate I discuss T-Shirt size a little later.
  3. Then place the ID cards in a row starting with the smallest ‘size’ to the largest, having 7 to 8 stacks or baskets. Some cards may have the same or near to the same ‘size’ and could be in the same column or stack.  Remember that the ‘Size’ = Work x Difficulty x Hesitation.
  4. Layout “Planning Poker Cards” or somethings similar like a Fibonacci Scale, place the “poker cards” out from lowest to highest, as shown in the figure below.  Place the ID cards beneath the “poker cards” as shown below.

Story line

Now, the “Planning Poker Cards” give you points for your stories.  An apple for an apple.

Don’t worry if some stories have to be placed in ‘the same basket’. Remember, we don’t measure the size in metres, kilos or other units of measure. ‘Story Points’ are not dimensions or hours, the thing that matters is the relative comparison of the queries among each other and their range on the Fibonacci scale.

Extra tips:

  1. When using “Planning Poker Cards” don’t use numbers larger than 13 or 20 . If you assess the work you’ve already done during the iteration you should leave space on the scale for big queries that are bigger than your current iterations.
  2. Don’t believe that there is work with no value (zero efforts). Even the smallest request takes some time. Use the card with a 0.5 value for this.
  3. Now that you have some scale where to place requirements or stories, every time you need to assess a new story the team members can compare it with the others that have already been evaluated.  This is the assessment of comparison and now that this is done you can relax and play poker. Hurrary!
  4. Early I said “if you don’t have any previous story points I would explain it”.  I’m often asked what a team should do that that hasn’t worked together before or this is a new type of project for them since there is no way for them to have queries from the last 3-4 weeks. I suggest that you take the requirements that have to be evaluated and place them in a line according to your understanding of size and even if later you realize that you’ve made a mistake you can rearrange them to the more appropriate size.  During each new sprint you will create better assessments of your story points and the productivity of your team.
  5. When the team calculates the ‘size’ of the ID cards. I recommend to use the following task sizes according to the t-shirt sizes”: small story 2 points; medium 4; and large 8.  With each new sprint your assessment will become more refined.

estimations

What are the main ways to calculate units of the team besides story points?

Hours

I don’t really recommend this option to follow. You may start from this point but finally I advise you to come to the story points planning (comparative planning not quantitative).

The explanation is simple of why hours become useless from an assessment point of view and when taking progress of the team into account. Imagine that a number of tasks during the sprint had increased so in this example the number of hours had increased as well with tasks that have to be done. The team pulls all its efforts together and finished the sprint successfully to the original goal. If this becomes the norm in time the team will to often view these extra tasks as overtime, since they are based on hours even if they really didn’t work overtime.  Soon the team becomes demotivated no matter how you try to explain your hour system.

Percentage of finished story points

I recommend this way of measuring progress to all the other ones. You should not only mark the number of completed story points but also calculate it as a percentage of the total number. It helps the team to focus on performing the functionality but not on the tasks. Besides that, this method helps the development of cross-functionality in the team. Like it is implemented in our first product for Slack Sprinterbot.

opengraph

Use both ways of measuring progress (simple number of story points and  % of their completing), placing values on one graph in different colours. This can help you get more balanced information.

How to track the team progress during the sprint?

The main scrum metrics allowing you to account for completed work and predict the sprint finish is theBurnDown Chart.

You need to put the number of the story points to the left on the Y-axis (whatever units you measure them in), and the days of iteration on the X-axis.

Sprint_Burndown

So on the 1st day of the sprint you need to place the number of the story points that you will have to execute, on the 2nd day a number of tasks should either decrease or increase but by the end of the sprint you should see a zero value to the story points.

Then we can freely expect that this iteration can be credited towards the assets of the team.(The final result will be demonstrated in the sprint review, see my other information).

You can ask:

“How can the number of story points increase during the sprint since they are supposed to only burn down?”

Not necessarily. Remember that a team in the understanding of Agile is a highly self-organized organism that takes the stories to the sprint by itself. During the sprint a team may see the necessity of some of the taken work decomposition that will lead to increase in the total size of the story points (tasks).

The team should decide whether they are ready to fulfill a new number of story points or if they should decide to eliminate some things from the iteration. The most importantly is that the elimination of some story points in the current iteration should not change the sprint goal.

Sometimes a totally opposite situation can take place during the sprint when the team gets to some work and realizes that this work is either unnecessary or the team has overestimated it. In this case the size of story point may decrease sharply.

You can find out more about these situations in my material about iteration planning. So, everyday after conducting daily stand up meetings you assess the work left for an iteration by marking on the diagram the number of story points burned.

What other metrics can be used for productivity planning?

Once you have a number of sprints, or even two or three, under your belt the main metrics of team productivity in Scrum that should be counted when planning your next iteration is: Velocity.

On the X-axis we place the number of sprints, on the Y-axis we place the story point numbers which have been completed and planned that are associated to each sprint (showing them in different colours on the bar chart).

gh_6-2-1_velocity-chart

Now, we can quickly view our historical productivity and use it as a reference point for our next planning of iterations.

The following 2 graphs allow us to plan with good accuracy: when we will finish tasks of a current iteration; and when a next release can be expected (assuming release consists of some sprints).

BurnUp Chart

It predicts when all the volume of current sprint works will be completed and whether it’s possible at all.

It is built with 2 index lines on one graph.

On the X-axis you should place iterations that are parts of the release (ex. Iteration 1,2,3)

On the Y-axis you should account for a number of story points that have been completed in each iteration so that this number is rising cumulatively.

burnup

The second graph, differing in colour from the first one, shows a total number of story points that is planned for the release. So in case your requirements do not really change and you together with your team positively evaluate the work to be done it will be just a straight line.

But more often it will be jagged making a new line for some period of time because planned story points had increased or decreased.

Thus, when knowing how many story points the team complete each iteration, the team can foresee when 2 graphs will cross in 1 point.

And knowing the given length of the sprint you can understand whether you will be on time for the deadline you’ve chosen for closing all the issues for the release or at your current rate of speed you will finish all the tasks late.

A similar graph can be built also for some particular sprint. In this case on the X-axis there will be the days of current sprint instead of the numbers of iterations. The second graph will demonstrate then a total value of backlog story points for a current sprint.

You will understand if you manage to finish all the tasks in sprint time or it’s the right time to start adjusting actions.

Release Burndown Chart

Similar to Sprint Burndown chart we have reviewed in the past chapter, Release Burndown chart shows how much work has to be done till the release.

On the X-axis there are iteration numbers, on the Y-axis there are the story point numbers that are planned for the release. Using the method of decreasing the remainder of a number of story points left till release after completion of every sprint that should be plotted along the axis. Such as you had 360 story points planned for the whole release. Then from the start of the first sprint you plot on the Y-axis a point at 360. During the first sprint you had completed 90 story points. Also there were some requirements decomposed and some changes in requirements took place that led to the backlog increase in 60 story points by the end of third sprint.

release burndown

Then after finishing your first iteration the next point that will be plotted on your graph will be 360-90=270.

But after finishing your third iteration the next point that will be plotted on your graph will be 190+60=250. So, it came as an extra scope since second sprint started.

With this you can easily see what productivity and sudden requirements influence on the common backlog you have during your work and when you will be able to complete the release at the current productivity rate.

Remember that Sprint BurnDown chart is drawn up similarly but predicting a completion of all the tasks during the current sprint.

What to do if there is too much work and the team realize that they won’t manage to finish all the tasks by the end of the sprint?

Actually everything is very simple in this situation. You realize that you can’t reach the whole goal of the sprint because of the big number of tasks you took for this sprint. Your burndown chart is telling you that you’re in the middle of an iteration but have completed only 25% of all the story points. Therefore, the only right option for you is to get rid of the tasks from the sprint that the team haven’t started yet and the ones that the team realizes it won’t complete on time.

sprint fall

You need to focus only on those stories that you and your team will be able to guarantee the completion to and present as a ready delivery in the end of the sprint.

Remember, this is the main purpose of the development by Agile. You should give some incrementally completed part of the functionality that an owner can already try and the revision of which won’t be needed.

Yes, probably using such an approach you keep consciously following the sprint even knowing that you’re not going to reach its goal and in fact, it will fail. Of course, all this can upset the team.

However, only this particular approach gives a maximum positive effect for business that is your customer of those stories you put in a backlog of development.

And you as a team first of all have to make the priority not for the formal completion of the sprints but for the bringing maximum value to your business.

That’s all for today. Soon to come more interesting stuff.

Your Standuply.