Agile Performance Measurement

The Problem of Measurement

As more and more projects shift to the Agile methodology more focus is put on the entire team’s performance and delivery. This in turn means it does no longer matter if you as an individual have completed all your assigned tasks if the team as a whole has failed to succeed. While the approach of “we’re in this together” is definitely a large step forward when delivering value and results it becomes problematic when managers are required to do individual performance assessments of each team member.

While I do not have the perfect solution I want to give some insight in my personal experiences and opinions on this topic.

There is no Team in I

The main problem with the situation is that Agile promotes “one team” thinking which is tremendously beneficial to the performance and success of a project. On the other hand high-performers want – and to some degree – deserve to be recognized for outstanding work. The issue of if and how to reward individuals in a team environment is a tough cookie on it’s own that I’ll skip in this context.

What I do want to emphasize is that in an Agile environment the individual rating of team members is made even tougher as you technically need to separate the contributions to the team result. Also the question comes up: “If I emphasize team results and team achievements, what do I set up as hard, measurable success criteria in the Performance Assessment?”.

For both questions there is no ideal, one-size-fits-all, silver-bullet solution. But there is perhaps a way to be both Agile and be able to recognize performance fairly.

Hard Facts for Everyone

When it comes to hard, measurable business results I would always recommend to do this on a team level. This means goals like release dates, quality metrics, X amount of leads converted into contracts, etc. should always be defined for the entire team. These global goals then go into each team members’ Performance Assessment goals. The important part is that everyone on the team has to have the same shared goals. It’s not okay to cherry-pick or let team members choose their own selection.

If you cannot define goals that can be applied to the entire team then it might be worth investigating why people with widely differing goals are put together as a “team”.

It might be preferable to openly discuss and define these goals in the entire team so that everyone can contribute. In the end it’s important that every team member can commit to the shared goals and be evaluated by them.

Using this technique you can enforce the “one team” spirit while at the same time laying hard ground rules for evaluation.

Going Soft

It’s pretty obvious that successful project work is a lot more than just following a set of defined, measurable goals. It’s also HOW the team approaches new challenges, works together and gets the value delivered.

This touches upon areas such as problem solving skills, inter-personal communication, knowledge-sharing and the general work values. As these things are less easy to define and may differ from the assigned/perceived role of a team member things will get difficult at this point. I would recommend splitting Soft Skills into two groups: General team rules and individual expectation.

The general team rules should encompass how the team approaches a given situation, it’s work ethic and conduct. This includes things like “We will share knowledge” or “We will identify the root cause instead of treating the symptoms”.

If you have defined the general team values when initially creating the team AND WRITTEN THEM DOWN – good job! If not it now would be a good time to have the team define these general values and write them down (e.g. as a poster on the wall) so it is clear HOW the team accomplishes it’s goals and goes about the daily work. You should make it clear that the team will be evaluated on each individuals follow-through on these rules.

It’s important to note that these rules should be a guideline and that it is okay and normal if people are not always 100% or even 80% on track – we’re still humans not machines.

For role-based expectations, e.g. if you have a team lead, architect or someone with a special role on the team you should define the expectations as you would with every personal goal – direct and in private.
As it is clear that these expectations are tied to the individual and the individual’s role this approach does not conflict with the general team rules and you can draw the line more clearly.

Matching Expectations with Reality

When it comes to comparing the expectations with actual results the approach largely depends on what you are evaluating. The hard, measurable team goals can be checked during sprint review and delivery gates. The approach will be more or less the same as with every other business goal. For the “Soft” values and skills things get very different very fast.

Any kind of evaluation of the softer goals requires experiencing how the team acts. This means attendance at the regular team events such as Scrum Meetings and Sprint Reviews is pretty much required. How will you evaluate somebody if you haven’t seem them in action? But by all means do NOT visit the meetings taking notes on each members performance! It’s more about getting a feeling where the strengths and weaknesses are. If you are not directly involved in the project because you are “just” the line manager make sure that you regularly check in with the project manager / product owner involved. Also make sure that the PM/PO understands what criteria you are evaluating team members on so you can get meaningful feedback and not just a general “performance was pretty okay”.

Another tool at your disposal are 360° reviews. These kind of reviews take in feedback from peers, managers, people-managed and the person in question to form a better picture of the performance. If you want to do this make sure you understand how the review process works as it is quite different from the regular manager-evaluates-employee approach and also be open about your intention to use it.

In the end you should be able to combine business results with feedback on the individual’s “softer” values to create an accurate evaluation without counteracting Agile’s spirit of teamwork.

The Model

ape

On Magic Estimations

Introduction

What if you could estimate about 40 Stories in just 10 minutes?
“Impossible!” you say? Well then – read on!

What are estimations really?

They are an attempt to guess what the future will bring based on a multitude of factors such as experience, hard data, gut feelings and the likes. Estimations are also always wrong. How much depends on the subject and the person or group estimating. But nevertheless they give us a valuable guideline and basis to do a conversation.

With that established let’s take a look at Magic Estimation, originally introduced by Boris Gloger on the South African Scrum Gathering.

Idea behind Magic Estimation

As established estimations are usually wrong but a valuable guideline. However time spent on estimations helps do the right things in the right order but does not directly contribute towards the project goal. A team can be very busy for months estimating this and that and not produce a single line of code. Magic Estimation tries to harness the wisdom of the crowd to quickly estimate a large backlog – and by quickly I mean “within minutes”. It therefore reduces the time spent on estimations while retaining their value.

Prerequisites

You’ll need a backlog that has stories in a shape that they can be (at least roughly) estimated. Following the “Story Maturity” method described in an earlier article of mine can help you with that. Apart from that you’ll need a team of five to ten people and 15 minutes of time.

How-To

  1. Print out all stories (one per piece of paper) or fetch the story cards currently in our backlog – it’s important to use paper. This will not work using a computer.
  2. Shuffle the stories and put them on a stack face-down
  3. Prepare a table where you designate areas to a specific story point. You could start with a “1” area and then go upwards. The idea is that people can place the story cards in that specific area to indicate their estimation
  4. For the next 5 minutes follow steps 5 to 9
  5. Every team member picks up a story from the stack
  6. IN COMPLETE SILENCE quickly skims the story and decides in which area on the story point scale to put it
  7. NO DISCUSSIONS
  8. Whoever placed his/her story picks up a new one from the stack
  9. Repeat until all stories are placed
  10. Write the current estimation on the story paper
  11. IN COMPLETE SILENCE everyone can move any number of stories to a new location (**)
  12. As soon as the story was moved strike-through the prior estimation on the story paper and write the new one down
  13. Do this for 5 minutes
  14. Collect results

(**) As an alternative of moving the stories around you can also shuffle the stories into a new stack and repeat from step 5.

Collecting results

Stories that have only one or two estimates (with one crossed out) on them will retain that estimation for this session.

Stories that have more than two estimates on them will be gathered and need to be looked over, as there seems to be widespread disagreement on the effort required. You can schedule the discussion for a later date or address it directly.

And with that you’re done!

The Bottom Line

Magic Estimation allows you to quickly get estimations for a large stack of backlog items. These estimations are rough – yes – but they will give you a feeling where you are with your project.

They can also trigger a lot of healthy discussions without spending too much time on backlog grooming – giving you more time to work on you project instead of sitting in meetings.

Why Traditional Prioritization is Evil

Have you ever had a conversation like this:

These are your five top priority items.“
– „Which one is the most important one?“
All of them.”

Or something like this:

We have to set the priority to High – all Medium or Low items will most likely not make it anyway.”
If so you’ve likely been in Priority Hell.

Priority Hell is a place where every single item is of utmost importance, where there is no clear distinction between all the “High Prio” items, where Priorities like “Very High” or “Urgent” have reared their ugly heads. Priority Hell is also a place where everything is being worked upon but nothing gets beyond “Almost done”.
Priority Hell is a scary place.

The Root of All Evil

You might ask yourself why I cast priority in such a bad light even though it is supposed to give us guidance on how to proceed where to concentrate our efforts. And you are right that priority in general terms is GOOD. Focus is also good. But avoid flailing around aimlessly in hopes of tackling the right things first. What’s bad is the way priority is implemented and used in most of the current work processes.

For the sake of this post I’ll concentrate on the implementation of priority in Rational Team Concert – acknowledging that it is used as the main choice in project tracking and collaboration inside SWG and a lot of teams already use it. However all of the critique can be applied to other tools such as MS Project or even Excel.

Now let’s look at how priority is implemented:

You usually have three options: Low, Medium and High.
Since you can assign the category freely what becomes High or Low is entirely up to you. Sound good so far? Read on.

Opportunity Cost

What prevents you from setting everything to high? Nothing – and over time there will be a lot more things on High priority that you would imagine. The reason for this – in my opinion – is that the freedom to assign everything to high alleviates the need to make a hard choice and to weigh elements against each other. Economists refer to this as opportunity cost, meaning the lost value of the option you discarded. And make no mistake – by having to choose between two elements in terms of priority makes you take into account the risk that the lower-prioritized element will maybe not make it. It’s a lot easier to just assign both to “High” priority and assume you do not have to make a choice.

Assume that you have your list of elements and got about six elements with “High” priority – which one is the most important one? Which is the second-most important?
“All of them are important, silly!” you shout in anger – but what will that get you? Your team will start on all of them in parallel either rigorously multitasking or splitting the resources across the different elements.
Following the stream of recent publications one can assume that multitasking is one of the biggest efficiency killers of the modern work era – that’s why for example Scrum dictates that no changes to the team or plan are made within the sprint and that team members are assigned to a single team only. The same with splitting resources – assuming that your workforce is not large enough to create six separate teams to work on these items you will most likely starve certain items from manpower and knowledge.

Both factors cause work to progress slower and less efficient ultimately leading to the situation where everything is “almost” or “80-90%” done but nothing is really finished!

If you are working with a small team of about four to nine people (managers excluded) it is absolutely necessary to avoid starting multiple tasks at the same time – otherwise your resources will be spread too thin. Scrum for example suggests that instead of fanning out the ENTIRE team will tackle the top priority story from the backlog before moving on to the next one.

A Question of Leadership

For that you will need a clear distinction of priorities. A ranking has to be established that also addresses the order of items within a priority.

This ranking is a mixture of both the technical need of the item and the overall business need in terms of the project.

This integration is a leadership objective that needs to be covered by the Product Owner to be successful. On most teams the Product Owner has detailled business knowledge but only limited technical insight. For a knowledge worker this is just the other way round: She understands an item on the technical level but may lack the business knowledge. Only by integrating both the business need (following customer value, strategic alignment, etc) and the technical need (architecture requirements, technical constraints, etc.) can a reasonable piority be established.

It ultimately boils down to the team, with the Product Owner in a leadership role, to steer the project to deliver the right value at the right time and not build an imaginary ivory tower.
This steering towards delivering value is what relative prioritzation can deliver.

Relative Prioritization

A team following a ranked list of elements can focus efforts on achieving the highest, unfinished element on the list before moving on. This ensures past work is actually completed before opening the next can of worms. Also you are directly following customer (or Product Owner if you will) value – achieving the most critical elements first.

Ranking – in Three Easy Steps

Okay – “Easy” might stretch the truth a little. Doing the changes is easy living them might be more challenging.

Step 1: Get rid of traditional priorities

This first step forces you to let go of traditional priorities as it effectively blocks access to and knowledge of it.

Step 2: Use the “Ranked List” view for all planning and sprints

Then set the “Ranked List” to be used as the default view so you have a clear picture of the project priorities at all times.

Step 3: Work with ranking for prioritization only

The last step is to step up on the discipline side by only addressing the top element on the list and only that.

If the element is too small – think about bundling work differently so your entire team can work at least a day on a single story.
If some people cannot contribute to an element find out why. The most common reason is lack of skill in a particular area. If so actually take this as an opportunity to do skill transfers using techniques like Pair Programming.
If the element concentrates on a single area of work but will later be integrated with other components find out how you can do a little slice of integration now. Agile & Scrum prefer doing a “vertical slice” or “End-To-End” approach by creating a sliver of integrated functionality on all layers of the project instead of big, separated chunks.

Apart from the tackling of work you will also focus on ranking new elements as they come it. If you let them rot at the end of the list – chances are they will not be picked up in time if your team follows the “Top element first” procedure diligently. By doing ranking close to the item’s creation you are also working on a fresh memory what this item was all about – thus reducing the time to remember or fetch information later.

Both parts of Step 3 will not only ensure completion of elements in the correct order it will also surface issues you might have with team skills, work package size and alignment with the “end-to-end” approach.

The Bottom Line

Traditional prioritization in the broad “High”, “Medium” and “Low” categories leads to ambiguous item priorities. Which one of the “High” priority items should be addressed first? Should all of them be addressed at the same time? Also the addition of new requirements and work items can bloat a category severely leading to the “Everything is High” syndrome.

The result is too many open work items, everything “almost done” but nothing really finished and a lot of time and resources lost on multitasking.

A way out of this situation is to do away with the traditional prioritization categories altogether and adopt ranking.
Ranking ensures that a clear and unambiguous order of work is established, items are prioritized in relation to each other and the team as the ability to focus all efforts on finishing the most important work item first before moving on to the next one.

Start. Now.

Some of the brightest ideas never saw the light of day. Some of the best projects were never started.

Why?

I think everyone is carrying around a large selection of ideas, new ways to get stuff done and interesting projects. But I also think that everyone knows the pressures of the day-to-day business having more tasks and objectives than can be completed in the time available.

With this time constraint in mind we begin to prioritize tasks usually putting day-to-day business needs first while pushing the other seemingly less pressing matters aside. This can go on for months and even years keeping you busy but piling up a lot of unrealized potential.

It’s this unrealized potential that carries the long-term benefits, such as learning something new, creating something magnificent or treading the road less taken. And it’s these activities that keep you in the game and ensure your division, your team and ultimately you stay up-to-date and relevant.

So why are we not doing all of this if it is so important?

The Psychological Hurdle

The problem with new things such as projects is that they are always something unknown. And the unknown can be pretty scary. You are leaving the proverbial comfort-zone and risk spending your scarce time and resources on something that may not pay off.

And looking at a lot of the projects and learning activities we’re talking about a lot of time and resources. Looked at the CISSP or PMP certification? Tried running up a side-project recently? That’s months of work. „I don’t have the time for that!“ you might be thinking and you are right.

Wait – what?

If you take a look at the effort as a whole and all the unknowns attached to the project that’s a lot of commitment demanded from you. And it’s totally okay to be unwilling to sign up for the entire package. But you don’t have to. Here is where Agile can help you in your personal and professional development.

 Agile and Scrum twist the notion of traditional project planning and commitment around by acknowledging that committing for a large amount of work over a long time period is risky – so you don’t do that.

 Instead you create a short term goal. Something that is maybe 14 or 30 days away.

You only commit to something that you can achieve in that short amount of time – as little as that may be. And you only commit the resources you can muster for that short amount of time.

 This may mean that you only commit to work on a single chapter of the CISSP/PMP certification guides or that you only do a small prototype for your project. You don’t write a book, you write a chapter or just a single page.

The goal is to get you started as quickly as possible.

Don’t stand at the foot of the mountain staring up – make the first few steps and then look back at what you have accomplished so far.

 If you find out that you are going the wrong way or don’t like where you are going – stop and look at other choices. That way you are always in for just a few steps and not committed until the bitter end.

But if everything looks okay and you’re feeling good – by all means, go a few steps more.

Empirical Improvement

Projects – by definition – are always something new.

We already talked about that something new is usually scary because it carries a lot of unknowns. Project Management theory (and practice) tells us that at the beginning of a project we know the goal we want to reach but have virtually no knowledge how to do this best.

The notion of planning everything beforehand is something that works for well-understood processes. This is called the „defined“ approach.

You ensure that for fixed given input you will always get a specific output when processed. Most of mass-manufacturing works by using defined approaches. Human nature, especially when one is risk-adverse, favors having a clear plan and defined parameters beforehand.

Projects break new ground so the way to reach the goal is usually not very well-understood, sometimes even the goal is hazy. Trying to define everything beforehand is trying to shepherd cats – it does not work. It’s ironic that at the point of least knowledge we try to do most of the project planning.

So let’s not do that anymore.

Agile and Scrum favor the „empirical“ approach.

As described by Ken Schwaber in his standard book „Agile Software Development with Scrum“ the empirical approach focuses on observation and learning instead of following a strict plan.

 In concrete terms you start your project based on the knowledge and assumptions you have now let it run for a while and then compare the results with your expectations. Through this comparison you learn more about the goal and the road to reach it. You also learn a lot about how successful your way of working is.

The important part here is that you actually spend the time on reflection and learning.

 I know of a few companies that have implemented ISO quality processes – including reviews – but apart from doing the reviews there is little outcome. Review results are nodded-off and then quickly tucked away. As a result you get the worst of both worlds: The added work of following the process and the lack of quality by not learning to improve.

 Agile and Scrum also recognize that mistakes will be made. Making mistakes is human. Making mistakes is okay.

So instead of focusing on trying to be flawless the goal is to take a look at why things went wrong and to look early and often. The earlier you find out that something’s going wrong the more time you have to act and improve.

The Bottom Line

The iterative, empirical approach of Agile & Scrum allows you to tackle large, risky projects without having to commit yourself until the bitter end.

 Instead you set yourself small short-term goals and only commit yourself for that time frame.

At the end of the time frame you take a look back and check if you are satisfied with where you are.

If you are okay and feel you are going in the right direction – set up another short-term goal and continue. If you feel you are going the wrong way you have the freedom to stop and reorient yourself.

 By adopting the empirical approach you do not try to plan for every step of the journey – just for the few steps ahead. You observe and learn as you progress adjusting your course and approach based on the experiences you make.

 With the combination of these two concepts you can quickly get to the fun part:

Getting things done. Achieving (small) goals. Going home at the end of the day knowing that you achieved something valuable.

So start. Now.