Agile Chronicles

Stuck on Velocity...

Okay... so it seems I am a little stuck on velocity this week. Maybe with one more post I can get this out of my system and move on to other topics. Let's talk a little more about what velocity is, what it is not, and how it should best be used by a project team.

Traditional Estimating

Most projects I worked on prior to discovering agile would start with a long list of requirements. The project manager would reach out to each of the resource managers for an estimate. These estimates would be done in real hours and used to calculate how many people were needed for the project, and by extension, how much the project would cost. In my experience, the business almost never liked the estimate, and would always ask us to do something to lower the cost.

As a project manager, I would go back to the resource managers and start the negotiation process. The most extreme example of this I ever participated in lasted almost four months. We negotiated scope, tried to do a bunch of up front design work, made assumptions about the project, and were finally able to cut the estimate down by about 50%. The funny thing was, I had been largely negotiating with each manager individually, and none of them really had much understanding about how the other managers were thinking about solving the problem. Their estimates were fundamentally out of sync.

We literally spent four months, and a bunch of valuable time from critical resources, haggling over a number (that at the end of day) turned out to be total fiction. The reality was that we had no idea how big the project would actually be. Furthermore, we had no idea which team members would actually be doing the work, so coming up with a reliable end date was nearly impossible. The only thing we could do was create a project plan, with the estimates we had available, and hope for the best.

Agile Estimating

Agile teams do estimate projects but use mechanisms that reflect the uncertainty inherent in the process. You have probably heard about the idea of story points and planning poker. This is one of the most common methods of agile estimating but not the only method. The general idea behind agile estimating is that you start by identifying a small, relatively understood requirement and assign it an arbitrary value of one (1). Each feature in your product backlog is evaluated against the well understood requirement. We ask ourselves, as a team, do we think the new requirement is 2x, 3x, or 5x more complicated than the one we are comparing it against?

The first requirement gets a point value of 1, the next requirement might get a point value of 3, while a third requirement might get a point value of 5; all based on the team's understanding of the relative complexity of each feature.

At this point, we have the size of the backlog in points, but it doesn't really tell us anything. Since points do not equate to hours, we still don't know how long the project is going to take. Some teams will do detailed estimating on the first few features to get a general idea of the size of each backlog item, but we really won't know how many points we are able to build each iteration until we start building them.

This is where velocity comes in.

The product owner prioritizes the features based on business value, the team gets to weigh in based on technical complexity and risk. Collectively, the team sets the build order of the features and gets busy writing software. Each iteration, the team will deliver some number of features, and keep track of the point values assigned to each item. The sum total of all the point values, associated will all completed features, is the team's velocity. It is fundamentally a measure of the team's throughput iteration over iteration.

Because we know the overall size of the backlog (in story points) and how many of the features the team can build each iteration (in story points), we are able to calculate when the project will be completed:

Iterations to Complete = Backlog Size/Points per Iteration

The key concept here is that traditional estimating almost never results in a reliable project cost indicator. Only by building working software in short cycles, and measuring our progress, will we have the necessary information to know how long the project is going to take. If you didn't read my last post on progressive estimating, this might be a good time to go back and take a look. Because we won't know how much we can do, until we actually start doing it, we might have to insert a few checkpoints where the business gets to recalibrate the project or decide to pull the plug.

The traditional view of velocity is that it is specific to the team that is doing the estimate. One team might give a feature three (3) points while the next team might estimate the same feature at a five (5). As long as the team doing the estimate is the one doing the work, this is not a problem. Remember, points are estimates of relative complexity... of relative size. The actual value does not matter. The only thing I care about is the team's throughput against the overall size of their backlog. It is the only thing I need to know to determine if my project is progressing according to plan.

Normalizing Velocity

That said... what happens if I have many teams that are working together to deliver a large, complex project. If one team calls a feature a three (3), and the next team calls a similar feature a ten (10), it might get difficult to communicate performance on the overall initiative. The team that uses the larger point values would obscure the performance of the team with the smaller point values.

To answer this question, I have seen many teams normalize a point to a common unit of measure. For example, my team will often think of a point as a single person day. We are still using lightweight estimating techniques, we just use the normalization value as a starting point for conversation. This approach can level the playing field and enhance communication between teams that are working on various parts of the same initiative.

While this normalization can have benefit on large scale agile implementations, there is some risk.

As soon as we start equating a point to a standard unit of time, we start trying to do math on velocity. The discussion usually starts by asking how we can improve the velocity of the team. Well... if the team has four people that can deliver 20 points per iteration, we could add a fifth person and increase velocity to 25. Or maybe... we find that the same team of four is only delivering 17 points per iteration. We want to know what those deadbeats were doing with my missing three points of capacity.

My personal opinion is that it is perfectly okay to normalize velocity across teams. As managers, we need to recognize what velocity is measuring... and what it is not measuring. Velocity is measuring throughput against an abstract estimate, it is not measuring productivity... and certainly not time spent on task. It is very common to see situations where the point estimate was 20, the detailed estimate was 25, the actual hours spent was 30, and the features delivered was 15. The team can use these numbers to get more consistent estimates, but again... all I care about as a manager is throughput and how I can help the team improve over time.

Even in an environment where we are normalizing velocity, there are many factors that can contribute to velocity variations between teams. Team size is an obvious consideration, but what about skillset and experience? We should consider team chemistry... we should ask if the team dependent on other organizations to get their stuff done. Maybe they've encountered impediments that have slowed their progress. This is all stuff that contributes to team velocity.

I coach teams to measure velocity so we can measure what is... to get honest information about how the project is progressing... then look to the other factors to determine how to make it better. The last thing we want to do is to inadvertently incent the team to manipulate the numbers to make the data look good.

Progressive Estimation and Tollgates

Kelly Waters from All About Agile picked up on my post yesterday about project velocity. Kelly made a great point about how we can use a toll-gate approach for estimation and project funding.

Here is Kelly's comment:

In some methodologies - for example RUP - there is an initial stage of the project ("inception") to do some high level scoping before seeking approval for full funding. It's also quite common in waterfall projects to complete the reuirements analysis before going on to request full funding. In these cases, only the initial stages need to be funded and then the team knows more by the time they ask for the rest of the money.

This is an essential concept for large scale agile development, and here is why.... coherent software architecture, UX and UI design, and requirements are not going to just emerge from multiple small teams working independently on a significant, enterprise level initiative. Without some degree of lightweight, up-front planning (to integrate the activities of several teams working together) it is unlikely that we will converge on a well integrated product.

Schwaber talks about this when he talks about iteration 0. Leffingwell talks about this when he refers to an architectural runway. In my opinion, these guys are dancing around the idea of Inception and Elaboration. This almost has to be done because RUP is such an unpopular topic in the agile community... otherwise people tune out. Ambler is more direct explaining these concepts when he talks about the Agile Unified Process.

By introducing these higher order agile concepts, we can help teams learn to effectively structure large scale agile implementations, and furthermore, help teams understand what decisions need to made and when... from both a busines and a technical perspective. A year or so ago (before I joined VersionOne) I did a post called the Agile Heart of the Unified Process (back when my personal agile blog had no subscribers whatsoever).

Rather than reinvent the wheel, I am going to grab some text from that article and update with some commentary to tie back to Kelly's original comments... the text in bold was added today.

Inception

The goal of Inception is to minimize business risk. This means that you have to understand the vision for what you are trying to create, have a high level understanding of the requirements, and have defined the characteristics of the systems architecture you intend to build.

Begin Inception with a single Scrum team. The team should at a minimum have a ScrumMaster, a Product Owner, an Analyst, a Quality expert, and an Architect. The product backlog will include the features necessary to create the Vision, Use Case Inventory, and Candidate Architecture. It may be necessary to include features in the Inception backlog that prove key aspects of the business vision.

Inception is the least code focused of all the RUP phases. This is due to the requirement to have an idea of where you are going before you begin the project. Since Inception is the least focused on creating working software, and without working software the team is not mitigating risk, it is in the team's best interest to make decisions quickly and get on with the business of delivering working software.

At the end of Inception, there is a business decision that must be made. Do we have a strong enough product vision, and a strong enough architectural vision, to fund the next increment of product development. This is a good time to reestimate the backlog and decide if the value proposition of the project still holds.

Elaboration

The goal of Elaboration is to minimize technical risk. This involves choosing the smallest subset of features that will prove out the significant technical assumptions made during Inception. At the end of Elaboration, the system should be fully functional with the subset of features you have chosen to build. The system should be fully tested and the team should be confident that the architecture is stable.

During Elaboration, you may still have a single Scrum team. It is likely that you will add several more senior level technical folks to help begin building the system. The product backlog includes the features necessary to prove the application architecture and to deliver any non-functional components required to build out the system. It is important that the Elaboration backlog contain features the team will need to collaborate at scale. Source control, build environments, automated testing infrastructure, instant messaging, and collaboration software should not be overlooked when preparing to scale an Agile team.

At the end of Elaboration, we have another business decision. Have we built enough of the application to validate our technical assumptions. Can we move forward with the project, scale into multiple agile teams, and move forward with confidence we have a workable and proven solution? This is the next point where you need to reestimate the backlog. After proving the solution, updating the backlog, reestimating based in what we have learned, do we have a business case for moving forward?

Construction

The goal of Construction is to mitigate logistical risk. This is where the bulk of the software is created and where the Agile team really begins to scale. The core team that helped build the foundation of the system during Elaboration is broken up to seed the newly created Scrums. Each team is organized around a major component of the architecture or a significant feature set within the overall scope. Each team is a complete cross-functional entity that is responsible for delivering its part of the system. These teams are coordinated by a master team that includes a more senior ScrumMaster, Architect, its own technical staff, and the Technical Leads and/or ScrumMasters from the component teams.

At this stage, the backlog is completely feature driven, progress is extremely measureable, and the teams are focused on delivering the bulk of the business value to the customer. As is true during every phase, the code delivered at the end of each Construction iteration is fully tested and integrated with the rest of the system.

Construction is the most like the prototypical agile project as described in most of the common literature. We have several independent feature teams working in concert. Each is tracking velocity and burndown and making product adjustments as necessary to converge into a solution that delivers the most business value possible. The business is involved actively assessing progress iteration over iteration deciding when enough of the product has been built to take the solution to market.

Transition

This phase deals with training and hand-off to the customer, final user testing, and resolving any issues that were not caught earlier in the process.

During this phase, the team should be able to scale back down to a single cross-functional team or two. The backlog is related to the remaining documentation, training, and defect resolution features remaining to get the product in the hands of the customers.

I think of transition like many talk about hardening iterations. This is the place where we clean up anything leftover. If we need any final user acceptance, doc work... whatever, this is where it goes. We are asking our customers if they are ready to accept the project deliverables.

Some Thoughts on Project Velocity

Companies I work with desperately want reliable estimates.

In reality, they need them so they can run their business. Businesses really want a crystal ball to look out into the future... to know pretty much what they are going to get, when they are going to get it, and some idea of what it is going to cost. The problem is that they need this information when approving projects... when making commitments to the market... and this is when we have the least reliable information about what we are building.

This need can result in pretty bad behavior on the part of both management and teams. To gain some sense of certainty, management does things like ask teams to provide estimates in real hours... and then hold the team accountable for delivering exactly according to that estimate. Faced with an impossible situation, teams are incented to game the numbers. They overestimate the project... to provide some necessary buffer... thus resulting in overinflated budgets. All this leads to a total lack of trust on both sides.

Velocity is intended to bridge the gap between management and the team. Teams get to measure project scope in relative units such as story points. Point estimates are specific to the team and measure how big one feature is relative to another in the backlog. Teams then begin building the features and measure how much of the backlog they are able to complete every iteration. After a few iterations, the team should have a pretty good idea how long it will take to complete the project.

Iterations to Complete = Backlog Size/Points per Iteration

For this equation to work, for velocity to be a meaningful metric, it has to be predictable... past performance must at some point become indicator of future performance.

Furthermore, velocity is only relevant to the business if we have a decent understanding of the overall scope of the project. That means we have to have a meaningful view into the backlog... for at least the upcoming release... sometimes the entire project. Measuring velocity against a constantly changing backlog doesn't give me a whole lot of information. If we are going to have any idea of what done looks like at the project level, we need to understand both backlog size and velocity... and understand how these metrics might change over time.

Velocity is fundamentally a measure of team throughput. How fast are we able to go... iteration over iteration... against a relatively stable queue of product features. These two metrics are the only thing standing between an agile team and total choas... at least when it comes to overall project performance.

To make all this work... both sides have to do their part. Ask not what your agile project can do for you... ask what you can do for your agile project. Here are a few thoughts on what managers and teams can keep in mind to keep velocity a reliable indicator of project performance... and maybe build a little trust while we are at it.

Responsibilities of Management

Realize that estimates are just that, estimates. As management, we can expect that estimate should get better over time, but what we get early... even in story points... is likely to change. We need to be prepared to deal with change.

Management should only use metrics like velocity as an indicator of project health and to help guide the project into a successful outcome. If management uses velocity as a lever or a performance metric, people will game the system to make the numbers look right. You'll get the numbers you want, you might just not get the product you want.

The size of the backlog has to be stable or the project is out of control. We can't just keep changing our minds indefinitely. We want to defer as many decisions, as long as possible, but we need to have a fundamental understanding of the product we are building and why. Constant churn will slow the team down and make everybody frustrated.

Responsibilities of the Team

Estimates are just that estimates. That said, we are accountable for making them better over time. At some point we have to be able to tell the business what they are going to get and when they are going to get it.

As team members we have to realize that our velocity must stabilize. Stable velocity is the life-blood of a predicable agile team. If we can't stabilize our velocity, the business will always see our project as out of control... because it is. We cannot expect the business to continue to invest never knowing what they can expect to be delivered.

We have to be honest with ourselves and our customers about what it really means to be done at the end of an iteration. Gaming the system to make the numbers look right will kill a project... hiding undone work in future iterations doesn't help either.

SDK Introduction

In my previous posts I’ve intentionally used the browser and URLs to demonstrate API functionality so that everyone, including non-developers, could experiment with our Open Platform.  For this post I need to get a little more technical, but stick with me because it’s not that bad.  

Recently I was adding acceptance tests to a story in our production instance of VersionOne.  As I entered the last test I realized that I had inadvertently created them task.   As I sat and contemplated my mistake I decided that rather than retype everything, I’d write some code to convert a task to a test and use that code to introduce our SDK.  

The VersionOne SDK is currently available in C# and Java.  The C# library can be used in PowerScript, a command line shell scripting language from Microsoft.  Since my target audience is both technical and non-technical people, I’ll use PowerScript for my examples. 

When using the SDK, the first step is to associate the VersionOne libraries to your project.  In C# this means adding a Reference and in Java you update your classpath.  In PowerScript you must load the appropriate DLLs before you can use the functionality they provide.  The following lines load the VersionOne SDK into the script environment. 

$apiClient = resolve-path "C:\V1SDK\versionone.sdk.apiclient.dll"
$objectModel = resolve-path "C:\V1SDK\versionone.sdk.objectmodel.dll"
[Reflection.Assembly]::LoadFrom($apiClient)
[Reflection.Assembly]::LoadFrom($objectModel)

Once loaded you need to create a connection to the VersionOne server.   To accomplish this you need the VersionOne application URL, a valid user for the system and the password for that user.  Using this information create a V1Instance object and validate the connection. 

$url = http://localhost/VersionOne/
$username = "andre"
$password = "andre"
$v1 = new-object VersionOne.SDK.ObjectModel.V1Instance($url, $username, $password)
$v1.Validate()

The V1Instance object is the starting point for all interactions with the server.  To retrieve a Task we create a Task Filter and set the Display ID to the value we see in the grid.  This filter is then passed into the “get” method for Tasks. 

$taskFilter = new-object VersionOne.SDK.ObjectModel.Filters.TaskFilter
$taskFilter.DisplayID.Add($taskDisplayId)
$taskArray = $v1.Get.Tasks($taskFilter)

The “get” method returns a collection of Task objects.  In PowerScript this collection is an array.  Before we proceed it’s important to verify that we have only one object in this array. 

if(1 -ne $$taskArray.Count)
{
 Write-Host "Only expected to find 1 Task but we found" $taskArray.Count -foregroundcolor red -backgroundcolor black
 Write-Host ""
 exit
}

Now that we’re sure that we have only 1 Task, it’s time to convert. 

In VersionOne, Task cannot exist independent of Stories and each Task belongs to only one Story.   The same is true of Tests.  When creating a Task or Test it must have a name and know which Story it’s related to.  Since we are converting a Task to a Test, we have both pieces of information.  Using the Parent attribute of a Task we can obtain a reference to the appropriate Story and SDK Story objects have the necessary methods for creating Test.  Therefore, we can create a Test from a Task with one line of script

# Create a test from the Task
$test = $taskArray[0].Parent.CreateTest($taskArray[0].Name)

Note how we are using taskArray.  The [0] is necessary because we want the first element in the array.   Once we have the Test created, we can populate the optional fields.  Here’s how we move over the Description and the Reference. 

$test.Description = $taskArray[0].Description
$test.Reference = $taskArray[0].Reference

Finally we save the Test

$test.Save("Created by Script")

Once the Test was created I had no reason to keep the Task, so I added this line to delete the Task.

$task[0].Delete()

We are done!  Running this script will convert the specified Task to a Test.   Since I had several Tasks to convert, I modified the script to accept the DisplayID on the commandline.  I then wrote a batch file to call this script with each of the Task I needed to convert. 

The complete script will be available in the next release of the SDK as a Sample application.   I modified the final version to accept all four parameters on the commandline (the VersionOne URL, Username, Password, and the DisplayID of the Task you want to convert.) 

What About Accountability?

One of my favorite lines ever was delivered to me by a pretty senior guy I was working with a few years back. I was pretty new to this whole agile thing and was struggling to understand how I could ever run a predictable project without my Gaant chart and a weekly status report. He said to me... look Mike, agile is about uncertainty... you just get what you get when you get it.

That was a far cry from how my teams had been operating in the past. I was more in favor of the line... you do what I tell you to do when I tell you to do it. That had always worked in the past and I really couldn't figure out why this situation was any different. In no way was I prepared to leave my project in the hands of a team that was unable to commit.

It's been a few years since that conversation, but I hear that sentiment echoed by many of the teams I've worked with since. Somehow, some of us in the agile community think that agile is about getting started and coding until you are done... with little planning in between. What happened to commitment? What happened to accountability?

Like most things in life, the truth lies in between these two perspectives. Agile is designed to deal with uncertainty. When uncertainty is part of the equation, no amount of up front planning is going to give us a plan impervious to change. On the other hand, moving forward with no planning is irresponsible. The business is entitled to have some idea of what they are going to get... and when they are going to get it.

After all, they are the ones paying the bills.. right?

The bridge between chaos and predictability is velocity. Velocity measures the throughput of the team. In other words, how much work they are able to get done iteration over iteration. I should probably go into more detail on velocity here but I am assuming you probably have been introduced to the concept. Click here if you need more info.

As an agile project manager, I don't really care about the units of velocity or even the actual number. That is specific to the team. What I do care about is that the number is useful. The team must be able to use that number to determine how much work they can do during the upcoming iteration. I also care about trends. Is the velocity increasing or decreasing. Have we centered around a specific number or is the metric varying widely iteration over iteration?

While ever increasing velocity is ideal... at a minimum I just need to be able to use past performance as an reliable indicator of future performance. It gives me insight into the health of the project.

Things brings me to my point about accountability.

An agile team is an empowered team. The team is empowered to do the estimate. The team can decide how much to bring into the iteration. The team can decide how the software is built. The team can decide how they organize themselves. The team can decide to break the work into tasks... they can decide not to. The team can decide who works on what.

The trade-off for all this empowerment is accountability. For agile to work, we need teams that can make and meet commitments. We need teams that are striving toward a stable velocity iteration over iteration. We need teams that value predictability. I am not going to tell the team what to do or how to do it, but they need to be able to deliver what they say they are going to deliver.

Period.

The team is accountable for knowing what they can do two weeks out... if we can't commit two weeks out, we are screwed.

Feelings, Thoughts, and Actions

Have you ever had to sit someone down and talk with them about their behavior. As a Dad I find myself having these conversations all the time. Son... I told you not to hit your brother. But Dad, he made me really angry.

To my kid, his feelings totally justify his behavior.

How about when you have to confront a team member about that heated altercation during the last retrospective? Bob... slamming your fist on the table was totally unacceptable behavior. But Mike, if I hear one more time that the build is late I am going to scream.

While there is clearly a relationship between how we feel, how we think, and how we respond... it is my opinion that our feelings NEVER actually justify our actions. Think about it, are we EVER entitled to get angry and punch someone? Are we EVER entitled to get our feelings hurt and lash out with a personal attack? Are we EVER entitled to treat others disrespectfully when we have been treated with disrespect? Even when it comes to positive emotions like love, affection, or friendship; our feelings must be tempered by our values and principles before we respond.

By understanding the right relationship between our feelings, thoughts, and actions we give ourselves the opportunity to choose how we want to behave. Knowing we have that choice gives us the opportunity to change. By understanding there is not an immutable link between our feelings and our actions, can work to decouple them. We can work to capture our feelings and break them down... we can understand them. We can become intentional about how we think and how we see the world around us. We can make a conscious decision to act in accordance with what is most important to us.

Validate Feelings

Feelings come from the deepest parts of ourselves... they are an integral part of who we are. While we may change how we feel over time, it is unlikely that anyone is going to coach them out of us. It is how we are wired. When coaching either my kids or my team members, I usually start by validating feelings. I know your brother made you mad, I would be mad too if he did that to me. By validating feelings you validate the individual and create the opportunity to learn in a non-threatening environment. It creates the opportunity to change how people think.

Guide Thinking

How we think can be influenced more directly... it is somehow less personal. We can learn about our environment and the people that are a part of our lives. We can gather more information about what motivates those around us and learn something about their intentions and circumstances. With new information, we can learn to think differently about what is happening to us. Why do you think your brother was annoying you? Maybe he felt lonely and wanted you to play with him. By guiding thinking, we are able to broaden the perspective of our team and create the opportunity to coach behavior.

Coach Action

We cannot be held accountable for how we feel... we cannot be held accountable for how we think. We can and will be held accountable for what we do. Our actions are governed by social and societal norms. Our behavior... our performance... can be measured and will form the basis for how others will evaluate us.

By thinking about problems differently, we open up a whole new range of possible options. Son, it is never acceptable to hit your brother. Maybe you could gently explain to your brother that you will play with him in a few minutes, or maybe you could include him in your game? Having more options available to us gives us more opportunities to behave in accordance with our values and principles.

When we decouple our feelings from our actions, we open up a broader palette of possible outcomes... outcomes not predicated by our emotions but based who we choose to be as human beings.