Agile Chronicles

T'was Agile

   

Twas the night before release and all through the land
Not a person was working, not even one hand

The code was all tested and checked in with care
Even the documentation was completed and there

The PMO Director & VP of Dev were nestled all warm and snug in their beds.
While visions of bonuses danced in their heads

Joe closed his laptop and Sue hit the sack
Knowing morning was near and they needed the nap

When from my pants pocket there arose such a clatter
I grabbed my I-Phone to see what was the matter

Off to the office I flew like a Senior VP
Tore open my briefcase and fired up the PC

When what in my unsettling mind might appear
But an incomplete test case as I trembled in fear

I regained my senses and checked out the code
Reviewed the regression and distributed the load

It passed all the tests now my mind was at ease
Knowing that many would certainly be pleased

That I put forth such effort on a cold chilly night
To re-synch our build and ensure all was right

Soon morning approached and I knew not to worry
As none of the product was crammed in a hurry

It makes my heart filled with holiday cheer
To know Agile Methods make everything clear

I logged out of VersionOne feeling all was now right
Happy Holidays to all and to all a good night!

Sort and Aggregate Functions in the VersionOne Open Platform

In the last post I discussed how to query VersionOne and create Story Cards using XSL.  To accomplish this we used the following query which returns all stories for a specific project and iteration:

http://localhost/VersionOne/rest-1.v1/data/Story?sel=Name,Number&where=Scope.Name="Release 1.0";Timebox.Name="Month C 1st Half"

The API allows us to control the Order in which the results are returned using the 'sort' parameter.  To use this parameter you need to add &sort={Attribute} to the query.  As an example, if I want the results sorted by importance, I would issue the following query

http://localhost/VersionOne/rest-1.v1/data/Story?sel=Name,Number&where=Scope.Name="Release 1.0";Timebox.Name="Month C 1st Half"&sort=Order

If I wanted them sorted by Estimate, I would issue this query

http://localhost/VersionOne/rest-1.v1/data/Story?sel=Name,Number&where=Scope.Name="Release 1.0";Timebox.Name="Month C 1st Half"&sort=Estimate

You can sort on multiple attributes by using a comma to separate the attribute.  In the next example the server will return the results sorted by Estimate and then by Name.

http://localhost/VersionOne/rest-1.v1/data/Story?sel=Name,Number&where=Scope.Name="Release 1.0";Timebox.Name="Month C 1st Half"&sort=Estimate,Name

You’ll notice that by default data is retuned in ascending order.  You can reverse a sort by adding a minus sign (-) or declare ascending explicitly by adding a plus sign (+).  The following query will return data reverse sorted on the Name attribute

http://localhost/VersionOne/rest-1.v1/data/Story?sel=Name,Number&where=Scope.Name=%22Release%201.0%22;Timebox.Name=%22Month%20C%201st%20Half%22&sort=-Namehttp://localhost/VersionOne/rest-1.v1/data/Story?sel=Name,Number&where=Scope.Name="Release%201.0";Timebox.Name="Month%20C%201st%20Half"&sort=-Name

The VersionOne API also provides aggregate functions that allow you summarize a collection of data.  To demonstrate this I’ll use the 'Children' attribute on the Story asset.  This attribute is a collection of all the Task and Test defined for a given story.  The result for the following query will contain all the Task and Test for a Story called "Order Check"

http://localhost/versionone/rest-1.v1/data/Story?sel=Name,Number,Children&where=Story.Name="Order%20Check"

If we only wanted to know the number of Task and Test, we would use the Count aggregate function on the Children attribute.  Here’s the query

http://localhost/versionone/rest-1.v1/data/Story?sel=Name,Number,Children.@Count&where=Story.Name=%22Order%20Checkhttp://localhost/versionone/rest-1.v1/data/Story?sel=Name,Number,Children.@Count&where=Story.Name="Order%20Check"

Another useful aggregate function is @Sum, which can be used to add the values of numeric fields. Task and Test both have numberic attributes for the estimated effort (DetailEstimate) and an for the remaining effort(ToDo).  Have the Sum of these attributes is useful and can be obtained by adding Children.DetailEstimate.@SumChildren.DetailEstimate.@Sum and Children.ToDo.@SumChildren.ToDo.@Sum to the query. 

http://localhost/versionone/rest-1.v1/data/Story?sel=Name,Number,Children.@Count,Children.DetailEstimate.@SumChildren.DetailEstimate.@Sum,Children.ToDo.@SumChildren.ToDo.@Sum&where=Story.Name="Order%20Check"

We can now enhance our Story cards to include the number of task and test, the total estimated effort and the remaining work.  We can also display or print the cards in order of importance.  Here the complete query

http://localhost/VersionOne/rest-1.v1/data/Story?sel=Name,Number,Children.@Count,Children.DetailEstimate.@Sum,Children.ToDo.@Sum&where=Scope.Name="Release%201.0";Timebox.Name="Month%20C%201st%20Half"&sort=Order &xsl=Custom/StoryCards.xsl

The Secret to Organizational Agility

Are you ready... here it is... to be agile you must... break dependencies at all costs!

Dependencies are created when any two discrete elements within your organization require each other to do all or part of their job. At the enterprise level, dependencies are everywhere. They are created by the very way we have decided to structure our businesses and how we setup and execute our project work. This makes them very hard to identify and even harder to remove.

I believe that breaking our dependency on dependencies is the core challenge associated with orchestrating any sustainable enterprise agile transition.

Agile transitions don't fail because Scrum is insufficient. They don't fail because people have not sufficiently applied the right XP engineering practices. Our problem is not a fundamental misunderstanding of the Agile Manifesto nor an unwillingness to become the right kind of agile leaders.

While any given organization may have some or all of these problems, they are not our primary concern.

The underlying issue is that agile project management principles, agile engineering practices, and agile leadership philosophies assume you have already eliminated your dependencies! Most teams are trying to become agile without validating that core underlying assumption, no one has even told them they should try!

Think about how we describe our ideal agile implementation...

First, we start with a small team of cross functional individuals with sufficient skills such that anyone can work on any part of the system. Next we allow this team to only work only on a single project during any given sprint.

We then grant this team a dedicated product owner empowered to make decisions on behalf of the business. We allow them to work on small independent functional requirements which can be reprioritized at any time.

We give this team an empowering individual tasked with removing any organizational challenges the team might encounter. We remove any process constraints and allow them to do the work any way they want as long as they deliver working software.

Lastly, we assume they have an object oriented architecture that lends itself to test driven design, continuous integration, and constant refactoring.

This scenario perfectly describes a team with no dependencies. They have 100% of everything they need to get the job done.

Let's look at what many teams deal with trying to adopt agile...

People are typically part of a team of specialists that are matrixed into a cross functional project team. Because these people are specialists, their services are not needed 100% of the time. To maximize the utilization of the individual, they are assigned to multiple projects and any number of teams.

Product owners are doing market research, meeting with customers, and attending trade shows. The team is left with a less than empowered business analyst to make the decisions for the business. In reality, the product owner was just a proxy for a large group of uninvolved stakeholders anyway. No one gets access to real customers or real market data.

Many teams are trying to sprint through product development using a traditional MRD or PRD. They are not using requirements written as functional threads of system behavior that can fully tested at the end of the sprint. These requirements are not able to be reprioritized as we learn more about the evolving system.

Many teams are working with traditional project managers who are doing their best to be agile, but have been trained to manage dependencies and tell people what to do. Even if they want to be agile, they are not usually empowered to do anything about the real impediments the project teams are working through.

Teams are trying to be agile with tightly coupled software architectures, insufficient test coverage, legacy code bases, and unable to do a daily build.

This perfectly describes a team operating under the crushing weight of dependencies created by flawed organizational assumptions.

Let's look at some of our common assumptions that lead to unnecessary dependencies….

We assume that we must optimize for individual performance and therefore we matrix team members across projects. This creates dependencies between projects that did not have to exist. We find ourselves managing who is on what project and when they will be available for other projects. We create a network of dependencies [between projects] that restricts our ability to change course. Any change has a series of painful cascading impacts on our portfolio.

We assume parts of our solution are just too complicated and must be handled by specialists. We choose to assign certain people to specific components. By default, we create dependencies between products that share the same components. We also create dependencies between requirements that impact the same component. We create project dependencies between teams (and between team members) when we have these specialists assigned to a team of specialists.

We assume that our product owners are too busy for the team so we assign someone else to proxy for the business. We create a dependency between the team and the business that results in a tendency to over-document and manage the relationship between contracts. When contracts are used we create dependencies between the team and the business that must be maintained, enforced, and put under change management.

We assume the market is demanding all the features at one time so we do not focus on small functional slices across the entire systems architecture. Why bother, it all has to be done anyway? This thinking leads us to consider everything before we can consider building anything. When the requirements are all dependent on each other, there is no need to prioritize, no ability to change our minds, and no ability to learn from the emerging system. That big up front design document starts to look pretty reasonable.

We assume that architectures cannot be decoupled or that test coverage is too expensive. This causes us to think about making every change when we are making any change. Batched changes are made outside the context of working software. They are integrated and tested all at once... probably near the end fo the project. By creating dependencies between code changes, we loose the ability to learn from our mistakes, refactor, and hold teams accountable for outcomes.

Dependencies force us to measure hours worked, or modules delivered, or lines of code, when what we really want to measure is working software.

Why does all this matter?

Some of these dependencies are real. Many could take years to overcome. Just realize that every dependency you accept, every one you choose to leave in place, is going to fundamentally limit your ability to adopt agile methods.

Every single one will require a trade-off that will reduce your effectiveness and ability to respond to change.

By focusing on project management technique, engineering practices, or even leadership philosophy first… we miss the mark. We are inclined to pick from a smorgasbord of tactics and choose what can work for our particular organization. We are not forced to deal with the foundational problem, and therefore pick and choose our way into a meaningless agile implementation, one that will not deliver the essential value the organization is trying to deliver.

By focusing on the relentless pursuit and elimination of dependencies, we position our organization to potentially adopt ALL of the agile best practices. We can then pick and choose based on our values and principles, not on the constraints of our tangled up dependency based organizations!

Reduce dependencies at all costs. This is going to become my mantra for 2009. Expect to hear more on this topic as the year unfolds!