The Secret to Organizational Agility

Are you ready... here it is... to be agile you must... break dependencies at all costs!

Dependencies are created when any two discrete elements within your organization require each other to do all or part of their job. At the enterprise level, dependencies are everywhere. They are created by the very way we have decided to structure our businesses and how we setup and execute our project work. This makes them very hard to identify and even harder to remove.

I believe that breaking our dependency on dependencies is the core challenge associated with orchestrating any sustainable enterprise agile transition.

Agile transitions don't fail because Scrum is insufficient. They don't fail because people have not sufficiently applied the right XP engineering practices. Our problem is not a fundamental misunderstanding of the Agile Manifesto nor an unwillingness to become the right kind of agile leaders.

While any given organization may have some or all of these problems, they are not our primary concern.

The underlying issue is that agile project management principles, agile engineering practices, and agile leadership philosophies assume you have already eliminated your dependencies! Most teams are trying to become agile without validating that core underlying assumption, no one has even told them they should try!

Think about how we describe our ideal agile implementation...

First, we start with a small team of cross functional individuals with sufficient skills such that anyone can work on any part of the system. Next we allow this team to only work only on a single project during any given sprint.

We then grant this team a dedicated product owner empowered to make decisions on behalf of the business. We allow them to work on small independent functional requirements which can be reprioritized at any time.

We give this team an empowering individual tasked with removing any organizational challenges the team might encounter. We remove any process constraints and allow them to do the work any way they want as long as they deliver working software.

Lastly, we assume they have an object oriented architecture that lends itself to test driven design, continuous integration, and constant refactoring.

This scenario perfectly describes a team with no dependencies. They have 100% of everything they need to get the job done.

Let's look at what many teams deal with trying to adopt agile...

People are typically part of a team of specialists that are matrixed into a cross functional project team. Because these people are specialists, their services are not needed 100% of the time. To maximize the utilization of the individual, they are assigned to multiple projects and any number of teams.

Product owners are doing market research, meeting with customers, and attending trade shows. The team is left with a less than empowered business analyst to make the decisions for the business. In reality, the product owner was just a proxy for a large group of uninvolved stakeholders anyway. No one gets access to real customers or real market data.

Many teams are trying to sprint through product development using a traditional MRD or PRD. They are not using requirements written as functional threads of system behavior that can fully tested at the end of the sprint. These requirements are not able to be reprioritized as we learn more about the evolving system.

Many teams are working with traditional project managers who are doing their best to be agile, but have been trained to manage dependencies and tell people what to do. Even if they want to be agile, they are not usually empowered to do anything about the real impediments the project teams are working through.

Teams are trying to be agile with tightly coupled software architectures, insufficient test coverage, legacy code bases, and unable to do a daily build.

This perfectly describes a team operating under the crushing weight of dependencies created by flawed organizational assumptions.

Let's look at some of our common assumptions that lead to unnecessary dependencies….

We assume that we must optimize for individual performance and therefore we matrix team members across projects. This creates dependencies between projects that did not have to exist. We find ourselves managing who is on what project and when they will be available for other projects. We create a network of dependencies [between projects] that restricts our ability to change course. Any change has a series of painful cascading impacts on our portfolio.

We assume parts of our solution are just too complicated and must be handled by specialists. We choose to assign certain people to specific components. By default, we create dependencies between products that share the same components. We also create dependencies between requirements that impact the same component. We create project dependencies between teams (and between team members) when we have these specialists assigned to a team of specialists.

We assume that our product owners are too busy for the team so we assign someone else to proxy for the business. We create a dependency between the team and the business that results in a tendency to over-document and manage the relationship between contracts. When contracts are used we create dependencies between the team and the business that must be maintained, enforced, and put under change management.

We assume the market is demanding all the features at one time so we do not focus on small functional slices across the entire systems architecture. Why bother, it all has to be done anyway? This thinking leads us to consider everything before we can consider building anything. When the requirements are all dependent on each other, there is no need to prioritize, no ability to change our minds, and no ability to learn from the emerging system. That big up front design document starts to look pretty reasonable.

We assume that architectures cannot be decoupled or that test coverage is too expensive. This causes us to think about making every change when we are making any change. Batched changes are made outside the context of working software. They are integrated and tested all at once... probably near the end fo the project. By creating dependencies between code changes, we loose the ability to learn from our mistakes, refactor, and hold teams accountable for outcomes.

Dependencies force us to measure hours worked, or modules delivered, or lines of code, when what we really want to measure is working software.

Why does all this matter?

Some of these dependencies are real. Many could take years to overcome. Just realize that every dependency you accept, every one you choose to leave in place, is going to fundamentally limit your ability to adopt agile methods.

Every single one will require a trade-off that will reduce your effectiveness and ability to respond to change.

By focusing on project management technique, engineering practices, or even leadership philosophy first… we miss the mark. We are inclined to pick from a smorgasbord of tactics and choose what can work for our particular organization. We are not forced to deal with the foundational problem, and therefore pick and choose our way into a meaningless agile implementation, one that will not deliver the essential value the organization is trying to deliver.

By focusing on the relentless pursuit and elimination of dependencies, we position our organization to potentially adopt ALL of the agile best practices. We can then pick and choose based on our values and principles, not on the constraints of our tangled up dependency based organizations!

Reduce dependencies at all costs. This is going to become my mantra for 2009. Expect to hear more on this topic as the year unfolds!

Comments

[Anna Forss]

Interesting post on an important but difficult subject. Thanks!

[Wally Lafferty]

This was a brilliant piece! And especially important for organizations who are transitioning to agile management from some other traditional process. We tend to focus on agile management for the development team, and forget that the Finance department often constrains the way projects are budgeted, and therefore the way resources are applied. We often forget that the Contracts department may be stuck on an SOW template that creates contractual dependencies on "requirements up front" and "testing at the end." A transition to agile should be considered in the context of a TRANSFORMATION of the entire organization to ensure ALL dependencies are broken down.

[Christopher Avery]

Mike,

Provocative! Thanks for the perturbation. I look forward to your unpacking this some more. For instance does this also put an end to delegation, outsourcing, performance management, and -- gosh forbid -- accountability management since you own it all?

And, I was going to comment on this over at my blog but then that would *depend* on your post remaining in place here -- and I'm not sure I can count on that as an acceptable risk.

(-;

Thanks,
Christopher

[Daniel Davis]

Sparking good thoughts, thanks Mike. Just read it once and intend to read another time or two, but raised an intiial question on if maybe should be some breakup for the term dependency? Aren't some dependencies encouraged, like in a SOA? Maybe its a difference between business dependencies and code dependencies?

Dan.

[Mike Cottmeyer]

Thanks Wally. It is hard to argue with your assessment of this post ;-)

[Mike Cottmeyer]

Thanks Dan for the comment.

Like many blog posts, this article was clearly and intentionally a little provocative.

I truth, I have started questioning all dependencies with my clients... especially around teams, projects, architecture, and programs.

Inevitably some dependencies remain and those are probably the good dependencies that you talk about. Sometimes dependencies remain that I would rather see broken. Either way those dependencies need to be managed.

If they are unmanaged, or overwhelming to the organization to manage, they will slow you down. Maybe if I was being more levelheaded, I would have said "minimize dependencies at all costs" rather than the "break" at all costs. I do think the stronger language makes people think harder ;-)

Thanks for reading.

[Mike Cottmeyer]

Thanks for your reply Christopher... I read your comment when I woke up this morning and have been trying to think of something witty to say all day ;-) I am giving up and just going to do a straight reply!

When I go out an talk to clients... the primary barrier to adopting agile methods, adopting the agile ethos, setting up accountability, setting up empowered teams, etc. is the amount of dependencies they are willing to maintain and trying to manage.

From my perspective the primary dependency challenges are between teams, projects, programs, and architectural subcomponents.

Most organizations are so mired in unnecessary dependencies, based on faulty assumptions, unvalidated assumptions... this seems the best place to start.

Removing dependencies is hard, and there is clearly no way to remove all of them... but... using this as your starting place for agile adoption helps many organizations understand more clearly why their agile implementations are failing and what they can do to help them along. Maybe a good caveat to my post is that any dependencies you decide to keep, you better make sure you are managing them effectively.

So... if you ever link to one of my posts, and you want to make sure that the link will be valid indefiantely, you will have to manage the dependency with my site. that is overhead and it might impact your ability to change YOUR post at a later date.

Therefore, you were probably better off breaking the dependency and replying straight into my blog... ;-)

[Chris Bernard]

Enlightening perspective on dependencies!

But when you say:

"Lastly, we assume they have an object oriented architecture that lends itself to test driven design, continuous integration, and constant refactoring."

... you seem to be assuming that OO is required for agility. Wouldn't you agree that a functional architecture can support TDD, CI, and constant refactoring just as well and an OO architecture?

And if not, why?

It would seem the programming paradigm is not important to agility, rather it is the architecture's modularity, loose coupling, and lack of duplication -- In short: flexibility -- that reduce the cost of code changes.

[Mike Cottmeyer]

Chris,

Your point is well taken.

I was thinking more about some of the tightly coupled mainframe Cobol environments I have worked in... and generalized the opposite of that to be an object oriented environment.

I agree that the programming paradigm is not important, but as you state... the modularity, loose coupling, and lack of duplication.

Thanks for the great comment and for keeping me honest ;-)

Mike