Agile, the Philosopher's Stone, and the sound of one hand clapping.
Say the word 'Agile' or 'SCRUM' in certain circles, and the air can develop a kind of jello-like quality, the even gate of time is seen to tick-tock ever so slightly slower, the lights flicker, eyes roll, people start fidgeting with their hands.
I have been a software developer now for 11 years, and have been hearing about 'Agile' for at least the last 8, and I have yet to work for anyone who has managed the grand feat of practising pure 'Agile/SCRUM'.
That's not to say that we haven't tried.
Anyone who regularly visits the church of software development will understand the pain involved in that statement. I think if anything, that pain gives way to the salient point of this article: that a methodology that is as universally and un-purely practised as Agile/SCRUM might lead one to question it's validity as a best practise.
Agile is like the 'space' to SCRUM's process oriented mastery of 'time', to use the equally difficult to understand idea of space/time in quantum physics as an example. Agile is like an ambient set of preferences (space), and SCRUM is process that puts those preferences into practice (time).
The base preferences of Agile are posted here for reference:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
Followed by the twelve fundamental principles underlying SCRUM:
- Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
- Business people and developers must work together daily throughout the project.
- Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity–the art of maximizing the amount of work not done–is essential.
- The best architectures, requirements, and designs emerge from self-organizing teams.
- At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
To the layman, these seem reasonable.
But ask any veteran software developer, and you'll come to see another perspective as we recount the various ways that these principles can be slung like mud bricks used to lay down the yellow brick road of our professional discontent.
Agile/SCRUM experts would say that is because we have not been using it in it's purest form.
Logically speaking, the form these opposing arguments take is the equivalent of a leminscate of blame adjustment: a never ending infiniti mobius strip loop of finger pointing – for what should essentially be a working, time-tested business process.
Many would say that this is a huge football-foam '#1' finger pointing at the irrefutable evidence that Agile is already/finally in decline.
As much as I love a good agile-scapegoating-band-wagon, this is not necessarily the one I would jump on just yet.
In a sense, and perhaps for me, Agile/SCRUM begins to approach in quality something not unlike the mythological 'Philosopher's Stone': the distilled concretion of an eternally unanswerable question.
Fundamentally, the business need that Agile and other methodologies of software development purport to solve is how do people without a fundamental understanding of technology direct a team full of people who are adept with the technology but who are not intimately aware of in the underlying business need.
Basically, any project management methodology is an API between people who know what they need but don't know how to do it and people who know how to do it but don't know what to do.
In the process of 'project management' the goal is to break down a large net of needs into quantifiable directives in a way that meets interdependencies between the needs themselves (IE: the mailing system infrastructure needs to be set up before the HTML emails can be created/tested), and the human systems that are driving the process (IE: the content of the HTML emails must be known and communicated before being codified, the Content Types/Forms must be created before content can be added, etc).
Typically, sorting out the mess would involve at least 2 players: someone from the client side who is expert in their needs (typically called a 'Product Owner'), and someone on the development side who is expert in the technology (typically called a 'Tech Lead'). Often and sometimes thankfully, a 3rd role is brought into the mix: someone who is expert at managing the process and keeping track of the flow. We typically call this person a 'Project Manager' or 'Scrum Master'.
These folks ideally would meet to sort out and package the net of needs and dependencies into 'Sprints' and units of development defined into chunks of directives that the developers use to make it all happen (user Story, ticket, Card, etc).
When push comes to shove, which it often does in this milieu, the MOST COMMON POINT OF FAILURE lies in the comprehensibility and timeliness of the directives being given to a given team.
This can be caused by a variety of ills both known and unknowable, but the base outcome is that a ticket that is not intelligible or not clear enough to begin work immediately triggers the stopclock of wastefullness as we sort through the various types of questions that come up - ranging from 'what URL are we talking about' to 'what architecture's or methods are we using to turn different types of content into web-pages' to 'where is that thing that needs to be in place before a developer can even start working on the problem'.
And then there is the time waiting for a response, and the many levels of hierarchy within the company that need to be scaled in order to get an answer that can then be turned into a clear directive. In the process, the documentation about what is going on becomes hard to parse as comments upon comments get added, or gets left off entirely depending on the working styles of the people involved in solving it and whether the thinking was done in a live conversation or not. And this presumes that the players involved have the organic capacity to write a clear directive - which cannot always be taken for granted.
There are a million things that get in the way of clear directives being communicated in a timely way to the engineering team - ranging from bottlenecks with a client to a mutual lack of complete understanding of the whole context on the side of engineering to a lack of consistency or a fickleness or lack of ability to structure a user story on the part of a product owner to an inability for a project manager to adequately educate or steer a process and essentially the point is that it is often the collective fault in small ways rather than any particular player in a particularly large way.
At the same time, each of these end up becoming a little de-railing influence off the beaten Agile track.
This is why the questions and issues that Agile and other systems purport to solve are essentially as solvable by a single outlined process as the sound of one cheek farting (obligatory fart reference for master Ogilvie, whose voice helps me remember to take it all in stride and at the very least make it interesting).
Where is the hope?
Tools are to be used. When using a tool or technology, it is important that the tool being used suits the particular task well.
Often times when we come to Agile/SCRUM, we begin with the tool and then we start trying to solve everything with it.
What if, instead, we approached Agile/SCRUM as the beginning of a really good idea and assumed responsibility for its development?
What if instead we developed an arsenal of tools associated to the Agile principles and described how best they can be used to suit different contexts? What if in the process of figuring it out for any given project, an evolved consideration of these matters begins to coalesce with an ability to be released back into the fold as an evolutionary improvement - much like the development of software itself?
The hope is when we stop expecting a system to meet all of our needs, and instead consciously engage in evolving the systems we use whether we call them Agile or Kanji or Waterfall or something else.
The key to supporting this 'milieu of evolution' is in creating an atmosphere on a team that is kind and supportive. An atmosphere that values learning above ego, and that listens. An atmosphere that has the ability to co-evolve in response to circumstances as they arrise.
Many Agile supporters would say that this 'milieu of evolution' is essentially Agile, and they might be right about that point.
But let's not stop there.
Until the culture of a team (and perhaps the Agile movement itself) comes to understand the dialectic use of 'our version of Agile' as a form of jedi-level engagement as opposed to a failure, conscious evolution of Agile as an idea and a methodological approach (an idea worth pursuing in my humble opinion) is not possible.
If the idea of Agile/SCRUM is a metaphorical or literal Journey of 1000 miles for your company (the word Agile here might be replaced with any methodology or prescription for software development), the first step is to engage as a team that consciously takes responsibility and care for it's conscious evolution.