Nowadays, Agile is “the hot skill” … if you are experienced and trained in Agile, then you can run in the front with the big shots across the industries.
But it was not always this way …
When in March 2006 – out of pure personal curiosity – I completed a two-day training and certification workshop as a Certified Scrum Master, the majority of the industry had not even heard the word Agile in their lives … let alone Scrum … or Scrum Master.
I soon took on the challenge of becoming an advocate of Agile approach and Scrum through many enterprises and helping with the adoption of this brilliant way of thinking by the market.
In those days, Waterfall ruled the arena and I was working as a PMP certified Project Manager for large financial institutions.
According to these enterprises, they had just finished the painful journey of perfecting their own implementation Waterfall process and their strong PMO (Project Management Office) teams and would have no appetite to start a new shift in their – just settled – paradigms!
It took some significant, prolonged effort by me and my peers across multiple industries to slowly convince this giants to brave the idea of experimenting with Scrum and carve out a few people from the Waterfall teams to form up as Scrum teams to check this new thing as well.
The positive results, in the shape of higher productivity, lower quality issues and faster time-to-market were so strong that accelerated the adoption process and Agile (and its most popular framework: Scrum) became the new modus operandi.
Even though Agile Manifesto was signed in 2001, and despite the fact that Scrum as a framework was introduced by Jeff Sutherland and Ken Schwaber in 1995 (Yes! Agile Manifesto was signed 6 years later but Agile as a concept was discussed in academic environments for a couple of decades before that.), the actual search for better ways of managing the work of a team goes way back in history!
When did the search for Find Better Ways to Manage Projects start?
The prehistoric human had to cover all the roles by him/herself:
Manager,
Idea generator,
designer,
builder,
tester and the
End-User!
Humans learned it would be much more efficient if they distributed responsibilities among the team members, so each one can become better at what they are doing (Specialization).
Now a New Problem had emerged:
How to get a team to work together effectively?
This led to invention of the Traditional Approaches
Through a few centuries of trials and errors, humans finally established the plan-based (sequential) traditional project management methods.
Using them, they even managed the work to build the worlds wonders and magnificent monuments – many of which – still standing today.
And from the project management point of view
They all had a Serious Problem: None of them would finish in Time!
And as it would be expected, they would Never Finish within their initially anticipated Budget.
The efforts for improvement of traditional approaches in tracking the progress against the schedule and budget continued for centuries.
More accuracy in tracking the efforts and progress was achieved through heavier in-process stages and gating (checkpoints). They also grew heavy in paperwork and documentation.
The better Time and Budget Management allowed projects to finish during the lifespan of the original project team. (No more generational hand-offs!)
But the extra documentation and multiple gating approach had a heavy price tag!
It made them even slower than before to react to any changes in the market!
The the Industrial Revolution arrived and:
- Manufacturing became more and more competitive
- Local markets grew into global trading
- Local competition had changed into an international race for customers’ money.
More than ever before – to survive and thrive – businesses needed to:
- Make sure the Product is exactly as customers needed it
- Sell it to them before the competition does it, or their demands change
But the Traditional Approaches kept missing the mark
… Let’s take a closer look …
The most famous traditional approach is the Waterfall approach.
The reason for this name is its close resemblance to a waterfall. Once one step is completed, the outcome pours into the next level and next until it reaches the end. The steps are in sequence and the movement is only in one direction.
… and they want to implement that Great Idea into a Product and send it to the Market!
Using Waterfall, the will start from the Requirements step, then once the Requirements work is done, the output (Requirements Document) is given to the Design Step, where the Design Documents are created, and once that is done, Development starts based on the Design Documents, and after that is done it goes to the Testing and when passed goes to the Release and there you enter the market with your great idea …
Awesome! Right? …… Well ….
History has shown that: This approach would ONLY SUCCEED …
Which would be far from a Real World Scenario in Any Market!
In reality, what we deliver will always be –to some degrees – different from what customers need at that point in time!
How Bad has it been with the IT Projects?
Historically close to 70% of Waterfall IT Projects have failed to accurately deliver what market needed at the time of their release.
Waterfall is slow to deliver (as it needs to finish one step completely and pass the gate setup right after that step, before it can go to the next step. The amount of work that needs to be put into place before the project qualifies to move forward sums to a long period of time, typically months – even years – before anything can come to the release date and get delivered to customers).
Waterfall is not designed to quickly respond to changes in the market (as it needs to do one full round of steps before it can test the product in the market, if we decide, in the middle of the project, to make a change, we need to re-do all of the steps we completed and put them through gating again. This way, by the time we have completed a response, such a long time has passed, that we most probably need another change to re-align with the market).
Even if we go a great job of capturing exactly what the market needs when we start our Requirements step, while we are following our straight path from that step towards the Release, market would drift away from that captured idea and go down a different path.
As time goes by this diversion becomes wider and the market expectation for an ideal product would change even further away from our starting Requirements.
That means by the time we deliver what we captured at the beginning, we have landed far away from where market needs us!
We need to also consider the fact that: Value of an Idea drops over time!
Because:
- Competitors may find a similar or better idea and enter the market earlier.
- Customers may start following a different trend or fashion.
- Customers may slow down on spending money due to a newly tightening economy.
- Customers may demand new features that they didn’t want before.
What else?
- We may learn – too late in the project – that we do not have the skill sets for the project.
- We may learn – too late in the project – that it is not feasible, technically or financially, to implement.
- We may learn – too late in the project – that this was not a very good idea to begin with!
When were the modern ways invented?
As you can imagine, The problems with the Traditional Approaches were noticed long before the 20th Century!
So for decades in the 20th century, Innovative & Pioneering Architects and Project managers were using Agile and Lean concepts to Finish their Projects Faster and Better Match Customers’ Needs without an official recognition of their approaches
The Secret Rebellion within the Waterfall ranks
Waterfall dictates a sequence of mutually exclusive phases, separated by gates where one phase must be completed before the next one can start.
But the reality of what is actually happening is very different!
Over the past several decades, the managers and leaders of production teams – looking for ways to improve their teams’ performance – had discovered that they can raise their team’s productivity through secretly allowing the work to slip through the gates and to let the team start working on the next phase before the previous phase is completed, just going ahead with a bare minimum output (Lean) from the work still-in-progress at the previous gate!
This became an un-written agreement among the team managers and the upper management to pretend it is not happening.
The team managers won’t talk about it and the upper management won’t ask about it.
The empirical success of these “lean” and “Agile” forward thinking process manipulations, triggered the ideas behind official creation of the modern approaches of today.
And this incremental growth of Agile (Lean) mindset within the Waterfall ranks developed into the foundation we needed in the industry for official recognition of Agile as an approach and its further development into frameworks such as Scrum, as we enjoy practicing today.
Cheers
Arman Kamran (The Agilitizer)
