Agile vs Waterfall
When you talk about project methodology you will find that an ever ongoing discussion is around which methodology is the best Agile or Waterfall. This is a topic that gets a lot of discussion (and often heated debate).
Just to make sure we are all on the same page, a definition of development methodology is helpful. To put this definition very simply, it’s a way of organising the work of (software) development. This is NOT about a style of project management or a specific technical approach. You will often hear these terms all thrown together or used interchangeably though.
The two basic, most popular methodologies are:
Waterfall: which might be more properly called the “traditional” approach,
Agile: a specific type of Rapid Application Development and newer than Waterfall, but also already around for a while, which is often implemented by using Scrum.
Both of these are usable, mature methodologies. Having been involved in software development projects, agile coaching and providing agile workshops, scrum master and product owner certifications for a long time, here are my thoughts on the differences strengths and weaknesses of each of the methodologies.
Traditional Approach or waterfall
In a traditional sense the Waterfall approach treats analysis, design, coding, and testing as discrete sequenced phases in a software project. This worked well when the cost of change was high. But now that it's low it hurts us in a couple of ways.
Poor Quality
The first challenge, when the project starts to run out of time and money, testing is the only phase left. This means good projects are forced to cut testing short and quality suffers.
2. Poor Visibility
The second challenge, because really working software isn't produced until the end of the project, you never really know where you are on a Waterfall project. That last 20% of the project always seems to take 80% of the time, which will cloud visibility of a possible delay till very late in the project, which makes it very difficult to mitigate.
3. Very Risky
The third major challenge is you've got to schedule a lot of risk, because you never know if you are going to make it until the end.
4. Very difficult to implement changes
And finally the most important challenge, it's just not a great way for handling any change. Effectively implementing a change will throw your whole project back in time
When you see these challenges in the traditional way of delivering (software) project you suddenly start to understand why 80% of all software project fail. When I talk to business people about IT project I always get comment along the general challenges of IT projects
Time-to-market for products is too long
Project failure rate is unacceptably high
Return on investment (ROI) delivered frequently falls short
Responding to change is difficult and costly
Customer orientation is weak
Software quality is poor
Productivity could be higher
Employee morale, drive and accountability is low
Widespread micromanagement is required
Employee turnover rates are too high
This you will hear often summarised in the sentence shouted out by frustrated business managers, after the IT project finally came to a deliverable; “IT always deliver a project outcome I cannot use anymore due to my changed business requirements or I did not want in the first place. The project delivery is way too late and way too expensive”
So how does the agile approach now actually differ?
The main difference is; Instead of sequencing these fixed stages you saw in the classical waterfall method, the agile approach believes these stages are actually continuous activities and happen each individual iteration or sprint.
By doing these activities continuously you will recognise that:
Quality improves because testing starts from day one.
Visibility improves because you are 1/2 way through the project when you have built 1/2 the features.
Risk is reduced because you are getting feedback early, and
Customers are happy because they can make changes without paying exorbitant costs.
By looking at this you might think, well that is settled then. Let’s all go agile. It is however not that easy and you will see that there are for both methodologies pro’s and con’s. The structure of your project will decide, which methodology will work best for you.
I have found that when looking into applying the the agile methodologies, in my opinion, the following pros and cons can be defined:
Pros
Agile promotes some of the best practices found in development environments. Some of the risk in a project should be reduced as the output of developers is reviewed early and constantly during development
When projects are genuinely new they usually require creativity. Requirements can then emerge as understanding matures and grows.
Flexibility can be higher than traditional methods - although this is not guaranteed. Changes (e.g. in prioritisation) can be introduced at almost any stage.
Agile encourages or requires frequent communication between developers and those who will ultimately accept and use the deliverable. This should pay major dividends when effective. For example, feedback can be incorporated into future iterations as increments are delivered and reviewed by users or a Product Owner or both. False assumptions made by developers can be recognised very early reducing impact. Agile gives us continual opportunities to learn via this feedback.
It should reduce the 'silos' that too often exist within project 'teams' - something that always damages projects (as it should result in a collaborative style of working).
It should result in far less re-work on projects as issues and changes should be picked up much earlier.
Collaboration is usually much higher with Agile. Although not guaranteed, this can result in more successful development environments, in terms of product quality (i.e. fit for purpose).
With the common use of visuals, (boards, charts, Kanban etc) it can be easy to see what is going on (literally) what is done and what is yet to do, assuming the visuals are well organised, presented and up-to-date. This in itself can be a major benefit.
Cons
Agile is simple to understand in principle but hard to do well in practice. It requires real commitment and first attempts are not likely to go very well.
It is less predictable what will be delivered at the end.
Agile requires high levels of collaboration and very regular communication between developers and users (e.g. Product Owner). This is always desirable but may not always be feasible and requires commitment and time from the business.
Agile is very intensive for both developers and users. There can be reasons that may prevent this for example if developers work on multiple projects at one time.
The only downside to the opportunities to learn is that people have got to be prepared to.There can be less of a blueprint of what the final deliverable will be. This can make it harder to gain commitment to the project by stakeholders at the early stage.
Agile can be challenging when there is a supplier-customer relationship. Customers typically want to know what they are getting for their money as early as possible. It can be far harder to estimate timescales and costs as there is less 'definition' to base estimate
Agile can be very challenging on much larger projects or where co-location is not possible (between developers and the business).
In Agile there can be a great reluctance (by some) to adopt or accept deadlines. Projects don't exist in isolation so when this happens it can be a real issue. Agile methods typically only address the product development and large-scale projects can be made up of many other elements. Two other obvious examples where agile methods are not typically strong are: dealing with lead times and major dependencies.
When we are looking into applying the the traditional or waterfall methodologies, in my opinion, the following pros and cons can be defined:
Pros
On well managed projects Waterfall may provide more confidence of what will finally be delivered earlier in the life-cycle.
Project team members don't need to be co-located although the risks associated with this must be managed carefully.
Where large-scale design or analysis is required, or the impact of downstream changes to design is very high, this is likely to be a far more suitable approach.
Where there are many interfaces and dependencies outside of the basic product development, waterfall projects tend to have the tools to model and manage these.
Cons
Many organisations and people really don't find defining requirements (up front) easy to do - especially early in some types of projects. The assumptions upon which early stage plans are based may be very flawed and too often are taken as being based on certainty.
Communication can be a far higher risk - especially when there is limited early review of outputs and deliverables or when one-way methods of communication are used to convey requirements
Risk in general can be far higher with Waterfall, for example as the scope for invalid assumptions is unlimited. If you add this to the high cost of making changes later in a Waterfall project, it is easy to see why some are very expensive, over budget and late. Too often, assurance of products being fit-for-purpose is demonstrated very late in Waterfall projects.
Waterfall projects don't have to be but tend to be made up of 'teams within teams'. Having teams,within teams can be a major disadvantage to any project.
So as you can see Agile and Waterfall are very different and it will not always be possible to choose between them both. Waterfall could be applied to virtually any type of (IT) project. Agile requires specific conditions to be in place to be possible but is not applicable to certain projects, especially those of a large physical nature. Most of the conditions required for Agile to be possible relate to the working environment and practices that can and cannot be employed by the whole project team, not just those responsible for the development. There also needs to be flexibility around requirements together with the capacity to deliver and accept product incrementally.
I myself have lead multiple project over twenty years applying both the traditional or Waterfall methodology and the Agile methodology. I have found that for me, as described above, it depends highly on the sort of project and the project organisation, which of the methodologies works best. I like for example to implement a hybrid method where under a “Waterfall like” Milestone umbrella planning, the individual milestones are delivered “Agile”. This will provide senior management with some outlook on deliverables, cost and scope, but will incorporate all the Agile pro’s in the delivery of these milestones.
To find your preferred way of managing projects, the most important deciding factor will always be: What will work for your project, your organisation and your team.