The state of Waterfall and Agile

When it comes to software development, there's always that million dollar question: which methodology to use, Waterfall or Agile? Everyone has an opinion about it, and often, preferring one methodology means disregarding the other as a valid option. Waterfall came first, Agile appeared more recently, but we're far from reaching a consensus about which better suits development needs.

Is Waterfall dead? Did Agile definitely take its place? Is one better than the other?

Starting from the very basics, here I'll clear what's the current state of both methods in software development, answering to some of the most common questions regarding each. All things considered, I'll also explain why we've decided for Agile instead of Waterfall in our own development process.

What is the Waterfall model?

The Waterfall model was named after its sequential phases that are arranged in a downward fashion, similar to actual waterfalls, representing the various steps of software development from one end to the other. It was published in 1970 by the computer scientist Winston Walker Royce, and it originally contemplated five different phases: requirements, design, implementation, verification and maintenance.

The Waterfall Method (umsl.edu)

Some variations of the original model emerged in the following decades, but the logic behind Waterfall remained the same. When a phase is completed, its output becomes the input for the next one, which starts immediately after the former. It contemplates all the essential points that compose software development and its simplicity made it very easy to understand and adopt immediately.

Waterfall ensures that each phase is completed before moving on to the next one, avoiding, for instance, starting development before the design work is completed, which could give way to possible incongruities in both ends. Additionally, through this model, it's possible to estimate the whole project's cost and effort needed right from the start, in the requirements phase.

While the Waterfall model was far from being the only possible approach to software development, it wasn't until 2001 that it experienced a real paradigm shift. It's cause? Agile software development.

What's Agile Software Development?

The principles of incremental software development were already being used through different scattered processes, but it was only in 2001 with the Manifesto for Agile Software Development, that Agile software development as we know it was introduced and popularized. This very straightforward document, put together by a group of developers in Utah, definitely broke the conventions and limitations of software development, providing a real alternative to the Waterfall method.

Agile Software Development (crmsearch.com)

Above stands a perfect example of what Agile aims to be. The different cycles - conventionally named sprints - seek to cumulatively deliver value, being part of a bigger picture that leads to project completion. That's where it differs the most from the Waterfall method.

Agile seeks to be more flexible, allowing regular increments and minimizing the time needed for planning by focusing on shorter timeboxes. Each iteration adds value to the product and provides working software as it concludes, planning the nearest step in more detail than the others that will follow.

Different subsets adopt Agile as a philosophy, similar to what happened with the variations of Waterfall. DSDM (Dynamic Systems Development Method), FDD (Feature-Driven Development), XP (Extreme Programming) and, probably the most popular, Scrum, all draw from Agile to execute software development processes.

Regardless of different opinions, it's a fact that Agile changed the way in which software developers decided their approach to new projects. Knowing what each methodology represented, what did this mean to Waterfall?

Did Agile mean the end of Waterfall?

There's almost as many people saying that Waterfall is dead as those who say that Agile is an insignificant trend. There's nothing wrong with having different opinions, but let's get down to the facts here:

Stack Overflow Survey (2017)

The Stack Overflow survey pictured above shows how Agile is, undoubtedly the main choice for software developers. Besides being the first choice, used by 76.9% of the community, the popular subset Scrum takes the second place with 65.2%. Further down, Waterfall can be found at the sixth place, with 26.9% usage.

At first glance, it doesn't look good for Waterfall. We can clearly state that Agile is the most popular method, and there's no way to argument against those numbers. However, it's a mistake to ignore such a high percentage of developers who use Waterfall. We wouldn't say, as an analogy, that Huawei is an irrelevant brand because it has just 14.6% of market share in the mobile phone business, behind Apple and Samsung.

It's a fact, however, that Agile exposed a few weaknesses and flaws of the Waterfall model, pointing out how hard it is to fix issues in previous phases and how it takes a whole process to have working software, with a high amount of uncertainty in between.

As the message spread, it became cool to be Agile, everyone wanted a piece of it and all processes became Agile as a consequence. No one is bragging about their Waterfall processes, every business, from marketing and economics to software design and development, wants everyone to know that they're Agile, even if they don't know themselves what it means. That's what's causing the whole fallacy supporting the argument that Waterfall is now irrelevant.

Agile Marketing? (antoniospecchia.com)

Everyone seems to be missing the point: neither Waterfall is dead nor Agile is the universally superior methodology for software development. Adopting one over the other for the sake of being better is just an illusion that often masks the inexistence of a proper process.

The current state of software development methodologies

Each software development challenge has its own particularities and, usually, the requirements phase in any method is the point in which the cards are laid on the table. However, before jumping right into action, there are a few questions regarding the nature and size of the project for which you should have an answer to.

Methods such as Waterfall and Agile exist as different approaches to solve variations of the same problems, and depending of their particularities, different projects may be better developed with either one or the other.

But when is, then, better to use each approach?

Waterfall is definitely the best pick for short-term projects. It was the best approach before Agile entered the picture and it still is as today. If you can complete a small project in a month, there's no real benefit in splitting it in several sprints in order to deliver value each week. It's best to focus on the end result right away.

Additionally, for projects in which there's no real point in delivering preliminary working versions or additional features besides the main goal, Waterfall is also the way to go. There's no real benefit in developing only a MVP with Agile, creating several sprints from a project that, given its nature, isn't meant to be split in more than a single phase.

Agile, on the other hand, is the way to go for mid or long-term projects that, given their complexity, require a more structured process. Also, it's the better method to ensure that the project doesn't stay in development for several months before presenting any result. With Agile, by the end of each sprint there'll be a checkpoint in which the project manager can test and approve the work that has been done.

The lack of proper checkpoints raises the risk in complex projects that adopt Waterfall, since it's harder to fix possible issues found by the end of development. The extra time that is allocated to plan the whole project doesn't ensure that the design and development phases will run smoothly right until the end, as most issues are as undesired as they are unpredictable.

A resumed comparison of Agile and Waterfall (officialconsumerreport.com)

In a similar way, just as you wouldn't use a flamethrower to get rid of a fly, you wouldn't pick Agile to develop a small app. The complexity of the means would cause more harm than good in both cases, even if in the end they would get the job done all the same. Even without considering the extra cost of time and effort that picking the wrong tool for the job would imply, it's obvious that choosing either Waterfall or Agile should be more than just a marketing move.

Why do we use Agile?

All in all, why do we prefer Agile to Waterfall? Given what we've seen until here, hopefully we're all past explanations such as "Waterfall is dead, long live Agile." As a general rule, one is not better than the other and, most times, it's a mistake to think otherwise. We've tried Waterfall for a long time, and made the shift to Agile when, given the nature of most of our projects, we found it to be the top pick.

In our Agile Development Process, we adopted a Scrum methodology, which is a subset of Agile, and made a few adjustments in order to meet the specific needs of our clients. We find this to be the better option for our projects, since we contemplate both the possibility of ending the project with a MVP - which would fall under the Proof of Concept phase - or go further through a set of sprints that will add incremental value to the product.

Once again, it would be a mistake to think that this is the best possible process to adopt in every project, and that's the reason why we review and adapt it to suit our needs when necessary. That's what led us to move from Waterfall to Agile in the first place, and if needed, further improvements will be made. However, it's never a question of creating universal solutions, as such a concept will never exist in software development.

Conclusion

It's clear as water, Waterfall is not dead, and Agile is not the best pick per se. That's the easier way to look at it, and also the most dangerous way of handling software development. It all comes down to your specific needs and to the best way in which you can meet the desired ends. Any further discussion about it just creates unnecessary noise.

What's been happening over the last few years is that being Agile is seen as a perk, a special characteristic that a business can get associated with that instantly proves its quality. It's neither everyone is Agile nor no one really knows what Agile is, but all the commotion makes it harder for anyone to truly understand what methodologies like Waterfall and Agile are all about.

The best way to go beyond looks and actually make a difference with a software development process is to truly understand the needs of each project, pick the method which suits it best and adapt accordingly.

At Imaginary Cloud, we simplify complex systems, delivering interfaces that users love. If you’ve enjoyed this article, you will certainly enjoy our newsletter, which may be subscribed below. Take this chance to also check our latest work and, if there is any project that you think we can help with, feel free to reach us. We look forward to hearing from you!