Between Waterfall and Agile
The Grey Area
What is the development theory that Steve Jobs used to develop the Apple II computer? Do you think that when Thomas Edison invented and mass-produced the light bulb, and many other inventions, he was thinking about what project development model to apply? What model was on the mind of Henry Ford when he revolutionized the automotive industry (besides the model T)?
If you have great visionary leaders who have a good understanding of the market, and talented people who master the state-of-the-art technology, with a bit of luck, you can develop a best-in-class product, and disrupt the market, without the need for any management/development theory. Following your success, people (not as smart as your people) will try to extract lessons from your good practices, theorize them, and sell them to other companies as remedies to their problems. Good practices will always help to improve the outcome, but will never fix the problem of narrow-minded leadership and unmotivated people.
This sets the ground for the sequel.
Product/Software Development
The development of a product/software comprises the following key activities:
- Requirements Gathering from the customer/user or market specialist. They define what the product/software is expected to do/be. Requirements are not technical solutions.
- Architecture & Design: this phase, which can be split into two phases in complex systems, translates the requirements to technical specifications. Architecture deals with the high-level design, and the definition of the interfaces and interactions between subsystems. Design deals with the components, functions, subsystems, etc.
- Implementation/coding of the specified design. This task can be done by the designer, by another developer, or automated.
- Testing: there are two types of tests, verification and validation (V&V). Verification checks that the design and implementation work as expected (e.g. unit testing, white-box testing). Validation checks that what we designed and implemented actually fulfills the requirements (mainly black-box testing).
- Release: in the case of software development, this phase entails the deployment and maintenance activities. In the case of a hardware product, it entails mass production.
There are different ways to perform these activities. The best way would depend on the type of the product and the market, the aggressiveness of the competition, the experience of your team, and so on.
The development practices fall broadly into two models: Waterfall (sequential, rigid, and one-shot) and Agile (adaptive, iterative, and incremental). In reality, there is a vast gray area between the two, and one can (or must) customize a model that is best suited for the project. Here are some models:
- Waterfall
- V-Model
- Sashimi Model
- Incremental Waterfall
- Unified Process
- Spiral Model
- Agile Mindset and Practices
Most of the models above are inspired by the software industry, by can be easily adjusted to other industries. For more details refer to Agility — Beyond Software Development. Here, software and product are used interchangeably.
The Waterfall Model
The Waterfall model is a sequential process, as illustrated in the following figure.
This model can be very efficient in the following cases:
- if the requirements are well defined and understood, and
- if the teams are skilled and have experience building similar product/software, and
- if no change (in requirements, design, or else) is required or might occur during the development.
Waterfall is for predictive and repeatable (business-as-usual) projects. Its major drawbacks are:
- No flexibility: once the design started, the customers cannot introduce any change in the requirements.
- Mistakes are costly: design errors and requirements misunderstanding are detected late, during the testing phase, when the error cost is the highest.
- Not customer-centric: the first release comes late in the process, and the customer/user needs to wait until the end of the development to test the product.
The V-Model
The V-model is an advanced version of the Waterfall that overcomes some of its shortcomings. By emphasizing the tests earlier, defining their acceptance criteria during the Architecture and Design phase, the team could avoid some errors and misunderstandings. This requires more upfront work. The V-model is presented below:
Besides reducing the mistakes (and their costs), the V-model has the same cons as the Waterfall model: no flexibility, nor customer feedback before the release. Thus, it remains better suited for the same type of projects as for Waterfall (predictive and repetitive).
The Sashimi Model
The Waterfall and V models are sequential and not adaptive; every phase relies on the previous one. They don’t allow for adjusting (shortening) the project schedule. They might result in poor resource utilization; for instance, during the design phase, the implementation team and testing team will be waiting and cannot work on the project.
The Sashimi model is a way to overcome these problems. It is based on the Waterfall model but allows for the overlapping between different phases when possible. The design team can start working on the requirements that were defined, while the marketing team is working on the rest of the requirements. The V&V team can test some functions while the designers are working on other functions, and so on.
This could shorten the development time and allow for better use of the skills and resources. The drawback of this process is that it may result in rework, and waste of time.
The Incremental Waterfall
Another variant of the Waterfall model is the incremental model. In this model, the project is divided into several parts that can be performed incrementally. This could allow for the overlap between increments (parallel workflow) to improve resource utilization.
Some projects are based on previous projects (product replacement, Kaizen, or next-generation), in this case, most of the upfront work and design are already there. The team can implement in increments to speed up the development and deliver value earlier. This is shown in the following figure.
Using the incremental model, the organization may profit from early value delivery, and get feedback and make necessary changes between increments if possible. The weakness of this model is that it may cause rework and increase the cost. This model allows for some adaptivity in the process, a value that is embraced by the Agile mindset.
The V, Sashimi, and Incremental models are variants of the Waterfall model. They work best for predictable projects, with rather experienced teams. The next two models (Unified Process and Spiral) are more suited for high-risk projects and allow for more adaptivity.
The Unified Process
The Unified Process (UP) is a framework that can incorporate other methods and models. It focuses on risk mitigation for big and risky projects. By allowing for iterative and incremental development, it increases the chances of success. It is a flexible approach, centered mainly on the architecture and use cases.
The Unified Process divides the project into four phases (with overlap):
- Inception
- Elaboration (milestone)
- Construction (release)
- Transition (final production release)
Even if the requirements are not all known early, the UP welcomes new requirements during the development.
By adopting the UP framework, the organization may profit from earlier value delivery. However, it is a rather complicated framework that may consume more resources. It is ill-suited for small projects.
The Spiral Model
The Spiral model is recommended for big projects with very high risks of failure (e.g. aerospace projects). It is a risk-driven cyclic model that comprises four steps:
- Determine objectives
- Identify and resolve risks
- Develop and test
- Plan the next iteration
These four steps are iterate over and over until we achieve the end product. The spiral model is iterative and incremental. It is adaptive and allows for changing requirements at any time during the development. It is a risk-driven that embraces the “fail fast to learn fast” mindset. It can incorporate other models and can be considered on the Agile side of the spectrum.
Agile Mindset and Practices
On February 11th — 13th, 2001, at The Lodge at Snowbird ski resort in the Wasatch mountains of Utah, USA, seventeen experienced software developers met to talk, ski, relax, and try to find common ground — and of course, to eat. What emerged was the Agile Software Development Manifesto, with 4 values and 12 principles that are summarized in the following figure.
Agile Practices
Agile is a Software development mindset that embraces change. It is neither a process nor a model, but a set of values and principles. It is a flexible approach for Software development that helps organizations to adapt fast to the market change. We can’t talk about the Agile process or Agile model, we rather talk about Agile Practices. They are proven practices to organize teams, manage workflow, track progress, and deliverables, etc. so that Software projects are conducted the Agile way.
The most popular Agile practice is Scrum, which is often mistaken for Agile. Other practices are Kanban, Extreme Programming (XP), Feature Driven Development (FDD), Crystal, Test Driven Development (TDD), Dynamic Systems Development Method (DSDM), etc.
Scrum is today an established project management practice, with defined roles (Product Owner, Scrum Master, Scrum Team) and rituals (sprints, daily scrums, sprint retrospectives, etc.).
For more details refer to Agility — Beyond Software Development.
