How to use Agile delivery in a “Non-Agile” world

Often I hear a business owners of (senior) business executives telling me that their organisation is not able to adopt the Agile methodology, due to the fact that they are bound to Non-Agile business contracts. They see no way of adopting the Agile principles within their organisation. having worked myself in big organisation for almost 20 years has taught me that the world is not black and white. If there is a will (or the right mindset if you like), there is a way forward.

Whenever we learn a new technique or process, we tend to start our journey by focusing on the “ideal” scenario or “pure” approach. Approaching a new idea in this way provides a basis for the intellectual debate, opposing viewpoints and perhaps new conclusions, all without the real world consequences. This approach also allows us to work out the underlying foundation of the technique, without having to worry about annoying details or complexities that might get us mired down and prevent any real progress. But as one might have heart saying, “Only in theory, are theory and practice the same.” What works in the lab doesn’t always work in the real world, and what works great in one scenario doesn’t always fit every other scenario.

It is the same in for example a software development environment. We learn new programming languages, methodologies, or techniques, debate the advantages and disadvantages of one approach versus another, and often choose a path we think is the best for us to focus on. “Agile” Development is, as mentioned before, such a methodology that is currently favoured by a large portion of the (software) development and delivery community and continues to grow in popularity, while older approaches like “Waterfall” or “Plan-based” development are considered obsolete or rigid and old-fashioned.

Supporters of Agile development often say things like “you should just use Agile”, “there really is no alternative to Agile”, “Agile is best”, or “Agile is better than the way you’re doing things now”. For them, they’re right. In their particular scenario, for the work they do, in the environment they work in, Agile is best path to take.

We all know that not all scenarios are created equal, Agile however, works best when certain basic assumptions are met, within the organisation and project framework. If those assumptions are not met the benefits of Agile are not as clear and Agile might not work the way it is promised to.

Some of these basic assumptions include:

Assumption: The customer is involved throughout the whole project delivery

Explanation: The customer (sometimes the end user, the person with the Big Idea, or a person or organisation contracting with you to build something) is involved at regular intervals during the project life-cycle. They help prioritise, mainly through close contact and conversations with the product owner, and define what needs to be done before the work can be considered done. They get to see what’s been built when it’s built. By continually reviewing and directing change, the customer is able to shape the product to fit their vision and business needs as their understanding of that vision evolves.

Challenge: Sometimes the customer is not always available. Sometimes the customer doesn’t even know anything about the system they’re contracting with you to build. Someone else told them to “do the thing”, so they issued a Request for Information (RFI) and/or Request for Proposal (RFP), took input from industry, and awarded a contract to a vendor who will “build the thing”. Sometimes the details of the thing were already determined by a third party – by law, industry standard practice, contract, prior work, or some other method or practice. In these cases, the assumption does not apply and the advantage it gives Agile is not realised. This doesn’t mean you can’t use Agile, it just means that you have to adapt. This requires a strong product owner, which has a clear vision on what “the thing” actually should look like. The great product owner, needs to able to derive a vision from the customers’ head, which they did not even know they had. It requires frequent and clear communication between the product owner and the customer

Other methodologies, including Waterfall or Plan-Driven Development, do not rely on the customer being involved throughout. They are designed so as to not require that level of involvement, and thus the processes they use do not suffer from its lack (though the final product might).

Assumption: Deliver early, deliver often

Explanation: One of the great benefits of the Agile methodology is that you get something in front of the end users early. Rather than building a giant giant system, you release once you’ve completed the Minimum Viable Product (MVP) or Shippable product, and continue to release as you implement new features or refine existing features. Facebook launched with far less than it has today, as did Google and Twitter. you will often feel like the app or site you are using today isn’t the same as it was yesterday – and that’s because it probably isn’t.

Challenge: Sometimes you can’t deliver early or at least you think so. Contract terms, legal requirements, or other factors may dictate that an early launch with anything less than 100% is unacceptable. I have worked on contracts to build systems, which were going to replace an existing system. We could not launch with anything less than the same functionality the existing system has. Sometimes the MVP and the Final Product have the same. In these cases, the assumption does not apply and the advantage it gives Agile is not realised. This doesn’t mean you can’t use Agile, it just means that you’re will have to be a little bit more creative. Deliveries will be done in Test or User Acceptance environments, where key users can have a first peak. While the end product is not launched yet, this does not mean you cannot deliver early or deliver often, the delivery will need to be done in a different environment to have the benefits

Other methodologies assume there will be but one delivery, once the system is built. They are designed around full completion, and include steps which explicitly work towards that goal.

Assumption: Prioritise by Business Value/Technical Risk

Explanation: This goes hand-in-hand with deliver early, deliver often. Because we build the most “valuable” first (and include items which may not have direct business value but are critical for technical reasons), we can be sure that the MVP launch will be compelling and make people want to use the system. The specific details of prioritisation include a number of factors or methods – for example something that is very easy to do might be done in an earlier sprint because there’s room in the schedule to fit it in, while a more valuable but complex feature is delayed until a later sprint. Risk should always be a factor in determining value prioritisation, as well.

Challenge: Sometimes value is irrelevant. Everything has to be built, whether it’s “Mission Critical” or “Nice to Have”. This is often the case with government regulations or contracts – any time a government contract states that a vendor “may” implement a feature, you can put your money on the vendor not implementing it. If everything is required, and you’re not delivering until everything’s done, the order in which you implement things isn’t necessarily as important. In these cases, prioritising by business value may even get in the way. Either way, the benefits Agile receives by this prioritisation aren’t realised. This doesn’t mean you can’t use Agile, it just means once again the “pressure” is on the product owner. The product owner, together with the business stakeholders will need to be able to come with an MVP for all features and implement that first, and expand on it in later iterations.

Other methodologies may include different guidelines for determining the order in which work is done. Riskier features may be implemented sooner so that they can be more thoroughly tested; or might be delayed until the end to see if the risk materialises and makes the feature irrelevant. Political pressure may dictate priority. No matter what methodology you use, though, you prioritise; and any methodology’s prioritisation scheme can be adapted to fit project realities.

Assumption: Change happens

Explanation: Agile embraces change. It’s built around the idea that change is going to happen throughout the project, and by identifying that change early you are better able to build the right system, rather than the system as envisioned by someone months or years ago. Agile’s embrace of change also ensures that if someone made a mistake in the feature needs, the mistake can be caught and corrected sooner.

Challenge: Change comes with cost. If we’ve implemented something and need to go back to change it, something else has to slip – the schedule, the cost, some other feature – change does not happen in a vacuum. The type of change we’re referring to here is a change to something which has already been completed, which was correct at the time it was implemented but is no longer correct due to a change in needs, discovery of some previously unknown factor, or some other reason. We are not referring to completion of a previously known-to-be incomplete feature, such as implementation of a data validation rule which we knew would need to be implemented but did not yet know what it was. The same is true if some new feature needs to be added that wasn’t in the original list. In the case of a Firm Fixed Price (FFP) contract, which is by far the most common scenario we encounter when bidding on government or non-governmental work, the contract may not permit for reduction in features, addition of new features, additional funding, or delayed completion – at least not without significant penalties and/or a contact modification. Yes, we should embrace change. Yes, it’s better (in the long run) to catch the change early, but if the contract doesn’t let us change the work we’re contracted for or the money we’re paid for it, we don’t have much choice. If we can’t get a contract modification, we can’t accept the change, although the features we have been contracted to build might still be object to changes (maybe on a much smaller scale) and we still should be able to do so. the Agile methodology, will help to define a mismatch from business requirements and contracted features early and will provide you with a bases for discussions on contract modification, if needed. I would consider this also embracing change.

If you can’t embrace change at all, the strength that doing so lends to Agile isn’t realised. Other methodologies are designed to reduce change during project execution – change can only occur before the project starts or after it has been completed (usually as a follow-on project, which often will lead to a very unhappy customer after the delivery.)

Assumption: You don’t know everything in advance

Explanation: This goes hand in hand with “change happens”.  In the old days of software development, the ideal was that you would fully document the system-to-be; then someone else would develop it. This pretty much never works as well as planned, though there have been some notable exceptions where this methodology worked far better than is typical (see for example “Cloning the IBM PC BIOS”). Agile assumes that you don’t know everything in advance, but rather, figure things out as you go. Yes, advance knowledge helps shape the project, but when you don’t know, you have to adapt to change.

Challenge: The government issues an RFP, and asks for FFP quotes. Because it is impossible to respond with a fixed price for an unknown, the government tries to know as much as they can in advance, so vendors can provide realistic bids, and so the government can identify if a bid is realistic or not. As with “change happens”, however, if the government fails to identify everything in advance they either (a) have to allocate more funding, or (b) sacrifice the features they failed to identify.

Of course, nothing in Agile says you SHOULDN’T know everything in advance – it just assumes you don’t. Other methodologies which are designed around knowing everything in advance, have built-in controls to ensure that as much as possible, this goal is met.

Assumption: Dedicated, co-located teams

Explanation:  Agile has a preference for (but does not require) face-to-face communications, with team members committed to the project and able to maintain a constant pace indefinitely. Daily stand-up meetings during the sprint, planning meetings and product demos at the beginning and end of the sprint, and other components of the Agile methodology work best when these assumptions are true.

Challenge: This works great when you can have it. Small businesses with multiple contracts of varying sizes and duration may need to be more flexible, able to change the team as practical demands dictate, and split peoples’ time across multiple projects concurrently. This often gets in the way of the higher degree of team synchronisation which Agile favours, although with the current modern technology and virtual teams this risk can be reduced significantly. With the right mindset of the team members and a creative Scrum Master clear agreements around timing, (skype) meetings and video conferences can be made. This will help to ensure the commitment and the synchronisation will be optimal.

Other methodologies which do not assume or require a high degree of coordination between team members or team member responsibilities may not be affected as much by the lack of a dedicated full time team, although it does promote forming teams within team and silo or island thinking, which will require mitigation to make sure everybody is still working towards the same goal.

Agile in a “Non-Agile” World

None of this should be taken to mean that Agile is a bad methodology or will not work– it isn’t. It’s great. The ideas of Agile have been around long before Agile was a thing and fall into a family of methodologies called Rapid Application Development. The idea is you start development early, continue to develop while you continue to refine requirements, test while you’re still developing, and respond to change as best you’re able. Agile formalises some processes and structures that were previously ad-hoc, and this formalisation helps the team plan ahead for all the things Agile helps to improve.

To be able to implement Agile development frameworks in my projects there were times when I could not just do it or recommend it as the appropriate project approach. More often than not, the decision comes down to two factors:  (1) Who is the customer and (2) how is the project funded? Sometimes I required to add additional external “agile resources”, like scrum master or product owner, or additional delivery environments (user acceptance, test) to the mix to be able to implement an Agile delivery

Working with the Government

When working with government customers, there is rarely an opportunity to deliver a partial product or system. As discussed earlier Government contracts usually dictate specific deliverables, and often the funding is set. There are no opportunities to get more money or modify the deliverables (well, we could probably add some free deliverables if we wanted to). This does not mean you can show the partial product earlier in dedicated environments. In a case such as this, usually I will offer a delivery framework using a combination of Plan-Driven and Agile techniques. However, in the end, we I am obligated in such contracts to deliver a pre-determined set of functionality; that is the risk we assume when we implement a contract of this nature.

Even non-government customers will generally have budget constraints, and want assurance that a specific set of functionality will be delivered at a specified cost. Agile doesn’t really support that requirement, as it is predicated on frequent change (and potential cost variability). Agile costs tend to be associated with the team size and time spent, i.e., a specific cost per sprint. Many commercial customers are unwilling to accept the risk of an open-ended delivery date and/or uncertain product.

Hybrid Methodology

Al of the above has lead me to favour a Hybrid methodology, where I, depending on what I agree upon with my customers, I will utilise a combination of iterative and more traditional methodologies to deliver projects. For instance, I may use a type of sprint internally within the development team as a way of organising the work. Utilising a backlog and frequent communications (like stand-up meetings) to keep the team moving forward.  I will conduct sprint reviews with the customer, but may need to limit (or completely curtail) changes, where I set out a Milestone plan, based on the more traditional “waterfall” method, so senior management can track budget and feature delivery.

Is this pure Agile? Not really, but I have long recognised that taking a purist approach to anything is not always the most productive path to anything. I (and my (internal) customers) can benefit from my knowledge and experience, and that is really our primary goal

I aways try to steer my customers and stakeloders to a more Agile understanding of their needs. I try to use Time and Materials (T&M) style contracts as much as possible, where we are engaged to do an amount of work rather than a specific set of tasks. We find that T&M combined with an Agile process reduces the risk for project delivery AND reduces the customer’s risk. The customer maintains the control they have in a pure FFP model, risk is shared, and the chances of success are greatly improved.

Always Be Agile

Even if a pure Agile approach to the project as a whole is not possible, you can still use it internally. Even if you are required to use a Waterfall-like approach to the project as a whole, you can still be Agile within each phase, delivery all the individual phases or milestones Agile. During development, you can still use sprints, you can still have a Product Owner, you can still do backlog prioritisation and grooming, welcome change, and manage risk in an Agile way.

Successful projects have been done using both Agile and Plan-Driven/Waterfall methods, and both methods have seen project failures as well. Software projects have many factors in their success, and development methodology, while important, is only one of those factors. The development methodology used guarantees neither success nor failure. When embarking on a software project, an educated customer and an experienced project team are both essential components for the success of the project.

Through education and understanding, I always work with my customers and stakeholders to best manage risk, deliver the right product on time and within budget, and ensure that my customers and their users are satisfied. I will always try to add Agile delivery in my delivery framework.