Agile Realities, or How to Be Flexible on a Budget
Hello, I am Scott DiPerna. I have been working in web strategy, design and development for the past decade, managing and directing projects large and small.
Today I would like to discuss how to use agile practices in project work when constrained by budget and timeline — especially for agencies who are providing web design and development services for their clients.
In our business as a web agency, we are hired as a vendor to complete projects that are intended to have an end.
Unlike development teams who work on a product within a business, and who may have greater flexibility with the time they have to deliver the product, agency vendors are always working against the financial constraints set by our clients.
We like to apply agile scrum methods to our development work, but this can be a challenge when you have a fixed budget and timeline because too much iteration on one or two features will very likely mean other desirable features will need to be cut from the project to fit within the budget.
Many Ideas for the Website
Generally speaking, most clients come to us with an idea what they want and need for their website. Often they come with many ideas, all competing for time, attention, and resources.
Ideas Exist at a Fairly High Level
When we begin to talk to clients about their project, their thinking about the site still exists at a fairly high level and in somewhat idealized terms. They’re often thinking about how to solve all of the problems they face, or hoping to solve several within a single project.
It’s not uncommon for clients to talk to us about solving problems that aren’t related to the website. In short, they’re thinking big picture. They’re thinking about their business problems — not necessarily their needs for the website and how those will address business problems they have.
That’s an important distinction to keep in mind when speaking with clients.
Ideas Are Rarely Measured Against Constraints
It is also rare that a client has the perspective or experience in the field of web development in order to evaluate those ideas against likely constraints, such as time and cost and feasibility.
In other words, they have a lot of ideas of what they think they need to solve their business problems, but not a good idea of what is realistic for their means. Not only are they thinking big picture, but that picture may be a bit “pie in the sky” as they say.
Ideas Are Rarely Articulated in Necessary Detail
Given that lack of experience or expertise in web design and development, the ideas clients bring to us aren’t typically articulated in the necessary detail to understand the costs and complexity of what they want, let alone enough detail to actually build it.
We don’t expect them to have all of the details, but it is important to appreciate the fact that the details are not established yet.
The Devil Is in the Details
And let’s be clear, the devil is always in the details.
When we talk about project features at a high level and in general terms, it is easy to misapprehend that two people have the same understanding of what that feature needs to be. It’s only when we dig into the nitty gritty details that we begin to see how differently two or more people may understand it. This is because at a high level understanding, everyone makes a number of assumptions about the details.
The Most Labor Intensive Work of Any Project Is Eliminating All of the Assumptions
The most labor intensive work of any project is eliminating all of the assumptions and gaining alignment on the details among everyone involved.
Given that our client potentially wants more from project than they can afford, and our understanding of those needs probably isn’t detailed enough to know how it should be done or which features are most important for solving the clients real business needs, our definition of the work and our estimates undoubtedly will need to be revised.
This re-evaluation of our understanding and re-estimation of the work required will need to happen throughout the project — likely every two weeks as we complete each sprint.
Estimates Are Educated Guesswork
The reality we need to understand is that when we estimate proposals in the sales process, we likely don’t know very much about what actually needs to be done. The estimate is based on a high-level understanding of the proposed project as described by the client, which is essentially a less-than-scientific guess of what we think the client will need. It is a somewhat empirical guess, in so far as we have worked with similar clients on similar projects, but each project and client is unique in important ways. No two are exactly the same — and the circumstances of every project are always different (different people, different personalities, different expectations, etc.).
This Client Doesn’t Exist
The one thing every client has in common is that they all have limited budgets. In over 15 years of professional experience, I have yet to meet a client without financial constraints. Their budget may be less than your estimate, or it may be more, but there is never an endless source of funds.
The fact that clients have budgets reveals some weaknesses in agile development methods when performed by contracted vendors. Clients have an idea of the final product they want and they know what they want to pay for it. Few have a tolerance for not-knowing how long it will take, how much it will cost, or how they’ll hold you accountable for getting what they want in the end.
When you describe an agile development process to a client, it may sound to them like an endless roller-coaster ride they’d prefer to avoid. We see the cycles of iteration in project work and see the inherent value of that approach because we work on development projects all the time. We know from experience that starting simply and applying detail over time is very efficient for addressing change in a project. Clients may see those cycles and feel motion-sickness.
How will they know what they are getting at the end?
I Know What I Want
I think of it like buying a car. Imagine that you know you can spend $20,000 on a car, and when you look around at other cars on the road, you love the BMWs you see. Now imagine, with this picture of your dream BMW in mind, the dealers says to you, “Well, you need this car to drive between home and work, right? Let’s build you a minimally viable car for $10,000 first, in order to achieve that goal, and then we’ll see what features you want to add after we satisfy your basic needs.”
Minimally Viable Product
That may sound okay in theory, but in your mind, you can imagine how this approach may lead to a misadventure that ultimately yields nothing close to the BMW you wanted. Sure, this vehicle can get you to work and back home, but it’s no BMW!
The dealer continues by saying, “We’ll keep adding things iteratively, piece by piece, until you have a car that is satisfactory to you. This way, you’ll only pay for the things you want most, and you’ll be able to change your mind along the way!”
It’s possible it may cost less than the $20,000 you have to spend, but really you have no idea if, after spending $10,000 for the MVP car, the remaining $10,000 will get you anywhere close to the car you want.
And who knows if choosing parts and pieces along the way will add up to something you actually want? When you go to buy a car, you want to know what you’re getting for the money you are paying for it.
Let’s See What Happens!
When purchasing a product, taking a “let’s see what happens” approach to building it is not going to instill you with great confidence about the outcome. And when a lot of money is on the line, that’s unlikely to be a risk you are eager to take.
Clients view their web projects in the same way. They have an amount of money they can spend. And guess what? They have a funny way of demanding to know what they can get for that budget. A “let’s see” approach won’t cut it. Clients want some certainty that they won’t be throwing their money away.
Also, a fully functioning fish-mobile is a steal at $20,000.
Even if the Product Owner with whom you are working understands agile in theory, she may have a very hard time selling that to the stakeholders to whom she must account for the outcomes of the project. As a Product Owner, she will need to be able to manage the expectations of her stakeholders by being able to explain the plan for what we intend to build, how we’re going to get there, and what assumptions and risks potentially stand in the way of success.
We have to make this clear and accessible to our clients in order to gain their trust and confidence in the project.
A Goal Without a Plan Is Just a Dream
Being agile does not mean you don’t have a plan, or that you don’t start with a plan that evolves as you progress through the project. You have to be able to commit to building something complete and launchable within the budget the client has. Even if they don’t ask for it explicitly, trust me, your Product Owner wants a plan for a finished product.
But how do you do that and still offer the flexibility to make changes that nearly every client discovers they want in the midst of the project? How do you allow for change without killing the budget? And how do you create a plan without creating so much detail that any introduction of change along the way inevitably causes massive re-working of the plan?
Don’t Finalize. Move Forward.
This is the key. The fundamental idea here is to create only enough detail for the client to agree that you are moving in the right direction and you should continue in that direction.
We aren’t looking for “approval” or finality — in fact, we don’t use the word “approval” when talking to the Product Owner because it implies finality. We simply ask if we’re moving in the right direction and if it’s good enough to continue moving forward. When a client thinks this moment is the final opportunity to comment or make changes, they will feel the urge to keep iterating until it is perfect.
Perfection is the enemy of progress, especially when there is a budget.
Define Just Enough to Move Forward
We only want to define enough detail in order to move forward. We start with rough shapes, and continually refine our understanding until we know enough about what we’re doing in order to justify adding greater specificity. You can’t do all the things at once, so start with foundation and build up.
Discovery & Definition
As a simple foundation for a web development project, it is common to spend several weeks on discovery, definition and strategy. During that time, we discuss with the client their business goals, how the website needs to support those business goals, the needs of the audiences and visitors who will use the site, and what actions and outcomes we want from those users which we hope will lead to achieving the business goals. We start at a high level (like business goals) and work our way into greater detail (like strategies for specific personas and pages).
Out of the discovery and definition work, we will have gathered enough information and detail on the project to organize features into buckets of logical functionality the client can understand and discuss. We call those large buckets epics. Some examples of epics for a website may be the homepage or a blog or an events calendar. They are fairly high-level features the client will easily understand.
Breakdown Epics into User Stories
We then break the epics down into smaller chunks that would make sense to develop as a single piece of work. We write user stories for those smaller chunks and give them rough estimates in story points. So, for example, if we breakdown the Homepage epic into smaller parts, we may define distinct components like hero section, testimonial section, call to action block, etc.
“Just Enough” User Stories
The stories at this point are fairly general. They don’t capture all of the requirements that are likely going to be needed, but they are “just enough” to give us a sense of their level of effort so we can estimate them, and therefore enough for the client to understand their cost.
For the hero section user story of the homepage epic, we may write something quite simple like,
- As a website editor,
- I want to be able to add to the top of the homepage a large hero image, a tagline, and a call-to-action button that links to an interior page of the site,
- so I can feature important and timely information from the interior of the site on the homepage.
That’s not enough detail for someone to actually build that component, but it’s enough information for a designer to roughly estimate the time to design it, and for a developer to estimate the time to build it.
Estimate Just Enough to Move Forward
In order to work within a budget, it is critical to be able to have some kind of estimate for all of the work you may do — even if they are simply big guesses at this point.
Estimate near enough to the level of detail you have in order to move forward. But most importantly, have some kind of estimate for every piece of work that may need to happen in the project based on your discovery with the client.
Constrain Unknown Work to a Time Box
Often times project teams will push back on this idea of roughly estimating everything. They will say things like, “We know the client absolutely wants this feature, but they’ve been unable to give us any information to even ballpark a guess. What do we do?”
The answer is to use a time box.
A time box is a somewhat artificial constraint of time you are budgeting for a particular feature or piece of work in order to give it a limit. Time boxes are assumptions we make, and thus pose risk to a project since we don’t know whether the time constraint we have set is realistic. We will want to replace those time boxes with a real estimates as soon as possible. Think of them as a temporary placeholders to account for features with unknown requirements.
In order to minimize the risk of the unknowns in a time box, it may also be wise to add in estimated time for a spike.
A Spike in Spikes
A spike refers to a planned spike in effort to investigate and answer some of the questions that require us to use a time box. It’s important to understand, and ensure your Product Owner understands, that a spike in effort means a spike in time and budget being utilized in order to do research or additional discovery into an unknown.
The purpose of both the time box and spike is to mitigate budget risk by constraining the amount time we intend to spend digging into a specific problem, answering a specific question, or building a specific solution.
Plan Just Enough to Move Forward
Once we have estimated all the stories of each epic in points, we look at the total level of effort to see if that can reasonably fit within the client’s budget.
If it can fit, great! If it doesn’t fit, we evaluate which epics we believe will provide the greatest VALUE and IMPACT for achieving the client’s business goals. We then recommend to the client which epics we think should be included in the plan, and which should be moved to the parking lot for a later phase of development.
It’s critical to estimate everything, so that the Product Owner can choose what features they want to fit within their budget.
Once we reach agreement with the client on which epics and user stories will be included and fit within the budget, we put those epics and stories into the backlog for the client to then prioritize based on business value and importance. This is not actually a promise to complete everything in the backlog. It is simply a description of what we think we can accomplish within their budget BASED ON THE INFORMATION WE HAVE AVAILABLE TO US NOW. That information is going to change over the course of our work, so what fits in the backlog (i.e. what fits in the budget) is going to change. We will continue to monitor and re-evaluate that after each sprint.
Design Just Enough to Move Forward
When we begin to consider the visual aspects of the site, again, we only do enough to answer the most immediate questions without delving too deeply into all the detail of the site. Like painting a portrait, we establish a foundation and then continually build upon it, layer by layer.
Move Forward with Style Direction
That process may begin with simply designing mood boards or style tiles to show the client some options around look and feel, typography treatments, colors, graphics, and image examples that will help the client think about their visual identity without getting caught-up in the details of page layouts, messaging, information hierarchy, and content organization. Once we move forward with style direction, we can begin to design static comps of primary pages, typically beginning with the home page.
Limited Static Comps
With the basics of identity already established, the client can now focus on the structural details of specific pages, starting with the home page and moving on to one or two key interior pages that will help us define the full design system. A “move forward” approach to design does not typically include creating static comps for more than a few pages. The reason for this is that page design is likely to be tweaked and changed as we move into HTML and CSS, so spending a lot of time producing static comps that will quickly be out of date is a waste of budget.
It is preferable to get into code as soon as possible once the website’s styles begin to take shape because in the design system we are building, styling changes in the CSS will apply to all pages where those style exist. Designing the remaining pages in HTML and CSS templates will streamline the process of integrating the design into the website’s CMS. This design process is a good example of doing just enough to progress to the next step, while also allowing for change to happen throughout the process.
Build Just Enough to Move Forward
Once we have a design system in place with prototypes of pages, and we have a set of estimated epics in which we are comfortable being able to complete within the budget, we begin development sprints. In development, we follow the same philosophy of building only enough to move forward with more work. For example, we build data structures and content types first to establish the foundation of the site before we do any theming or page building. This enables us to expend the minimal amount of budget to validate the data model with the client prior to building a lot on top of work that may need to change once the client sees it.
The development process uses many aspects of agile because it aligns well with the “Move Forward” approach. We work in two-week sprints. We plan our sprints by committing a number of user stories we think we can accomplish in those two weeks based on our estimates and team resourcing capacity.
We write acceptance criteria for every user story so that we can test and validate that we have achieved the intended outcome when the work is complete.
We meet as a team for daily fifteen-minute scrums (usually in the morning) to discuss what we are working on and flag any blockers. At the end of each sprint, we demo our work to the client, and deploy it to a staging environment so the client can do their own testing.
And everything goes perfectly according to plan.
But then change happens and muddies the water.
Inevitably, when the client begins to see how the functionality of site actually works, they want to change and adjust things. This is very understandable. Up until this point, discussions around the site have been abstracted from real, hands-on interaction, but now the site has become more tangible. The client will understand the site (and the project as a whole) more and more after each sprint, which will lead to more adjusting along the way.
It will be tempting to simply create new user stories for changes as they come up and toss them into the backlog, like you would stuff clothes your grandmother made for you into the back of your closet.
But that is a bad idea.
For one, it avoids the difficult question of how this new work will be done within the budget since we already have a scope of work we set to fit within the budget. Two, it doesn’t bring awareness to the client that they will need to choose between this new work and the functionality they’ve already requested. Three, it may hurt grandma’s feelings.
Please Don’t Disappoint Grandma!
So, what do we do with change to ensure that no one ends up disappointed?
(simply putting it off until later doesn’t count0
We manage it.
Change has to be addressed continuously throughout the project. It needs to be given shape and size and a place to live. This happens on a continuous basis, sprint to sprint. New scope will be identified by the client, and you will need to capture that as a change or new feature, estimate it, and then put it in a place where the client can see the size, cost and impact of the new request.
We often call that place the “parking lot.” It’s where we place the things that don’t fit yet. It’s a place that is visible to the client, but is not in the backlog. Only work that fits within the budget goes into the backlog.
Let the Product Owner Choose
In order to add something new into the backlog, the client Product Owner will need to remove something of equal or greater size from the backlog. These are tough decisions to make, but they are not your choices to make for the client. Empower the Product Owner to make informed decisions by ensuring they understand the size and effort required for the work, along with any consequences related to the changes or additions.
Backlog Is a Fixed Container
In other words, the backlog is a container with a fixed size. Only the budgeted amount of work can fit into the backlog. And the client must choose which things to include. If we are working within the confines of a budget, we can’t continue to add work to the backlog without end,
like stuffing clowns into a car.
Our job is to identify the shape and size of the additional work, and present the realities of the desired work versus the budget. The Product Owner can decide how important the new feature is, whether it is worth more than other features in the Backlog, and whether they would be willing to sacrifice those lower-priority features in order to have the new feature.
It’s true, clients won’t love having to choose between all the things they want, but they will be even more unhappy if the project runs over budget or can’t be completed. Having a choice at least gives them some agency in controlling their project. And ultimately, all of our clients want to have as much agency and control in the project as they can.
When the project is complete, and you have delivered the most important features within budget, and all of those demanding stakeholders are satisfied, your client will be very grateful for your guidance and discipline.
They may even thank you.