The do’s and don’ts of building useful proof of concepts

And how they increase efficiency in software delivery

Once in a while, the technical checks you need to do while refining new features will require you to code a small sample project. While these proof of concepts are an essential part of modern software engineering, using them correctly can be very challenging. Stakeholders might be reluctant to invest the necessary time since it is basically working on throwaway code. Or even worse: they might try to convince you to continue building on top of the prototype to recover some of the efforts. But none of this should be an issue: let me guide you through some tips and tricks that help me and the team stay on the right track and ensures that we use our time wisely.

Know why you’re doing it

Building software products is expensive, so you don’t want your engineers to waste time on building the wrong product. Whether you’re in an agile context or not, conceptually you always have a discovery phase (what am I going to build?) and a delivery phase (actually building it). It’s easy to see that changing ideas while you’re in delivery will always be more expensive than doing it in discovery. Creating prototypes and proof of concepts during discovery are a great help to reduce that risk.

It’s important to understand that proof of concepts and prototypes are not the same thing. Some prototypes can still go very broad in scope, since they are mostly mockups or low fidelity designs which are low effort to create. This is not true for feasibility prototypes or so called proof of concepts: they are more time consuming to create since you need to write code. The good news is that you can easily counter this concern by keeping a strong focus on the scope of the engineering effort required.

We all know that there are many forms of prototypes, and they all share one common goal:

The overarching purpose of any form of prototype is to learn something at a much lower cost in terms of time and effort than building out a product. — Marty Cagan, Inspired

It’s all about validating your concepts and understanding the risk you take when you start building. This risk can take many forms, so be sure to know which one you’re tackling:

• Usability risk: will users understand how to use my product?
• Feasibility risk: can my engineers build it?
• Value risk: do my users actually need this?
• …

The biggest mistake you can make here is that you want to validate multiple risks with one prototype. If you’re combining everything in one prototype, then how is that any different than actually start building the product without validating it? If you encounter this scenario: sound the alarm and set the goals straight. Why? Well, each type of risk has its own methods to analyze and mitigate the effects, with each method having its own focus and its own needs. Combining them, drastically increases the scope of the prototype you create.

Let’s focus on the feasibility

In my experience, covering a usability risk with wireframes and user tests is a common practice in most companies, but proof of concepts to cover a feasibility risk are often overlooked since they are considered to be expensive, especially in an agency context.

Reducing this risk, also means including engineers in your product discovery. They will help you understand how easy it is to build the product you envision and how to make sure that you can actually deliver on your goals within the timing and budget constraints. Sometimes, people are reluctant to include engineering in this creative process, claiming they have enough experience to know that a problem has already been solved by another party. But in fact, it’s very easy to overlook the subtleties in your use case that determine what is technically possible. Even if it is indeed possible, it doesn’t mean that building it is reasonable within your constraints.

Don’t consider engineers to be showstoppers in this process. It’s true that not every engineer is suited for discovery, but you need at least one of them who is. If you consider every engineer to be a showstopper: ask yourself if it’s not just you who only started to include them at the point where you already fell in love with the idea before actually validating it.

Even the most senior mobile developer will not know all the details about the Android and iOS APIs. Most of these APIs are only used in specific scenario’s, and probably your scenario has its own very specific edge cases. This is where proof of concepts come into play.

Consider the example of a mobile app that needs to communicate with a Bluetooth device while your application is in the background. If you’ve never done this, you could start investigating the relevant documentation of Apple and Google. But if your app really depends on this background use case, I wouldn’t advise to just draw your conclusions only by reading specifications. You can’t just take the assumption that everything will be fine. You need to test it and really experience what the limitations are. It’s clear that you can’t pinpoint all of these concerns with product managers alone, you need engineers to help you in this process.

Start with making a checklist

But what exactly are you going to test? And when do you consider the test to be successful? At what point do you have enough confidence that you can actually build the real product? You definitely need an answer to those questions before you begin. So let’s take our Bluetooth example and break it down.

• You need a Bluetooth connection with a real device to validate this
• You need that device to behave like the final product will do. Will it send a continuous stream of data? Or only once every 30 minutes? This also means that this device can be a mock device for the purpose of this proof of concept, but it definitely needs to communicate over Bluetooth to correctly validate your scenario
• You need a sample app that interacts with this device and can be monitored to understand its behavior when going in background mode
• You want to see the effects of real life scenarios such as going out of Bluetooth range, can you recover from them?
• Will the performance be acceptable?

With this set of requirements, you should be able to correctly validate your use case. See how we didn’t include things like Bluetooth pairing (you can connect to a device without pairing), a user interface, or anything that needs to be done with the data once it arrives on the phone (e.g. sending it to the cloud)? We didn’t do this because we know our team is already familiar with all these topics and we know the constraints. Don’t forget to align on this list with all involved stakeholders. These are the research questions for this proof of concept, nothing less, nothing more.

This is really the clue of correctly scoping a proof of concept: don’t test what you already know. It only adds to the effort required to implement the proof of concept, but it doesn’t deliver you any new insights. Remember: we don’t want to waste any time on non important edge cases. Cutting corners is a good thing in proof of concepts! Our one and only goal is to validate our assumptions in the most time efficient way. The time you spent on a proof of concept should be limited: just enough to answer your research questions. If it takes more than a fraction of the actual time you’ll spend building the full product, you’re doing it wrong.

Continuously think about the goals in your checklist

Suppose you’ve just implemented this proof of concept. Now it’s time to validate it, you might even give it to your quality engineer together with the agreed requirements to stress test the sample app. Afterwards, you can take the results back to your team.

In the case where everything went successful, you’re actually done here. The proof of concept has served its purpose, you know what you have to know. It’s likely that new concerns will arise once you have an answer on the initial research questions. Be cautious here: before you continue building on your proof of concept to start answering these new questions, go back to the drawing board and ask yourself this:

• How crucial are these new concerns for your product flow?
• Is it wise to continue building on this proof of concept to test the new concerns? Or is it faster to start from scratch because you cut too many corners that will introduce overhead in this scenario?

As soon as you’ve covered the crucial risks for your product, you should stop working on this proof of concept. You’ll reuse knowledge when you start on the final product, not code.

If the proof of concept was not successful at all, you’ve also learned a lot. It might mean that you can’t reach the goals set in the discovery phase. You can now start looking for alternatives or just decide to drop a feature. Are you a showstopper now? Definitely not, you just prevented to waste a lot of money in the delivery phase. Don’t underestimate the engineers that worked on this validation, while implementing this proof of concept they might even have come up with creative technical ideas to reach the same goals with a different approach. Reflect and iterate.

TL;DR

• Keep focus, a proof of concept should have a specific goal and should only be used for that goal
• List your concerns, if an assumption can really make or break your product and you have doubts about it, make a proof of concept
• A lack of knowledge on very specific topics is normal, don’t feel ashamed to propose a proof of concept
• Don’t test what you already know
• Create a checklist upfront, agree on it and do your validation based on that list.

If you’ve followed these steps, you will certainly be able to better guide the other members in the discovery process. The confidence that you can build the envisioned product has grown and you don’t need to set crazy high estimates to cover up for a feasibility risk. If at this point an estimate is still high, it really means that it’s a lot of work. Can you be 100% sure of the amount of time you’ll spend on it after doing proof of concepts? Absolutely not. You’ve only checked the crucial assumptions and are now much more familiar with the technology and constraints on your product. Isn’t that one of the most important prerequisites for smooth software delivery?