(A guide for non-technical managers)
Before making significant investments in a strategic R&D initiative, it is common practice to do some smaller tactical work to validate your approach, identify problems and ultimately to de-risk the eventual initiative.
There are many different names for this category of work, for example creating a proof-of-concept, a prototype or a working mock-up.
I have worked on many of these at both CityCAD Technologies and Charlbury Technology, and I wanted to share some tips on how to manage these and keep projects on track.
1. Make sure everyone has the same understanding of what is being done
Normally this is pretty clear, but be aware that there are fine distinctions between many of these terms, and there is a risk of misunderstandings. For example, the terms 'prototype' and 'proof-of-concept' are sometimes used interchangeably, and there are often overlaps.
Below I've listed my understanding of what each of these terms means. Some people may have slightly different definitions, but the important thing is that everyone on a project agrees on the same one!
It can avoid problems if precise definitions, objectives and success/failure criteria are agreed at the start.
- Wireframe - a diagram of the structure of a software solution, normally describing different parts or screens, what information needs to be displayed, and what is needs to do. Note this is different from a 'mock-up' as it may not show the planned design or appearance of the product.
- Proof-of-concept - a small project to validate a technical approach i.e. to check that the idea can work at a fundamental level. It does not provide a fully working version of the eventual product, and may be quite basic visually.
- Mock-up - a small design project to show what something looks like, normally a series of static images produced in a tool like Figma. While it may describe a workflow with a series of images it will not be interactive.
- High-fidelity wireframe (or 'Hi-Fi Wireframe') - this is a wireframe (see above) but with a greater level of graphic detail, to the point where it approaches the detail of the 'mock-up' and the interactivity of the 'prototype'.
- Prototype (or 'Working mock-up', which I consider to be basically the same thing) - this is a small project to create a working tool which gives users a good sense of what the final thing will be like. This is important for getting early user feedback and validating ideas about user experience or positioning.
- Pilot - this normally happens much later, and is a trial run of a practically complete solution in a production environment in the real world. (Also this might be something you have built, or it could be something you have bought).
Prototype completion is a critical step
Project plans often show a prototype stage followed closely by work starting on a production system, and this puts pressure on everyone to follow it.
The completion of a prototype is a critical decision point where the prototype is tested, feedback is considered, and you should ask the question again 'does it make business sense to continue with this?'
If the feedback from potential users is mixed or it throws up some critical questions about the feasibility of the product, then don't assume you can fix everything while creating the final product.
If there is still uncertainty, consider building a second prototype to validate any solutions and gain greater confidence that your approach is working. Prototypes are cheaper to build than production software, so this can save you a lot of time and money in the long term.
Be wary of the 'prototype' that ends up becoming the production system
Sometimes prototypes are created as throwaway products, or with that intention, but frequently they end up being converted into a production system.
This isn't a bad thing in itself - a lot of work can be invested in them, and there's no point unnecessarily replicating a lot of work. From the outside, prototypes can look very similar to the end product, and this will prompt senior managers to ask the reasonable question 'why can't we use this?'
The problem is that prototype code can have high levels of technical debt - shortcuts that were taken to create the prototype quickly. Technical debt makes the codebase difficult to understand, slows down future development work, and increases the overall cost of maintenance and bug-fixing for many years.
To manage this risk, you can either:
- give your engineering team team a clear brief that it might need to be adapted into a production system later, so that they can anticipate this and build the prototype in a more robust, long-term way.
- if there is a late decision to turn a prototype into production code, then give your team as much time as you can afford to address technical debt. This will save you time and money in the long term.
Keep control of spikes
This is a question that often comes up in the context of prototypes and proof-of-concepts. Someone will suggest that you do a 'spike' to do some research to answer a question.
A 'spike' is a term used in Agile software development to describe a short time-boxed piece of research work that might take a few hours. They are essential parts of the workflow, and used to answer technical questions, de-risk future tasks, or refine a time estimation.
In theory a spike is a very small 'proof-of-concept', and conceptually it is the right place for proof-of-concepts to be created. However, be wary of spikes as they can easily grow out of control. Make sure the objectives of a spike are clearly defined, and the time-boxes enforced.
If you need a second spike to finish the task, then do it, but keep them under control.
Make pilots as realistic as possible
Running a pilot of a new system is a complicated challenge. You have to keep existing systems running, choose who is going to use the piloted product, where it is going to be used and how you are going to assess it.
Pilots, by definition, are only done by a small subset of users. This reduces the risk to the organization, but creates the risk of hiding problems that only emerge once it is adopted by everyone.
For example, this might be a conflict with a security protocol or a localization issue with legacy data (number formatting, character encoding). Or it might be a hidden performance issue that emerges when the product needs to query millions of database records instead of thousands.
When assessing a new product during a pilot, always consider the things that you haven't tested as well as the things you have tested.
Not everything can be prototyped
Projects which depend on a large, clean dataset or a custom machine-learning model do not lend themselves to the prototype or proof-of-concept approach. They cannot be tested on a small scale and may not deliver any value until the whole dataset is available and all of the model training is done.
You may be able to prototype some AI solutions, for example ones which interface with third-party models, but unfortunately not always.
Sometimes you have to do everything before you know if you have something.
Do you agree with the definitions above? How have you managed these issues?
If you're navigating any of these challenges at the moment, let me know if I can help.
Contact us