One of the most popular posts that ever featured on Bromford Lab is nearly 10 years old, “What’s The Difference Between A Test and a Pilot”, written and illustrated by Tom Hartland
Tests: A quick, low-cost, and low-risk way to learn about a problem and potential solutions. Tests are often used to gauge how a target population will react to a delivery.
Pilots: A static way to evaluate a proposed solution, based on ideas generated during prototyping and learning from testing. Pilots are more expensive, resource intensive, and time-consuming than tests, and are used to demonstrate a minimum viable product. Pilots are usually fully formed ideas that can't be changed once they're started, as doing so risks the integrity of the evaluation.
The innovation approach we were running back then emphasized evidence-based experimentation instead of lengthy pilots and reports. The goal was to get things wrong quickly and learn from mistakes.
Fast forward to 2024 and we are referring to our four place pilots internally as ‘pilots’ although I’d argue they are nothing of the sort.
We are trying different approaches in four different places with different strengths and different challenges. The four places are roughly representative of all the homes across Bromford - if things can work in those places then they should be spreadable.
They are a mixture of a test and prototype - a prototest if you will.
The prototests - six month phases of activity in which we get teams to interact with each other and with customers - are designed to identify emergent practice.
Emergent practice in innovation is a bottom-up, adaptive approach that focuses on identifying and nurturing innovative practices that organically arise within an organisation or team. It contrasts with traditional, top-down approaches that seek to impose predetermined solutions or "best practices."
So when people say “What are the measures of success?” - we don’t know. Because innovative practices are only just beginning to emerge as part of the test.
As Simon Penny wrote: “Testing is about learning quickly in a controlled environment, shielded from the main business; enabling a greater understanding of the problem at hand and the component parts of a possible solution we have in mind to fix it. Tests are all about proof of concept; they are a relatively quick, low cost, low resource, low risk form of learning.”
That definition is good. But our first forays into place are not in a controlled environment.
They are not fully shielded from the main business.
Therefore even if they failed that does mean our hypothesis is incorrect.
The first six months are really just observational work to see what those practices are, which might be measured in the future, if they generate successful outcomes.
If our place based work is to be successful, we are going to have to learn to live with an awful lot of ambiguity.
