In a handful of instances the lab has been called upon to review a pilot that's already running. The teams have wanted to make sure that when the pilots officially end we can say "Yes!" its worked or "No!" it has completely crashed and burned...
Very quickly we learned to involve Carole, Alun and Rich (Bromford's strategic research team) when it came to evidencing pilot success or failure - because they're boss at it. Play to your strengths and everything...
By working together so closely we ended up co-developing the new 'pilot pipeline' to ensure everything we roll out has clear aims, objectives and measures of success in advance. We're even able to test the measurements out in advance to make sure they work like they're intended to. All in the name of collecting good, robust data.
Kindly, Alun Morris has volunteered to explain the importance of good data in this weeks lab blog - Why Good Data Matters....
By Alun morris
A true story. Some names have been changed.
Prologue
I put down the phone, smiling, and wait for someone to ask me about the call I’ve just taken. No-one does, so after a moment I suck air in through my teeth and chuckle loudly.
“Well well well.”
Still nothing.
“That’s interesting. Didn’t see that coming. Well, I suppose it’s only to be expected of course.”
Moira looks up from her work.
“Sorry, were you talking to me?” she asks.
I pretend to have misunderstood the question.
“Oh, it was the just the Innovations Lab. They need someone to write a blog on dating, and they thought of me.”
“Why would anyone want you to write a blog about dating?”
I pause; I had briefly wondered about this during the course of the phone call, but had decided not to think too hard about it.
“Well”, I say, “I suppose because I’m, y’know… why wouldn’t they want me to write a blog about dating?”
“Because you’re single and haven’t been on a date since October”.
“Well that’s true, but I fail to see how it’s relevant.”
Moira fixes me with a look somewhere between pity and exasperation that makes me feel I might not be convincing her.
I pick up my notepad. “She specifically asked me to write a guest blog called – “ and at this point I turn the notepad round so she can see what I’d scrawled on it – “Why good dating matters”.
“Do you think there’s any chance you misheard, and you’re supposed to be writing a blog called Why Good Data Matters”?
“Oh god no, who’d want to read that?”
Why good data matters
Here’s the classic “Relative Trains” puzzle, first published in “Finnemore’s Gazette of New Mathematical Diversions ” in June 1922.
Train A leaves Stourbridge heading towards London at 09:00 on Monday morning, travelling at a consistent speed of 30 mph. Train B leaves Wolverhampton heading towards London at 09:45 on Monday morning travelling at a steady speed of 45mph. London is 10% closer to Stourbridge than Wolverhampton. Which arrives first?
If you answered, “Train A”, you’re wrong. Train A will never get there. There are no direct trains from Stourbridge to London, you have to change in Birmingham.
If you answered “Train B”, you’re wrong. Train B doesn’t exist.
If you answered “both trains will arrive at the same time” then you’re wrong on all levels.
The correct answer was Train C, which as Finnemore puts it:
“[T]ravels at an infinite speed, and therefore occupies all points in the universe simultaneously”.
So, what does this have to do with data? Well I was going to use Finnemore’s puzzle as a metaphor for what happens under the Bromford Deal, and show how the trains are like the members of the Deal household.
Finding out that Train A is headed towards London but will never get there is like finding out that an unemployed household member who we’ve told to join the work club in order to improve their employment prospects is currently out of work, but if we had their date of birth it would reveal that they’re only 6 weeks old.
Finding out that train B doesn’t exist is like finding out that the customer to whom we’re currently addressing all correspondence doesn’t exist, and only appears to exist because someone passed out on their keyboard and accidentally created a new entry in contact database.
Train C represents all those customers who are living in our homes without our knowledge – not as squatters, but just as missing household data (for example, one or more of train A’s parents are probably living in the household too). That would have been the metaphor, but I haven’t really got the hang of metaphors yet, so there it is. More of a simile I suppose.
Those of you who remember welfare reform can probably see how the above instances of incomplete, incorrect and missing data might cause problems.
For example when we were calculating the likely impact of The Bedroom Tax we needed to know the age, gender and relationship of all the trains in our properties. We didn’t have much of the data we needed, meaning that our initial estimates… actually our initial estimates were pretty good, even though I used a random number generator to come up with them. So that’s a bad example. However, due to a weird statistical fluke, the underoccupation data we got back from the first few local authorities to share their information with us showed that for those particular LA’s our estimates were a long way out, and we were underestimating the problem by about 33%. So at that point we had to draw up a worst case scenario in which our estimates for all other LA’s were also out by 33%. And let me tell you, when your figures for the potential rental income risk from welfare reform are based on the results of a random number generator plus 33%, they make for pretty scary reading.
So, what does all this mean for you? In your day to day job, how does it impact on the business if you don’t collect accurate data? I don’t know, that depends on who you are what your job is. I’m pretty sure that if we were having this conversation one-to-one I’d be able to come up with some pretty good examples of the importance of good data entry that apply specifically to your role.
Epilogue
I stand waiting for the door to the Innovations offices to open. My pass doesn’t work on the 5th Floor at Exchange; I’ve heard rumours that no-one’s pass works here; that there always has to be at least one colleague inside to let people in. Personally I think this is rubbish, and subscribe to the theory that whichever member of the Innovations Lab arrives first each morning has to scale the outside of the building and force a window open.
After a minute or so a hatch slides back, revealing a letterbox sized hole in the door at around eye height. It’s filled by a face I don’t recognise.
“What do you want?” asks the face.
From inside the office I hear what sounds like [REDACTED], accompanied by a peel of maniacal laughter. I’m pretty sure I also hear someone say [REDACTED].
“What was that?”
“Nothing. Go away.”
“I… I have a blog? I was going to email it but I was told you’ve gone ‘off grid’ ?”
“Give it here!”
I reach up to hand the sheet of A4 through the gap. As my hand reaches the letterbox the face sticks its own hand out, grabbing my wrist.
“Give it here, give it here.” My arm is pulled through along with the paper.
“Go away! Go away! Shoo.”.
It lets go of me, and I retract my arm hurriedly. The hatch is quickly closed. I breathe a sigh of relief and head back down to relative the safety of the basement, pretending not to notice the bitemarks on my fingers.