A guest blog from our Insight Team
Prologue:
“So, everyone is clear on section one, yes?”
Bill from Finance nervously raises his hand at the back of the room.
Moira glowers at him. “Yes?!”
“Erm, for question 6, according to instructions I’m supposed to say I hate my boss?”
Moira: “And?”
“Well, it’s just she’s sitting right next to me”.
Bill is right; in fact they’re sharing a chair.
Moira looks at Griz. Griz squeezes his way through the rows of colleagues crammed into the room, and takes Bill’s tablet from him. He taps the screen twice and hands it back.
“There you go”, says Griz cheerfully. “Now I’m the one who said they hated your boss”.
Moira: “Right, section 2. Health and Wellbeing. Question 1: ‘How often do you experience night-terrors whilst in the office’. All clear?
Sydney raises her hand.
Moira does her best to sound patient. “Yes?”
“Are you sure these are confidential? Only I’m meant to answer ‘Constantly’, and I’m worried that makes me sound a bit, well, mad?”
Moira looks at me.
I cough. “All data is absolutely confidential; answers can’t be tracked back to the individual who submitted them. Responses may be used in aggregate form only. Data is sent directly to the Wonderful Workplace Company; the information we receive back is completely anonymised. The company receives an overall Wonderful Workplace score and correlation data between the different questions, factors and said overall score. For full data protection statement please see the Wonderful Workplace website.”
Sydney doesn’t look completely reassured, but Griz is now standing behind her to make sure she answers correctly.
Moira: All done? Next question. ‘How much do you agree with this statement: Compared to someone who has poor health but is generally optimistic, I would describe the movement in my own health over the last 12 months as more positive than negative’.
- - - - -
Getting 127 colleagues together in a room that comfortably seats no more than 12 and dictating answers to them for a colleague engagement survey that is supposed to be completely secret may seem questionable; dishonest even. But what’s the alternative: We trust everyone to answer honestly, and hope that the aggregate of those answers will form a set of results that lead the people doing analysis to the conclusions that we want them to form?
Don’t be ridiculous.
Imagine you go to the dentist. Wait, imagine you don’t have a dentist. Imagine you have a toothache. You phone a dentist, and 90 minutes later you have seen the dentist, received professional treatment through the NHS, and your gob is starting to feel better. You are delighted with the service you’ve received. Stop imagining: this is what actually happened to me. I don’t know why I didn’t just say that to start with. Anyway, the next day I get a text message.
Of course the answer is 10. I am a rational reasonable person.
Then I get another text.
Seems pretty similar to the previous question – I mean what kind of sadist hated the overall experience but would still recommend their love ones go through it? - but again I give full marks.
And another.
I had a toothache, which was dealt with quickly and efficiently. Under these circumstances, the welcome would have needed to be extraordinary for me to remember it, let alone consider it important. Given how good my overall experience was, the receptionist would have had to hit me in the face with a frying pan for it to spoil my overall impression of the service. This did not happen.
Similarly, if I’d had to wait several weeks for an appointment, and then had the wrong tooth extracted by a wandering idiot with a pair of plastic pliers, the only way the welcome could possibly have been good enough to stay in my mind is if Scarlett Johansson had been there to thank me enthusiastically for choosing her dental surgery. Sadly, this didn’t happen either.
Obviously, I answer 5. Firstly the service was great, so not giving top marks to an aspect of it I don’t care about seems petty. Secondly, I don’t want to give the NHS the impression that “quality of welcome” is an area that can be improved and is, therefore, something to expend resources on.
Final text “How happy were you with the way your treatment options were explained to you?”
Of course my choices weren’t explained to me. I’m the kind of idiot who doesn’t go to the dentist for 6 years then is surprised when he gets toothache. Why would I want any choice in my course of treatment, when there’s a professional with years of medical training who can make that choice for me. In my case this question is clearly not applicable. However, this isn’t an option. Again, it would be petty not to give it top marks. Again, I would not want the NHS to think “Oh, the one area this person would like us to improve upon is patient choice” (because patient choice is a silly phrase first introduced by people who were tasked with improving perceptions of the NHS rather than improving the NHS itself).
So I’ve answered really positively about everything. They will know that everything was great with my visit. They will not think they need to invest money in things that don’t matter to me. Unless of course, everyone else behaves just like me when answering these surveys
My first two answers, which were to “overall” questions were honest; I would definitely recommend to a friend, I was happy with the service. The other two questions, I effectively lied about. So I have just told the NHS that I was really happy with the specific aspects of the service they chose to ask me about – welcome and patient choice –and that I would also recommend my overall experience to a friend. They did not ask about waiting times, or the actual treatment I received, the things that were important to me. So assuming everyone else uses the same logic in their answers as I do, they will see that all people who would recommend their dentist to a friend are also happy with the welcome they received and patient choice. Or to put it another way, all the people who say they received fab welcomes and great patient choice, would recommend the NHS to a friend.
Now let’s assume that some people had to wait ages, and were treated by a wandering idiot drafted in to due to a shortage of qualified medical professionals. The truth is, after their horrible experience, they probably won’t remember much about their welcome, but they’re unlikely to feel inclined to score it as 5/5. Similarly, even if the wandering idiot had stopped to explain their treatment choices, unless one of those choices was “Would you like to step into this time machine and pop back to a point before you made this appointment”, top marks again seem unlikely. So the NHS sees that the group of people would not recommend to a friend are the same group of people who also tend to give lower scores for welcomes and patient choice. So my “helpful” survey completion strategy has probably added an extra layer of correlation/causation confusion to be navigated by whoever analyses these responses for the NHS.
Now of course not everyone does respond to surveys the same way, but hardly anyone answers every question objectively on its individual merits. Other common examples include most of us will experienced at some point include “I’m having a bad day and taking it out on this survey”, “The question is asking about my ‘usual’ or ‘everyday’ experience, but I’m basing it on the best/worst experience I ever had because that’s just what human beings do” and “If I tell the truth on this survey, will it make me look like an awful human being?”
All questions in a survey should really have an “I don’t care” option; that way you can honestly tell the questioner that the thing they’re asking about just isn’t important to you, and could they please focus on your other answers. Failing that, if you have something you really want to feedback, you don’t have to wait for someone to send you a survey. Just go and speak to someone. It’s usually a more effective way of communicating your feelings than aggregating them in with everyone else’s multiple-choice answers. I think I’m going to send my new dentist a thankyou card.
- - - - -
Epilogue
Moira: So, final question. How much do you agree with the following statement: ‘If a dishonest person with a good heart wanted a job here, I would write them a letter of recommendation.’ Now, this is one of Drofmorb’s custom questions, which means they think it’s really important. So you must all leave it blank, ok?”
I’m impressed with the way Moira has handled the room. When we first tried this a year ago only 15 of us took part, and we just got everyone to say they were unhappy with their pay but happy with most other things. We’d learnt the error of our ways, and been much more ambitious this year. 95% of Middling Services have turned up; this gives us a great chance to really influence policy, but it also takes a lot of effort to herd that many cats, especially considering that 364 days of the year the rest of the business regards us basement dwellers with a mixture of contempt and fear.
Griz: Can they go now?
Moira: Yes.
Griz unlocks the door. “Closing time! Everybody out! You don’t have to go home but you can’t stay here. Any more for any more! Come in number 7 your time is up…”
He breaks off, as he’s in danger of getting caught in the crush. No-one needs much encouragement to leave. The room is warm and stuffy, and during the preceding 90 minutes voices have been raised, board erasers have been thrown, and no-one has been allowed to leave to use the loo.
Sydney is looking thoughtful. “That woman just leaving” she says, pointing at the crowd. “Isn’t that our Head of Human Beings”?
“Yeah. Almost all of Human Resources were here” explains Moira.
Sydney is frowning. “And we were all filling in this survey in a very specific way, in order to convince HR that the best way to improve colleague engagement scores is to give us all a payrise?”
Moira nods. Griz has made his way back to us, and he’s looking somewhat thoughtful too.
“So,” he begins slowly. “If we all want a payrise, and HR know we do… I mean if they know we’re filling in the survey… I mean if they were filling it in with us…”
Sydney interrupts. “Why the bloody hell was the point of all that?”
Moira sighs. “Of course they know we want more money. Everyone wants more money. They want more money. But you can’t have a payrise just because you want one. We have to prove that it correlates with colleague engagement. That it’s the only thing that correlates with colleague engagement.”
“But that’s the stupidest thing I’ve ever heard!” protests Sydney.
I’m a little bit hurt. It took me a long time to work out the exact answers everyone needed to give to ensure that whoever was analysing our results saw a big red arrow pointing from “More money” to “Wonderful Workplace Award!”
“Hey. Trust me. This is the best chance we have of getting a payrise this year.” I try to sound confident as I say this, but I am starting to have doubts.