Hallowe’en night and a haunted viewing room

Who would be a qualitative researcher, eh? This post is not at all influenced by my having something of a difference of opinion last night with some clients over a group discussion …but ever the Polyanna, I’m turning this into something positive, by lecturing everyone on how to avoid the pitfalls of watching discussion groups.

So it was Hallowe’en night, I’d missed my son’s party to leave for the discussion groups, but left him my spooky compilation CD in the true manner of a 40-something Dad (featuring way too much of The Fall, little bits of Bauhaus, The Specials and yes The Monster Mash). All started – and almost until the end, continued – very well. I’d negotiated my way through the white water ride of a potentially tricky group, doing some creative NPD with some 18-24 year old lads. I’d got through a tight guide nicely; and more importantly, the lads had really engaged with the discussion (not always the case in 18-24 male groups) and told us as much about themselves as we dared hope for. I was just preparing myself to accept the plaudits modestly from the viewing room.

But when you step out of the hubbub of the discussion room and venture back there, you never quite know what you’re going to get. Especially with new clients. As soon as I opened the door I could sense some kind of spell in the room: deathly quiet; furrowed brows; no eye contact. It was a qual research Hallowe’en nightmare. All that was missing was the bell tolling, a howling wind and a wolf silhouetted by the moon. What had gone wrong back there?

It wouldn’t be appropriate to do a post-mortem here. But assuming I’m not completely deluded about how to moderate a discussion group after 13 years (having been, inter alia, head of marketing qual at Ipsos MORI) how could this happen?

Of course, there is no ‘perfect’ in qual and you always look to improve, but that aside, I’ve long been fascinated with the disconnect between a group that a qual researcher would recognise as a success and one that clients recognise as a success. I also supervise groups, sitting with clients, so have experience on both sides of the mirror here. Watching and commenting on groups is not as easy as it may look, especially if you’re not so comfortable with qual.

So I’ve drawn up a few basic rules to help discussion group viewers get their heads around what they are seeing and hearing – aimed at those who do not hang out in viewing facilities as much as I do (lucky you!):

• Don’t worry if it doesn’t make immediate sense – wait for the analysis. To follow a cooking analogy, you are watching the chef gathering some of the ingredients. You are not watching him/her prepare the dish and you are not going to be able to taste it yet.
• For the same reason, value your first insights but treat them with caution too. An initial insight may be valid, it may not. Again, you can only tell by analysing properly in the round.
• Don’t take what group participants say literally. What they tell us and show us is evidence we use to work them out, it is not gospel.
• What people say spontaneously, unprompted, tends to tell us much more about them than what they say when put on the spot (with some exceptions). Moderators try to minimise their interventions – so the discussions are as natural as they can be, given that we’re in a viewing facility with a bunch of strangers here.
• The moderator hears, sees and picks up a whole lot more, by being in the room and looking people in the eye. Ask the moderator what he/she felt about the vibe, body language etc. But the moderator does not have the freedom viewers have to switch away from the group for a few minutes to take a thought and ponder it. So those viewing have more of a chance to develop thoughts (though they risk missing something important when they do). Conversation between moderator and viewers after the group should be based on this understanding of these relative strengths and weaknesses.
• Follow-up probing is important in groups but it has to be used selectively. It can bore the participants not being probed; it is hungry on time. But worse, it can make people over-rationalise and self-justify, rather than be themselves (not just the person being probed but the others who think, “It’s me next”). Qual research at its best comes at things indirectly; asking the moderator to do more direct probing is not always the way to get more understanding.
• Remember, this is the bit of the research process where we build around participants, not the other way around.
• Different audiences need different moderating styles. A naturally chatty group has one set of challenges – e.g. keeping them on-topic and cutting people short without disempowering them – a quiet group has another. Typically the moderator will seem to be “working harder” with the latter but he/she is really working equally hard with both.
• Good recruitment is important, but you can’t and shouldn’t seek to over-engineer the minutiae of each person you invite. They are people and if they don’t fit neatly into a box, that’s normal. Of course if you just have totally the wrong people, that’s different and is a big issue.
• Qualitative research can be very counter-intuitive. It is not just a research technique – it is a whole approach to gathering and making sense of information that will jar with the way many approach the world and especially their working life. It’s more of a leap for those without an arts or humanities background, because critical analysis skills are the most relevant ones for interpreting the meaning of discourses. It is instinctive to seek to measure, to seek order and to seek direct answers; it is also drummed into most of us in school. But qual explores, highlights and makes connections; it embraces mess and lets it breathe and live so we can study it. Order is only really imposed at the final stage of sense-making. This can feel very uncomfortable for those used to being in control right the way through. It also requires trust in the researcher’s ability to do the sense-making.
• Moderators often “play dumb” in groups as a strategy for getting people to explain themselves. It does not mean they are dumb. Though they can be.

I’m not the first qual researcher to be frustrated by clients having a different take on a group and I won’t be the last. Aren’t we just the misunderstood geniuses? But if one future group viewer takes note of one of these tips, I’ll be at least as happy as Bela Lugosi (and hopefully less dead than him). Of course, sometimes, it is our fault …