Saying And Doing

We’ve talked a lot about how people are poor witnesses to their own behaviour, and every now and then you find a very neat demonstration of it. This week, we’ve bumped into TWO great examples.

In the first study, researchers asked how much British people drank. They then compared the self-reported results to alcohol sales figures. The outcome? Between 40% and 60% of sales volume was going unreported! In other words, on average our self-declared drinking is around half what we actually consume.

Too tempting?

In the second example – from the early 00s – people were asked what their dietary habits were and also what they ate yesterday. When it came to vegetarians, there were startling discrepancies. 6% of people surveyed identified as vegetarians, but of those, up to two-thirds had eaten red meat, poultry or fish the day before. Self-identifying as vegetarian is often not a strict description of dietary behaviour, these results suggest. Instead it’s more like a social marker: I identify with vegetarian ideals or, more practically, I would like to be treated as vegetarian.

What does it all mean? I asked Peter Harrison, BrainJuicer’s Creative Director: “Our system 1 provides a self-defence mechanism to help us maintain a positive self-image, it’ll rewrite history by either glossing over bad things we’ve done or emphasise the significance of the other factors – “I wouldn’t normally eat meat, but the bacon smelt so good and I hadn’t eaten for like 2 hours!”  It would be quite depressing if we didn’t have this ability and were acutely aware of all our failures  – so you can see why it’s important for us to lie to ourselves – at least, that’s what I tell myself!”

Door No.15: Survey Optimisation

The 15th in our Advent Calendar of Experiments – apologies for the delay in posting today’s.

The Experiment: Anyone involved with running a website of any scale is probably familiar with split-testing – automatically splitting the flow of traffic to a site in order to test different versions of it. Split testing (or A/B testing) is used to optimise the conversion rates and internal traffic of sites – all you need is that steady flow of eyeballs and you can run as many experiments as you like.

Split testing is also a great example of how useful DIY research tools can be in improving large-scale research, since you can also use it very easily for survey optimisation. When we first got our Optimizely subscription, we decided to try a little experiment in question wording.

We took a series of pop music stars – the best-sellers of 2011, led by Adele but including the likes of Coldplay, Lady Gaga and Amy Winehouse – and we asked people to simply pick “which of these acts” they liked. Or that was what half the sample saw. We also had the test set up to show half our sample the exact same question and list, framed as “artists” instead of “acts”. So what happened?