Star Wars

For researchers, how to present the information you collect and analyse is a big question.

We believe that information should be presented as clearly and simply as possible. System 1 plays as large a role in processing research information as it does dealing with any other kind, so making findings easy and intuitive to grasp is a big part of making sure they lead to action.

That’s one reason we’ve started using “star ratings” to show the results of our concept tests. A five-star concept is a “next big thing” – a potential game-changer for its category that should be backed as heavily as possible. And a one-star concept is a “high loss risk” – an idea that simply doesn’t have the potential to succeed and should not be progressed.

"Calm down Chewy, you'd have been a 5-Star concept!"

“Calm down, Chewy, you’d have been a 5-Star concept!”

Obviously companies will aim at five-star concepts. But these are extremely rare – most good ideas will get four stars (which means they’re ready to go) or three (which means they might need a tweak or two). We put the rating system together to really discriminate between decent, great and amazing ideas like this – and to spot the polarizing concepts which split reaction but have a kernel of greatness.

The fact remains, though – one-star concepts are far more common than five-stars. In fact, around half the concepts we test end up with one solitary star.

And at first sight this seems a bit unfair. Brutal, even. If you’ve worked hard to innovate and come up with a concept, and are going to the expense of testing it, it’s harsh emotionally to get results that tell you, in very certain terms, that this idea is going nowhere.

Some of this is down to our choice of a star system. People have become very used to star ratings in everyday life – they are one of the engines of e-commerce – and the intuitive appeal of a star rating is a big reason we picked them. But Amazon’s star ratings work very differently. There, five stars is common – in fact, it’s the norm – and a one-star review signifies a real kicking (or a disgruntled competitor!).

So this is why a one-star rating seems harder to take than, say, a red traffic light or a stop button. But that’s exactly why we like it.

Think about the fate of concepts in the real world. The vast majority of product launches fail – the consensus is that around 1 in 10 launches are successful.

Faced with those odds, even a solid, three-star product will need good marketing – and a dose of luck – to succeed. A poor concept – or even a deeply average concept – has no chance. But deeply average concepts do get launched – it’s simple psychology that humans overrate things they’ve worked on and created, seeing them in the best possible light. That’s part of why people do research in the first place.

In which case, it’s our duty as researchers to say no when consumers tell us a concept is deeply average – and to say it as clearly and loudly as possible. So we’ve no regrets about handing out so many one-star reviews.


Go on, tell us how you feel!

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s