The Radio Advertising Bureau Blog

I was reminded recently of how easy it can be for received wisdom to influence how we do our research and how important it is to recognise received wisdom when confronted with it and to question it.

Let me explain…

I was working with an fmcg advertiser to set up a radio test for a major household brand. We wanted to measure radio’s impact on sales using a standard geographical test and control structure.

Now finding the right test and control areas can be challenging at the best of times (I won’t expand on this now but if there’s sufficient interest I might write a separate blog on it in the future).  Suffices to say that, after a lot of investigation, two areas in the North were by far the best candidates for the job:-

-          Manchester and Liverpool in the North West for the test

-          Leeds and Sheffield in Yorkshire for the control

So far so good…but then the question arose as to whether people travelling from one area to another could damage the integrity and robustness of the test. What if someone living in Leeds heard the radio ad when they were in the car travelling to Manchester for example? Surely quite a few people do this, don’t they?…or do they?

Suddenly our plan looked like being scuppered by a creeping doubt.  And whilst my own instinct was that in reality the numbers cross-commuting wouldn’t be very big and certainly not enough to impact on our test, I was going to need more than just a hunch to allay everyone’s concerns.

An hour or two of intensive desk research later and I had enough evidence to substantiate my hunch. For example, data from the ONS Annual Population Survey showed not only that the average commuting distance was significantly less than the distances between my test and control cities, but also that over 96% of residents in both Leeds and Sheffield worked in their immediate surrounding areas with no significant numbers of residents travelling as far afield as Manchester to work.

So our test methodology was robust after all – phew!

I’ve come to two conclusions from all of this. Firstly, received wisdom can occasionally be a useful catalyst for stress-testing a particular methodology (in this case for example it was right for someone to ask the question, if only to make us check the facts). Secondly, and I think most importantly, received wisdom should never be the only premise on which we make a final decision.  If in doubt, ask yourself “Where is this assumption coming from? And why should I believe it?”

As researchers we should always challenge accepted norms and beliefs to get the best out of our research.  And let’s face it, it also makes our jobs more interesting…after all who wants to just do what they are told?!

So back to my day job where I wait with baited breath for the results of our radio test…

No related posts.