noun 527936 334b91

When is it important to question statistics?

Working in the market research industry means having a huge love for data! Being so close with data and reporting statistics to our clients often makes us question statistics when we see them reported. As researchers, we can’t help but ask questions such as:

· Who was interviewed and by whom?
· How were these questions asked?
· How many people were interviewed?
· Were the samples the same or are there differences that should be flagged?

The famous quote from Benjamin Disraeli “lies, damn lies and statistics” is often used to remind us that data can be misquoted or used out of context. The desire to tell a compelling story and grabbing readers’ attention through an impactful headline is natural, but can lead to readers being misled.

 Making an impact...but at what cost?

A recent headline from one newspaper caught our attention: “Obesity rife among care workers and NHS nurses.” The headline suggests that this particular group suffers from widespread obesity and it infers that this is a greater than expected problem. The article went on to explain that compared with the general population, the obesity rates for nurses are ‘deeply worrying’.

 Looking at the data behind the headline, it transpired that 25.1% of the nurses interviewed are classed as obese, whereas in the general population the obesity rate is recorded as 23.5%. A difference of 1.6%.

 One of the tools used by market research is significance testing, which enables us to look at two different samples of data and to establish whether changes in two data sets are statistically significant or whether differences are part of the expected variance (which can be attributed to factors such as the size of the sample). 

Significance testing on the data behind the headline shows that the difference between nurses and the general population is not significantly different, therefore this renders the headline somewhat misleading.

Here at Research by Design, we always take care to ensure that the data we report is checked for numerical and statistical accuracy and also logic checked to ensure that the sentiment of reporting remains as neutral as possible. Though we do love a good headline, our priority is ensuring that what we are reporting is accurate, underpinned by data and reported in a way that is not biased or misleading.

 By Lindsey Nadin

This entry was posted in Blog, Research, Market research, Methodology, tagged Surveys, Polls, Responses and posted on January 29, 2018

<< Back