Find It Out – Research Secrets and More

You can tell that news is sparse on the ground – unlike the snow. The newspapers have already done a blanket coverage on the snow and how the UK again skidded to a halt, so they can’t do that one again. Instead, the press is trumpeting on about how terrible it is that 9% of boys can’t read properly when they leave primary school.

Apparently BBC Radio 4 asked the Department of Education for the number of children who failed to reach level 2 reading age, the standard expected for seven-year olds, and found out that around 18000 boys aged 11 had a reading age of seven or less. This was in contrast to other statistics that have shown a steady rise in standards – with children achieving the expected minimum level 4 having gone up from 49% of children to 81% in the last 15 years.

Seemingly, even worse, in some areas – for example Nottingham – 15% of boys failed to get past the level 2 reading level.

The problem with all this isn’t the statistic but the lack of context. When reporting information (whether for competitive intelligence, general business or marketing research, or whatever) it is essential to include the context. A figure on its own is meaningless. In fact, those figures for Nottingham could be brilliant – if five years ago, 30% of boys had failed to get past the level 2 reading level. It would mean that the numbers of children failing had halved. Conversely if the number had gone up from 5% then this would be a massive indictment against the teaching profession who were failing to motivate and educate their pupils.

In fact, the original story from the BBC does give some context.

In 1995, the proportion of 11-year-olds getting Level 2 or below in English – the standard expected of a seven-year-old – was 7%. In 2010, it had fallen only to 5%.

The figures show the problem is worse for boys. Overall in England, 9% of them – about 18,000 – achieved a maximum of level 2 in reading.

This shows that in fact, performance has improved overall, with underachievers falling from 7% in 1995 to 5% of all children now. However without a longer-term trend it is impossible to put much value into the statistics – especially as other research reported by the BBC looking at seven year olds showed that children with special educational needs, and from deprived homes (meaning that they were entitled to free school meals), were the worst performers. A third (33.6%) of seven years olds on free school meals failed to reach the requisite level 2 in writing and 29.3% failed to reach this level for reading. In contrast, the children who did not receive free school meals did much better – only 12.1% failed to reach the required level for reading, and 15.5% for writing.

I’m actually surprised that some mathematically-challenged journalist hasn’t picked up on these figures and claimed that providing free school meals results in children under-performing at school. In reality, all the figures show is that such children have barriers to learning that schools have to try to overcome. This may be because the children are under-stimulated at home (and so start at a lower level than their peers), come from homes where English is not spoken by the parents or are of lower intelligence overall. (In fact, intelligence tends to fall on a normal curve. If 10% of children outperform – and have a reading age 3 years ahead of the norm, you can expect that a further 10% will have a reading age 3 years less than the norm).

The lesson from such statistics and reporting is simple: before publishing statistics in the press or in a business report provide a context.

This context can be temporal – looking at how figures change over time. In the case of the school statistics, they appear to have improved over the years for both the low and average achievers – a testament to the teaching profession. Context can also be seen when comparisons are made – as in the comparison between children on free school meals versus those not entitled to this benefit.

Strategic decisions based on figures should only be made when context is included. Without it, the figures mean nothing, and should be left to melt away, like snow.