Archive

Posts Tagged ‘statistics’

9% of 11-year old boys can’t read! So what?

December 17, 2010 1 comment

You can tell that news is sparse on the ground – unlike the snow. The newspapers have already done a blanket coverage on the snow and how the UK again skidded to a halt, so they can’t do that one again. Instead, the press is trumpeting on about how terrible it is that 9% of boys can’t read properly when they leave primary school.

Apparently BBC Radio 4 asked the Department of Education for the number of children who failed to reach level 2 reading age, the standard expected for seven-year olds, and found out that around 18000 boys aged 11 had a reading age of seven or less. This was in contrast to other statistics that have shown a steady rise in standards – with children achieving the expected minimum level 4 having gone up from 49% of children to 81% in the last 15 years.

Seemingly, even worse, in some areas – for example Nottingham – 15% of boys failed to get past the level 2 reading level.

The problem with all this isn’t the statistic but the lack of context. When reporting information (whether for competitive intelligence, general business or marketing research, or whatever) it is essential to include the context. A figure on its own is meaningless. In fact, those figures for Nottingham could be brilliant – if five years ago, 30% of boys had failed to get past the level 2 reading level. It would mean that the numbers of children failing had halved. Conversely if the number had gone up from 5% then this would be a massive indictment against the teaching profession who were failing to motivate and educate their pupils.

In fact, the original story from the BBC does give some context.

In 1995, the proportion of 11-year-olds getting Level 2 or below in English – the standard expected of a seven-year-old – was 7%. In 2010, it had fallen only to 5%.

The figures show the problem is worse for boys. Overall in England, 9% of them – about 18,000 – achieved a maximum of level 2 in reading.

This shows that in fact, performance has improved overall, with underachievers falling from 7% in 1995 to 5% of all children now. However without a longer-term trend it is impossible to put much value into the statistics – especially as other research reported by the BBC looking at seven year olds showed that children with special educational needs, and from deprived homes (meaning that they were entitled to free school meals), were the worst performers. A third (33.6%) of seven years olds on free school meals failed to reach the requisite level 2 in writing and 29.3% failed to reach this level for reading. In contrast, the children who did not receive free school meals did much better – only 12.1% failed to reach the required level for reading, and 15.5% for writing.

I’m actually surprised that some mathematically-challenged journalist hasn’t picked up on these figures and claimed that providing free school meals results in children under-performing at school. In reality, all the figures show is that such children have barriers to learning that schools have to try to overcome. This may be because the children are under-stimulated at home (and so start at a lower level than their peers), come from homes where English is not spoken by the parents or are of lower intelligence overall. (In fact, intelligence tends to fall on a normal curve. If 10% of children outperform – and have a reading age 3 years ahead of the norm, you can expect that a further 10% will have a reading age 3 years less than the norm).

The lesson from such statistics and reporting is simple: before publishing statistics in the press or in a business report provide a context.

This context can be temporal – looking at how figures change over time. In the case of the school statistics, they appear to have improved over the years for both the low and average achievers – a testament to the teaching profession. Context can also be seen when comparisons are made – as in the comparison between children on free school meals versus those not entitled to this benefit.

Strategic decisions based on figures should only be made when context is included. Without it, the figures mean nothing, and should be left to melt away, like snow.

Lies, Damned Lies, Statistics & Facebook

June 10, 2010 Leave a comment

I’ve been impressed with the numbers of people using social networking sites – and the importance of social networking for marketing has become significant over the last few years.

Facebook claims 400 million users (i.e. nearly 6% of the global population that is approaching 7 billion people). I’ve always thought that this figure must include duplicate accounts – as I don’t believe that most people in China, India, Africa and many other areas of the world have Facebook accounts (or even computers – although the numbers are growing). The World Bank stated that there were just under 300m Internet users in China and 52m in India in 2008. (There’s a great graph of this at Google’s Public Data tool – that shows that in 2008 there were around 1.5bn web-users).

Even taking account the exponential growth – let’s assume that web users globally are now over 2 billion  people – Facebook’s figures imply that 1 in 5 users have a Facebook account.

I know of many people who don’t have an account and some who refuse to get one. In my age group (over 40), I’d guess that the majority don’t. So where this 400m figure came from and what it includes is a key question.

It now seems that Facebook has been boosting it’s membership figures. I just read this article from one of my favorite sites (www.pandia.com). Apparently Facebook has been telling advertisers that it has 1.6m users in Oslo. The trouble is that the greater Oslo metropolitan area only has 900,000 people. Facebook apparently counts members by IP address – and I guess that it is feasible that this could include users who access the site via Oslo based web-servers. However not if you consider the next statistic given. The Facebook advertiser tool says that there are 850,000 Facebook users between the ages of 20-29 in Norway – which is 235,000 more than the total numbers (613,000) in that age group.

This over-inflation isn’t just a Norwegian issue. According to CheckFacebook.com (a site that tracks data from the Facebook advertising tool giving Facebook membership numbers), almost 63% of online users in the UK now have a Facebook account. That’s 27m out of a total UK population of 62m. In some countries it’s even higher. Apparently all (100%) Nicaraguan, Qatari and Bangladeshi web users also have a Facebook account, as do 99% of Indonesians, 98% of Filipinos, 97% of Venezuelans, and 85% of Turks.

It’s possible that these statistics are true. However, if so, I’m sure that they also include occasional and infrequent users as well as dormant and duplicated accounts.

One of the most important types of competitive intelligence analysis is to not take everything at face value. When presented with figures, it’s important to sense check them – wherever possible by using other sources (e.g. official population statistics). Only then should such data be used in decision making. You should also ask whether there is an incentive to exaggerate or under-estimate statistics. If there is such an incentive, it is likely that this will be done, at least in the published data. Decisions made using such erroneous or manipulated figures will probably be poor decisions and fail to achieve the expected results. In the case of Facebook, the incentive in exaggerating membership figures is that they can then boost their attractiveness to advertisers, and consequently their advertising revenues.

Google – public data explorer

March 10, 2010 Leave a comment

I’ve just been pointed to a new Googlelabs initiative – the Google Public Data Explorer. This promises to be a useful tool for finding public data in one place. (It’s always worth keeping an eye on GoogleLabs as they often bring out new ideas and products. These are kept together until ready to launch – and can be found from http://www.googlelabs.com.).

The data is not new – although i think some of the presentation is. I don’t recall being able to manipulate the figures from Eurostat so easily (but then that may be because I’ve not had to use Eurostat for a while). Eurostat – the European Union’s statistic service – is large and complex (or was). With Google a couple of key Eurostat databases (unemployment statistics, minimum wage, consumer price index) now become easily manipulable. Other databases include OECD, World Bank and a number of US databases.
Hopefully many more databases will be added – and eventually the service may become a one-stop-shop for global statistics, replacing the need to visit various local country statistics services (e.g. the UK’s Office for National Statistics).
Even though there are currently only a handful of databases available many of the most important types of data looking at GDP, population trends, health, etc. are available – plus interesting, but probably less critical examples, such as Internet users per 100 of the population. In this example, I compared the UK and US with two of the emerging power-houses – China and India for Internet usage. I found it interesting that the UK had more users per 100 than the US but not surprising that China and India were so low, despite the total web user numbers in China being higher than those for the US and growing rapidly. It would have been possible to add any of the countries on the left to the chart.