These questions are not intended to be conclusions. They are in some respects a call for more information and more enquiry.
1. What kinds of literacy did they test? (There are very many different ways of describing and assessing literacy. For example: did it weight comprehension as equal to eg spelling? Or indeed how were the different aspects weighted?)
2. Is the comparison between countries based on averages? Do they compare high scorers with high scorers with low? If not what do comparisons of averages really tell you? Assuming that the literacy testing has some validity, if the comparison is with averages for a whole country, this is not helpful or useful. Perhaps of slightly more use would be to know how different education systems succeed (or not) with their 'best' literacy performers and with their least successful performers. However, whether this kind of testing will reveal why they perform differently or what an education system might do about it, is another matter.
3. In comparing young with old does anyone think that this is a comparison between educational inputs? Because it isn't. It's a comparison between input and input-PLUS-life experience. Decline cannot be proven with this set of stats. So, testing older adults with younger adults has introduced another variable into the comparison and so is not valid in itself.
This then has repercussions in comparing countries for the obvious reason that older adults' life experiences have been very different from country to country. For example, if the older people had a high proportion of illiterates then clearly, a younger generation exposed for the first time in that country to literacy would make that country look as if it was advancing more quickly than a country where there were much fewer older illiterates. Other factors affecting this input into the adult cohort would be immigration of non-native speakers, a sample skewed towards those who had 'literate' employment ie where they could improve their literacy levels as they worked, and so on.
4. How comparable are international literacy tests? Are they really like for like? One immediate problem would be, say, with spelling - if that's what was tested. Languages which have very regular spelling systems with the same letters (graphemes) always matching up to the same phonemes present different problems from those that are not regular.
5. How statistically significant are the differences between the countries? This is the old league-table problem. Consider a football league. Manchester United are first, Manchester City are second. If the difference between them is 12 points, this tells us that Man U are, in effect, four wins better than Man City. If the difference between them is 1 point, then Man City could overtake Man U if Man U were to lose and Man City were to win. If the differences between countries in the league tables is small, it is insignificant and politicians should really say next to nothing about it. As I understand it, the differences are very small - they are 'statistically insignificant'. I'm prepared to stand corrected on this.
6. Does this data point to anything in terms of method re literacy teaching? Or is it just data to agitate for competition via the narrative of decline? This is crucial. I notice that some observers made a great deal of the fact that Japan appears to have done well and Japan does a lot of rote learning, large classes and home cramming. (Whether this suits high and low scorers alike, is not clear!) However, Finland did overall just as well. So, why aren't the papers rushing to Finland to find out what their secret is? Answer: it doesn't fit the reactionary pedagogy favoured by this government.
7. What use to educationists is global competition talk? Is education about recruitment for capitalism's requirements to compete? Is that the defining purpose of education now? Even so, are literacy and numeracy tests the sole basis on which national education should be compared?
8.Is there an agenda here about masking the disastrous performances of European capitalists by blaming education? It wasn't bad spelling that caused the banking crash.
