Science of science standards

Abandon your moderate smugness about your academic skills and try to answer any of these old maths and chemistry exam questions:
Old mathematics paper
5 decades Chemistry papers

Well, I think I could get a few right, especially with a bit of a run-up to revise and some practice. But I would still have failed dismally. I couldn’t even get near to passing an exam that I’ve actually got the paper qualifications for – from the days when it was “harder”.

The Royal Society of Chemistry has petitioned the government about a fall in school science standards.

Armed with the first hard evidence of a catastrophic slippage in school science examinations standards, the Royal Society of Chemistry (RSC) has launched a Downing Street e-petition calling for urgent intervention to halt the slide.
And tomorrow morning a devastating RSC report demolishing the myth of record-breaking science education performance will land on the desks of all Members of Parliament.
The RSC report, also being supplied to industrialists and educationalists, raises major concerns over the disappearance from schools science examinations of vital problem-solving, critical thinking and mathematical manipulation…..
… Even bright students with enthusiastic teachers are being compelled to “learn to the test”, answering undemanding questions to satisfy the needs of league tables and national targets. The RSC has powerful evidence of the decline in standards, adding to the revelation that students are able to receive a “good pass” with a mark of 20%.

I’m not saying that they are wrong. They make very valid points in the report .

However, the public face of their “first hard evidence” seems more like a publicity stunt than science. In its pop-science media version, it certainly looks like the application of poor social science The RSC “hard evidence” is their exam test (5 decades Chemistry papers)

The RSC ran a competition based on exam questions culled from 50 years of chemistry O Level/GCSE exams. They found that the students (identified as “the most promising scientists” by their schools) who sat the tests “averaged just 25% of available marks.”

“Although the winner of the RSC competition got 94%, the fact that many highly intelligent youngsters were unfamiliar with solving these types of questions, obtaining on average 35% correct from recent papers from the 2000s and just 15% from the 1960s, points to a systemic failure and misplaced priorities in the educational system, rather than shortcomings in individual teachers or students. (From the RSC website)

Hmm. The content of the Chemistry curriculum has surely changed over 50 years. The percentage of correct answers fell as the questions went further back in time. This suggests at least one alternative explanation – the less familiar they were with the material, the worse the students did.

(Which is not surprising, given that teachers have had to “teach to the test” whenever they have prepared students for public exams. Students would be quite annoyed if they found themselves being prepared for exams from 50 years ago, because – however good their knowledge might be, it still wouldn’t allow them to do well on a 2008 paper.)

Surely, control samples should have been used. For instance, groups of students who weren’t already identified as the science stars; groups of students given only recent questions; groups of adults asked to sit the same tests and/ or recent exams. It would also have been more convincing if they hadn’t marked and set the tests themselves.

Not worth the bother? Obviously, because this would imply it was a real investigation, not a way to get their petition some public interest. (Such as I am showing here…. D’oh. Chemists:1 Me:0)

The RSC aren’t stupid. Their full report shows that they are aware that changes in the subject make it impossible to draw easy conclusions. They point out that they aren’t blaming teachers or students.

However, unless they are almost too unworldly to shop for their own pipettes, they would know that this will get publicity only because the whole report will be presented in terms of “exams are getting easier”.

This is an ongoing debate. It’s sometimes seen as a bit churlish to suggest that exams are easier, presenting some implied insult to the efforts of current students.

However, by very simple mathematics, if education policies are qualification-driven, demanding very high target levels of achievement, exam qualifications must get easier.

It is not possible to demand that 100% of 16 year-olds pass exams unless those exams are so easy that anyone can pass them (By definition. The clue’s in the target figure, for the mathematically-challenged.)

What teacher in their right mind would enter their dimmest or least interested students for a tough chemistry exam? Why would they encourage any students whom they think won’t get a good exam mark to develop an interest in science, when their school will go tumbling down the league tables if only 5% manage a pass?

The RSC is actually saying this. Their real argument is about the system not the easiness of exams.

But who’s going to notice that while we are all feeding our own prejudices about the youth of today getting dumber? (Try one of those exam papers if you ever start to believe that.)