Today is A-Level Results Day, and along with the results, we await the other two great journalistic certainties of this time of the year. The first is Sexy A-Levels, where it appears that only blonde girls receive results on this day (see what the Financial Times reveals about this phenomenon). Fortunately, or unfortunately (depending on your viewpoint), though my students may do very well, I don't expect to see them on the front of the newspapers any time soon. For one thing, half of them are male, and not one is blonde. And for another, if any photographer tried to get them to jump up and down they'd rightly tell the photographer to fuck off.
The other is the annual put-down to the kids who've just received the culmination of at least two years of study - that exams are getting easier. Old farts who should know better roll up to say that A-Levels aren't what they were in their day, usually in the same breath as calling for the cane to be returned to teachers' disciplinary repertoire (and perhaps simultaneously accusing same teachers of being lazy and incompetent). Occasionally, people actually bother to ask teachers what they think about the exams. Up to now, my hunch has been that the A-Level exams I sat relied on parroting off facts and figures, whereas current exams test students on their application of knowledge and critical thinking skills.
Well, I don't like putting about anecdotes without any data. So I set about getting hold of the papers I sat in 1997 and 1998 from UCLES (now part of OCR). They arrived last week, and it has been a pleasure reading through them all. Honestly, I do not know how I managed to get a B overall in the exams - they were heinous. The papers almost exclusively demanded recall of facts, numbers and definitions. The first question of the first paper asks candidates to "state the normal range of glucose concentration in the blood". Awful question, showing absolutely no understanding of anything about glucose levels other than an ability to remember an isolated number.
Many of the questions are similar. In 1997, Unit 1 (Biology Foundation) asked:
I'm still going through the exams, and my incoming AS and A2 classes are in for a surprise as I'm going to set them a few questions from these papers (some of the big old essay questions will make good homework assignments). However, I wanted to do a quick test that might get us thinking. I took my papers, and compared them with the papers sat by my A2 class over the course of their A-levels. So that was June 1997 Unit 1 and 2, March 1998 Unit 8 and June 1998 Unit 4. For my A2s that was June 2010 Unit 1 and 2, and June 2011 Unit 4 and 5 (they all resat Unit 4 in June, but that was okay because I resat Unit 4 as well!).
I looked at the command verbs in each question, and looked for the number of each one in each paper. I then separated the verbs into Bloom's Taxonomy low-order and high-order thinking skills. I considered "suggest" to be a high-order verb, since it required synthesis of previous knowledge and deep learning. And what do you know:
There is a significant difference between the percentage of questions testing low- and high-order understanding in the 1998 UCLES and 2011 Edexcel papers. I bunged the numbers through a t-test. The p-value for these data was 0.0005 - extremely statistically significant. This looks like pretty good support for my hunch - deeper questions are being asked, students need to think more about the science they study, and they are being given new and unfamiliar situations to analyse and evaluate.
So, if you are someone who measures learning and understanding of a topic by how many facts someone can parrot off Mastermind-style, then of course you'll think exams are getting easier - there are few questions that you will consider up to your exacting standards of memorisation and recall. But then again you probably think that knowing the precise years of reign of the Tudor monarchs is enough without understanding the societal upheaval surrounding their reigns. And your name is probably Michael Gove. And you're probably a twat.
But if you realise that numbers can be looked up, and that being able to apply what you have learnt to a new situation is not only a symptom of deeper learning, critical thinking (not to be confused with Critical Thinking) and a more holistic education, but indeed one of the most crucial skills to impart to any prospective scientist (nay, any prospective professional in any industry), then you will see the newer exams as a more rigorous way of assessing these skills. For me, if my students successfully demonstrate these skills, then they deserve all the A grades the exam boards want to throw at them.
Now I'm off to see if the exam boards have indeed decided to chuck some A grades in the direction of my kids.
The other is the annual put-down to the kids who've just received the culmination of at least two years of study - that exams are getting easier. Old farts who should know better roll up to say that A-Levels aren't what they were in their day, usually in the same breath as calling for the cane to be returned to teachers' disciplinary repertoire (and perhaps simultaneously accusing same teachers of being lazy and incompetent). Occasionally, people actually bother to ask teachers what they think about the exams. Up to now, my hunch has been that the A-Level exams I sat relied on parroting off facts and figures, whereas current exams test students on their application of knowledge and critical thinking skills.
Well, I don't like putting about anecdotes without any data. So I set about getting hold of the papers I sat in 1997 and 1998 from UCLES (now part of OCR). They arrived last week, and it has been a pleasure reading through them all. Honestly, I do not know how I managed to get a B overall in the exams - they were heinous. The papers almost exclusively demanded recall of facts, numbers and definitions. The first question of the first paper asks candidates to "state the normal range of glucose concentration in the blood". Awful question, showing absolutely no understanding of anything about glucose levels other than an ability to remember an isolated number.
Many of the questions are similar. In 1997, Unit 1 (Biology Foundation) asked:
(i) State the direction in which sodium ions will move across the membrane during depolarisation. [1]Whereas in 2011, Unit 5 (Energy, Exercise and Coordination) asked:
(ii) Explain how the impermeability of the axon membrane to sodium ions helps to maintain the resting potential. [2]
(c) Explain how the structure of the axon cell membrane is related to the conduction of nerve impulses. [3]The specifications have no doubt changed massively. I remember doing plant cell biology, and I remember photosynthesis. However, I don't really recall much about ecology or evolution, though a question on 1997 Unit 2 (Central Concepts) suggests otherwise. The UCLES specification is clearly very human-oriented, and it's quite nice that I get to do more with the other 99.99999% or so of species with Edexcel now.
I'm still going through the exams, and my incoming AS and A2 classes are in for a surprise as I'm going to set them a few questions from these papers (some of the big old essay questions will make good homework assignments). However, I wanted to do a quick test that might get us thinking. I took my papers, and compared them with the papers sat by my A2 class over the course of their A-levels. So that was June 1997 Unit 1 and 2, March 1998 Unit 8 and June 1998 Unit 4. For my A2s that was June 2010 Unit 1 and 2, and June 2011 Unit 4 and 5 (they all resat Unit 4 in June, but that was okay because I resat Unit 4 as well!).
I looked at the command verbs in each question, and looked for the number of each one in each paper. I then separated the verbs into Bloom's Taxonomy low-order and high-order thinking skills. I considered "suggest" to be a high-order verb, since it required synthesis of previous knowledge and deep learning. And what do you know:
There is a significant difference between the percentage of questions testing low- and high-order understanding in the 1998 UCLES and 2011 Edexcel papers. I bunged the numbers through a t-test. The p-value for these data was 0.0005 - extremely statistically significant. This looks like pretty good support for my hunch - deeper questions are being asked, students need to think more about the science they study, and they are being given new and unfamiliar situations to analyse and evaluate.
So, if you are someone who measures learning and understanding of a topic by how many facts someone can parrot off Mastermind-style, then of course you'll think exams are getting easier - there are few questions that you will consider up to your exacting standards of memorisation and recall. But then again you probably think that knowing the precise years of reign of the Tudor monarchs is enough without understanding the societal upheaval surrounding their reigns. And your name is probably Michael Gove. And you're probably a twat.
But if you realise that numbers can be looked up, and that being able to apply what you have learnt to a new situation is not only a symptom of deeper learning, critical thinking (not to be confused with Critical Thinking) and a more holistic education, but indeed one of the most crucial skills to impart to any prospective scientist (nay, any prospective professional in any industry), then you will see the newer exams as a more rigorous way of assessing these skills. For me, if my students successfully demonstrate these skills, then they deserve all the A grades the exam boards want to throw at them.
Now I'm off to see if the exam boards have indeed decided to chuck some A grades in the direction of my kids.
No comments:
Post a Comment