Lies, Damned Lies and Mann-Whitney

Last week N Burnett emailed me asking what an exam question on Mann-Whitney might look like. The simple answer is I don't know. I'm not involved with Edexcel directly and I don't have special insight into their exam questions. However, my students need some help on this too, so I'll offer some thoughts. This is just me "thinking out loud", not offering any sort of authoritative advice. If there are any Edexcel lurkers visiting this blog, perhaps they'll de-cloak and offer some hints on whether I'm on the right track.

The obvious place to start is by looking at the old Specification. This featured inferential statistics in Unit 2. It also featured multiple-choice questions and some of them looked like this: 

This question from 2009 wouldn't be repeated as a multiple-choice question, but something similar could be asked about the type of data or the experimental design that enables you to carry out a particular inferential test. 

Here's another one from 2012:

Once again, new students won't be getting multiple-choice, but you could be asked what method, level of data or experimental design goes with a particular stats test. (The type of hypothesis is a red herring - it doesn't change the stats test you use).

The old Specification also had longer questions that were closer to the style of question new students will be facing. Here's another from 2009:

The new exam wouldn't give you nearly as much information as this, The Examiner now expects you to use the statistical tables at the front of the answer book and work out the critical value yourself.

So in the new Spec., the question might be more like this:

With this question, it's up to you to realise that "a correlational study" means using the Spearman's Rho critical value table. You have the observed value of +0.519 and N=20 and you can work out from the description that this is a 2-tailed hypothesis (because it doesn't say whether we're expecting a positive or a negative correlation). With this in mind, it's easy to use the table at the front of the booklet to work out the answer to question (b):

You can see that an observed value of +0.519 is bigger than the critical value in every column except that last one on the right, making the results significant at up to p≥0.01 (but not p≥0.005). In case you worry that you'll forget whether the observed value needs to be bigger or smaller to be significant, don't worry: it tells you at the bottom of the table.

How helpful!

I'm assuming here that the Examiner will give you an observed value and enough information to use the right critical value table. But what if the Examiner wants you to work out the observed value for yourself? After all, the front of the answer book also contains those eye-splitting mathematical formulae.

Are you expected to do this in the exam?

It's worth looking at the Exemplar Exam Paper among the SAMs (Sample Assessment Materials) available on the Edexcel website. The exemplar for Paper 1 (Foundations) contains this encouraging rubric:

However, further down the page, there's another rubric:

So, let's get this straight: you may use a calculator (if you want to) but you don't need any other materials. This sets a pretty clear cap on how much maths the exam can expect from you in the time available. Without a calculator, I imagine you may be expected to do some basic addition and subtraction, maybe simple multiplication, but not complicated square roots, long division or the multiplication of large numbers.

Chi-Squared, in particular, is difficult to do without a calculator.

So we can fairly confidently say that students won't be given a complete Mann-Whitney to work out from scratch. And indeed, when we look at the exemplar questions in the SAMs, this is what we find:

This is a bit like the old multiple-choice questions: you have to use the information to figure out whether it's an experiment or a correlation, nominal or interval level data, independent groups or repeated measures; then you can identify which statistical test should be used.

Or this one:

This question expects you to do 7 simple subtractions, then work out the squares of 7 small numbers and add up the total. You need to be able to use the formula from the start of the booklet:

Riii..iiight. So that total I worked out, I multiply it by 6. Now divide the whole thing by, wait a minute, what's 6-squared? What's 7 times 35? What, I have to take the whole thing away from 1 now? I may have to do this without a calculator, right, if I forgot to bring one? For 4 marks?

I'm not very happy about this. It's not that the sums are difficult. They're just fiddly and time consuming and, especially if you don't have a calculator, it's easy to make a careless mistake and have to start all over again. And all of this for a 4 mark question that you need to answer in no more than 5 minutes. And what, really, does it reveal about anyone's understanding of psychology?

Hopefully, this is just a draft question and the real exam won't contain anything like it. Or perhaps the real exam might contain simplified versions, with some of the arithmetic done for you. If it doesn't, I might be advising some of my students "just skip the maths question: you don't need a measly 4 marks that badly in an exam that's out of 90 and you can always have a go at it if you have time spare at the end; focus on the essays instead!"

Nonetheless, the mark scheme for the SAM paper is interesting:

The first two marks are pretty easy to get even with a paper and pencil and no calculator. You get the third mark for showing how the numbers fit into the equation, even if you don't actually work the equation out. It's only the fourth mark that's awkward to work out if you're not a whizz at long multiplication.

If the arithmetic questions in the exam are marked like this, in stages, then even candidates who just suck at maths and who forgot to bring their calculators could still pick up a couple of marks - and it looks like there will only be one question like this in the paper. I'd still recommend skipping it because, once you hit your stride writing essays, you don't want to break that stride by switching to fiddly arithmetic. But if you have a couple of minutes at the end of the exam, it would be worth coming back to a question like this: you get 1 or 2 marks just for filling in the table!

Of course, there may be some new information about the exam coming along soon - or some may already be out there but I've been too excited about Game Of Thrones to notice. If anyone's got better insights into this part of the exam, please comment on this Blog!

In the meantime, here's a document pulling together all the statistics questions from the old Specification, which makes a nice practice exercise for candidates preparing for the AS or school mock exams for the A-Level.

Questions on Statistics - old specification.pdf
File Size:	2956 kb
File Type:	pdf

Download File

---

Belatedly, here's another document that contains more analysis of the SAMs questions aimed at students and 4 new, detailed exam-style questions of my own on the statistical tests.

Stats Questions - practice.pdf
File Size:	1529 kb
File Type:	pdf

Download File

---