While New Zealand moved to a standards-based education system over 33 years ago, students and teachers suspect that students are still graded on a bell curve. Are they?

The first time I was introduced to the idea of bell-curve grading (AKA norm-referencing, relative grading, and grading on a curve), my design teacher in high school stood up in front of the class and said, “Technically, I can only give two or three of you in the class an Excellence.” That’s the top grade in the NCEA system after Merit, Achieved and Not Achieved.

Uninspired, I took this as my cue to wag. Bell-curve grading means you can’t have a class full of A students or a class full of failed students. A standard grading curve would mean a minority of students get very high grades, a minority get very low grades, and most get an average pass, regardless of their performance. 

The idea seemed archaic, but before the development of the New Zealand Qualifications Framework in 1991, ranking and scaling was used in the School Certificate exam. Internal exams were initially scaled to ensure only 50% of students gained a C grade or higher, and then grades were adjusted to be consistent with results from other schools nationwide. 

Professor Gavin Brown from the University of Auckland researches testing and assessment and how they impact students and teachers. “Ranking does not indicate clearly what students know or can do,” he says. “It simply positions people in order.” 

The adoption of standards-based assessment meant that bell-curve grading was meant to be a thing of the past. Under NCEA, “your performance is judged according to criteria that reflect competence to excellence, independent of how other students perform or how you perform in other subjects,” says Brown. “What you can do should not be downgraded just because others can do more.”

This is called “criterion-referenced assessment”. NZQA deputy chief executive of assessment, Jann Marshall, says that students who sit NCEA are graded “on their ability to demonstrate the skills and knowledge set out in the standard, rather than being ranked against each other to fit a bell curve.” 

However, Shannon*, a teacher at a low EQI (previously known as high decile) Auckland school, says that while “it’s really hard to know whether scaling or bell curving is happening,” they’ve noticed trends where a certain number of Excellence-deserving students are scaled to a Merit in the moderation process, especially when there is a large number of them.

Shannon says bringing all of their students up to an Excellence grade can be realistic, given the students don’t have learning difficulties. “But there are often sneaky barriers in place with the marking criteria ending up being subjective to the marker in my subject,” Shannon adds, “particularly as a creative one.”

South Auckland teacher Jamie* also suspects scaling and norm-referencing may be happening: “I’ve seen teachers award students a higher or lower grade than they should have been to fit a standard distribution.” Questions around the changing cut scores and Profiles of Expected Performance (PEP) regularly come up in teachers’ meetings and conferences, Jamie says. “I do think the whole idea of the PEP shifting each year shows that they’re looking at last year’s [results]…to then make sure that we’ve not got too many people getting an Excellence or too many people getting an Achieved,” Jamie adds. 

The term PEP keeps coming as I talk to more teachers, but none of them can actually confidently tell me what it is. The only public information that NZQA provides about PEPs are two very similar myths and facts documents referring to them as bands of acceptable tolerances for external assessments.

I ask Marshall for more information. She says that PEPs are ranges that indicate the expected distribution of results based on the previous year’s results and marking panel feedback on how students performed. If marking panel leaders see a significant deviation from these PEPs, it could indicate inconsistency in marking and/or the difficulty of examination, she says.

Marshall said PEPs have been in place since 2009 and are “generally well understood by the sector.” But Jamie disagrees: “It’s kind of up to ourselves to find that stuff out.” Shannon doesn’t know what a PEP is.

PEPs aren’t the only factor moving grades around. The cut scores, which determine the numerical range where an Excellence, Merit, Achieve or Not Achieve grade sits, change at the time of marking. Cut score adjustments are “based on discovering if the questions were too hard or too easy compared to the standard,” Brown explains. “If the test by accident was really hard, then the cut score for Achieved should be lower than otherwise.” 

Brown says this is normal and appropriate. He says it’s not until test results are analysed that test writers can be sure that the questions were set at the right difficulty. “This is hard to do and [NZQA’s] standard-setting protocols are correct,” he adds. “This is not scaling.”

The assumption with PEPs and cut scores is that in a national assessment, characteristics of the cohort do not change significantly from year to year. Jamie worries that if PEPs and cut scores are scaled off results from “schools in higher socioeconomic areas where kids typically have higher English literacy levels”, then schools in lower socioeconomic areas like theirs would be negatively impacted.

So would NZQA be happy if all students happened to achieve an Excellence grade one year? 

Marshall says that would be extremely unusual and if it occurred, NZQA would need to investigate whether the examination was too easy and whether the marking panel was too lenient. “However,” Marshall adds, “where students demonstrate the skills and knowledge set out in the standard to Excellence level, they would be awarded an Excellence.” 

This year, NCEA pass rates have dropped for the third year in a row. “When pass rates drop,” Jamie says, “pressure is first put on educators as possibly being the prime cause.” Jamie doesn’t have an answer as to why rates are dropping, “especially given that students had a range of support mechanisms throughout Covid lockdowns.” But they don’t think students are getting dumber.

So are students being graded on a bell curve? “There is a possibility that none of these things we worry about are actually happening,” Jamie says, adding that the confusion might come down to a lack of understanding by teachers. But, he adds, it might also be that NCEA is not being “super transparent in how and why they undergo all of their processes.”

*Names changed to protect employment security.