spinofflive
It is unknown how many of Stats NZ’s stick figures completed the census
It is unknown how many of Stats NZ’s stick figures completed the census

SocietyJuly 10, 2018

Stats NZ under fire over ‘very serious’ shortfall in digital first census data

It is unknown how many of Stats NZ’s stick figures completed the census
It is unknown how many of Stats NZ’s stick figures completed the census

Statistics NZ has conceded that the 2018 census response rate may be down by almost 5%, sparking concerns that some groups might not be captured by the survey. It has also led to a delay in the release of first results till March next year, and an apology from the government statistician

The 2018 Census, conducted using a much heralded “digital first” model, has fallen short, with the announcement of a 4.5% decline in responses, a rethink on methodology and a delay in the issuing of results.

The need for further “imputation”, in which supplementary information is used in an attempt to plug the census gaps, has raised concerns that some groups of New Zealanders could be unsatisfactorily captured in results.

The shortfall, enough to require what Stats NZ called a “revised methodology to compensate for the missing data”, invites fresh questions about the wisdom of emphasising online responses in 2018, with the old-fashioned door-knock approach only used as a follow-up.

The announcement came in a paper published late last week by Statistics NZ, titled “2018 Census: Potential impacts of revised methodology”. It confirmed the announcement – largely unreported – that the census had seen a significant drop in response. “Interim figures show that we have full or partial information for at least 90% of individuals, compared with 94.5% for the 2013 Census,” the paper explained.

While the target for issuing first results had been October 2018, that had been pushed back. “Because individual responses are lower than we had planned, we need more time than we’d originally anticipated to draw on other information sources and new methods to achieve the highest quality dataset … We are now looking towards a first release of census data in March 2019.”

Indications of a 4.5% drop in response were “very serious”, said Thomas Lumley, professor of statistics at the University of Auckland. “The point of the census is that it’s complete, and it’s what you benchmark everything else to. Ninety per cent is really not good.”

Statistics NZ said that in some cases it will need to draw on “information from the 2013 Census and administrative data to populate missing variables”. Among the “administrative data” sources used for imputation are the Department of Internal Affairs, MBIE, the Department of Labour, the Ministry of Education and Inland Revenue.

“The government has a lot of data on nearly everyone,” explained Lumley.

“Anyone who has for example paid taxes or got medical care or immigrated or anything like that, there are records the government has. And a lot of that data is linked together in the Integrated Data Infrastructure [IDI] …

“Suppose you need to know somebody’s income. If you know their age, and where they live, and their occupation, that gives you a range of plausible incomes, and you can either do a best guess or you can have multiple plausible values, which accounts for the uncertainties, and that lets you fill in quite a bit of information,” he said.

“It helps that the administrative data is fairly complete. But there is going to be some people who are double counted and some aren’t counted at all. The real problem is if there are groups who are poorly represented – if for example there are particular ethnic groups or particular age groups who are missed more often. If you’ve got a small minority group and a 10% hole, you can lose quite a lot of people into it. So it depends how good the coverage of the administrative data is to get the counting right.

“To some extent some vulnerable groups will be captured quite well, because the IDI has more data on them. So people on benefits, for example, the IDI will probably capture quite well. But some other groups may be underrepresented.”

The full impact would not be measurable in the short-term, said Lumley.

“The thing about imputation is while it can work really well, it’s very hard to tell how well it has worked in a particular setting … In five years’ time, looking at this data retrospectively, assuming the next census works better, they’ll be able to redo imputation and look retrospectively, and then the data will improve. But the next five years it’s going to be lower quality data, and we won’t know where it’s lower quality.”

Lumley said it was unknown whether the digital first approach was to blame for the drop in responses.

“What it may reveal is that they tried to save too much money with digital-first. Because if you’re going to do a census at all, it’s worth paying to get it right. There’s a lot of value to a good census and much less value to a not very good one.”

Government statistician Liz MacPherson acknowledged there were shortcomings in the “digital first” approach, in which the census was predominantly conducted online, with the traditional door-knockers coming into play only a follow-up phase in an attempt to reach those who had not responded, and issued an apology to those who struggled with the new system.

“While we can’t be sure yet why we have a lower response from individuals, there are a number of factors we will explore as part of our planned review. We already know we didn’t get everything right,” she said in a statement.

“We built new systems and processes to run this census, and while the majority of New Zealanders were able to take part without a hitch, we know that some people did not have a good experience this year. I have had mixed feedback from people. For some it was the easiest census ever; for others it has been a frustrating experience. For that I am sorry.

“As with every census, we will be undertaking a full independent review to ensure we can make improvements next time.”

In response to questions from the Spinoff, Statistics NZ said that a 4.5% figure was not yet reliable.

“It’s too early to confirm the final response rates yet. We expect to confirm the final response rate with our first Census data releases, scheduled for March 2019. In each census the final response and coverage rates are calculated by the Post Enumeration Survey which estimates the number of people the census should have counted,” it said in an email.

It added: “Stats NZ applies robust analysis and processing to the data to ensure it is accurate, and of high quality. The interim individual response rate we reported on was based on full or partially completed individual responses from field operational data.

Stats NZ said it was confident that the reduced census reponse would not make the data incapable of “accurately representing hard to reach groups”.

“We have been investigating the use of administrative data to complement census data for a number of years. This work underpins the revised methodology for 2018 Census and will deliver high quality data and high value data for our users,” it said.

Statistics NZ pointed out that there was “a long term, international trend of declining census response rates”. Because of this the 2018 Census “has a strategic objective to make more use of administrative data to improve the quality of census data”.

Asked whether the drop could be attributed to the digital-first shift, Statistics NZ said: “The digital-first approach was a big change for 2018 Census. We set a target of 70% online participation and our interim figures are showing that more than 82% of responses were online.”

Asked whether budget constraints had an impact on the drop in response, Stats NZ said: “We are undertaking a comprehensive review of the 2018 Census operation to gain a full understanding of why people did or didn’t respond. This will include a review of the budget.

In last week’s paper, Statistics NZ described the chief advantage of a national census is that “data is available at a neighbourhood level and provides detailed characteristics of small population groups. For the 2018 Census we used a new model for collecting the information. We focused on online participation followed up with postal reminders and household visits for those who had not taken part on census day.”

It added: “One of the goals of the 2018 Census was to improve data quality while modernising. The objective was to ensure accuracy of national counts and reduce variation in subnational response rates. Our interim calculations show that we have not reached our target coverage rates of 94 percent or higher …  However, our interim calculations show full or partial information for about 90 percent of individuals. The interim response rate varies across subpopulations and small geographic areas. To be able to provide good-quality information for all subpopulations and small geographic areas, we need to have very high coverage and response rates across all of New Zealand.

“Given the interim position of individual response rates for the 2018 Census, we are looking at expanding our imputation approach. We are investigating how we can impute households, and cases of item non-response. Both item and unit imputation will improve data coverage and, occasionally, data quality, but not for all census variables. If we do not impute, there will be large amounts of missing data that will affect the overall quality of the dataset.”

Keep going!
Danyl vs Danyl vs Danyl (photo: Reservoir Dogs)
Danyl vs Danyl vs Danyl (photo: Reservoir Dogs)

SocietyJuly 10, 2018

A ferocious debate between three implacable enemies about free speech

Danyl vs Danyl vs Danyl (photo: Reservoir Dogs)
Danyl vs Danyl vs Danyl (photo: Reservoir Dogs)

Phil Goff’s decision to ban two right wing Canadian provocateurs from Auckland council venues has a lot of us re-examining our views on hate speech, free speech and censorship. Danyl Mclauchlan sat down with Danyl Mclauchlan and Danyl Mclauchlan to debate the issue.

Liberal Danyl: Okay, let’s try and think our way through the whole Phil Goff vs Lauren Southern and Stefan Molyneux free speech issue.

Left-wing Danyl: What’s even to think about? A couple of awful people wanted to come to our country and stir up hatred against members of vulnerable minority groups, and now they aren’t. Good outcome.

Conservative Danyl: I agree. It is a good outcome. And I wonder about the wisdom of dragging this issue out. It feels like some people in New Zealand are looking at the US, which is tearing itself apart over these very divisive culture-war debates, identifying it as a way to get attention for themselves and saying ‘Hey, let’s do that over here!’ Let’s not.

Liberal Danyl: I’m just worried about the implications of Phil Goff making captain’s calls about who can and cannot speak in public. It sets a precedent.

Left-wing Danyl: The precedent that Nazis aren’t welcome here seems fine.

Liberal Danyl: Let’s start with that. For seventy years the left have called anyone to the right of Rosa Luxemburg a Nazi, so when I hear you support banning Nazis and punching Nazis, it sounds suspiciously like a pretext to ban and beat up anyone who disagrees with your own very marginal views. Are these people Nazis? What even is a Nazi? Who decides? Isn’t it just an insult at this point?

Conservative Danyl: Southern supports White Nationalist groups in North America and Europe-

Left-wing Danyl: ie the Nazis.

Conservative Danyl: She identifies as a libertarian, while Molyneaux identifies as an anarcho-capitalist. He’s been referred to as a leader of the alt-right but rejects the label.

Left-wing Danyl: Southern was detained by the Italian Coastguard for blocking a search and rescue mission for refugees drowning in the Mediterranean. Is debating whether it’s polite to refer to someone like that as a Nazi really the hill you want to die on here?

Liberal Danyl: I think these distinctions are important, because it’s not hard to imagine a future in which, I dunno, Judith Collins becomes Mayor of Auckland and announces that she’s going to prohibit groups and speakers on the left from renting council venues because look at the history of Communist totalitarianism, they’ve even worse than Nazis, yada yada yada. And justifying it by citing this as a precedent. Or maybe muslims won’t be allowed to speak because terrorism. People on the left operate in this frame where it’s your team who are the good guys versus the bad guys who are all nazis, so this is a win for the good guys. But we’re a society of laws and norms, and when politicians override those norms because it’s publically popular then it’s actually a loss for everyone with views outside the mainstream.

Conservative Danyl: I highly doubt a hypothetical Mayor Collins would do any of that. But you’ve gotten ahead of what free speech even is. Free speech means you can say what you like and the government won’t arrest you. That’s it. There’s no free speech obligation for the council to rent you a venue so you can advocate for white supremacy. That’s ridiculous. People can still watch Southern and Molyneux’s videos online. If they do come here – which it looks like they won’t – then they can try and hire other venues. They still have total freedom of speech.

Liberal Danyl: That definition of free speech worked very well back in the 19th Century, not so well today when the state is much larger and more powerful. If the government decided they were going to ban, say, Greenpeace from having any representation on Radio New Zealand and TVNZ, both owned by the state, wouldn’t that be an attack on their freedom of speech? Would you say ‘Well, the government isn’t arresting them so they still have total freedom of speech?’ Can the state fire lecturers or teachers who say things they don’t like? Feels to me like that would be a breach of freedom of speech.

Left-wing Danyl: Wouldn’t you have to ask why they were being banned or fired? Because there’s no absolute right to free speech, on or off state-owned media. You can’t threaten to kill someone because that’s a form of speech that is not allowed. You can’t repeat defamatory statements. You can’t breach suppression orders. Lecturers can’t violate their students privacy. Freedom of speech is compromised in all kinds of ways for pragmatic, totally uncontroversial reasons which no one ever challenges until a couple of white supremacists come along and you’re all suddenly like ‘Oh no! We have to let them speak because we have to have total freedom of speech!’ No we don’t. This comes under a category of offensive speech which is not protected because the social harm it can cause outweighs the public good of free speech.

Liberal Danyl: Who decides what speech is so offensive it can be banned? We’ve all seen the paradox of tolerance cartoon, but who decides what views are so intolerant they meet the criteria for censorship? Because anyone can point at an ideological adversary and say ‘Those ideas are dangerous’, or ‘I’m so offended by those arguments that no one should be allowed to express them’. Isn’t free speech about supporting the right of people to say things you don’t like, and don’t agree with? If these ideas are so terrible why not just debate them? Why doesn’t the left show everyone how bad these arguments are by refuting them instead of running and hiding behind de-platforming and performative outrage?

Left-wing Danyl: That is a complete straw man. I hear ideas and arguments that are offensive to me all the time and I debate them all the time. No one is saying ‘Offensive speech should be banned.’ The argument here is that some speech is harmful. That there are vulnerable groups in society who are subject to persecution and violence, and that people like Southern and Molyneaux help cause and perpetuate that violence. Their speech has real and tragic real-world repercussions. It’s all very well for you to say ‘Hey, let’s debate them because we should have total freedom of speech,’ when you don’t actually suffer the consequences of harmful speech that demonises minority groups. This is a much more difficult argument for champions of free speech to address, because it puts the right to free speech up against even more fundamental rights, like the right to life. That’s why we get this faux indignation and outrage about the intolerant left being afraid of offensive speech. It’s easier to divert the debate than accept that some speech is harmful and needs to be restricted.

Liberal Danyl: But that brings us back to my earlier question. Who decides what speech is harmful and how that harm is weighed against the wider public good of the right to free speech? Take the artistic censorship debates of the 20th Century. Back when Gore Vidal published The City and the Pillar there was a widespread belief among psychologists and psychiatrists that homosexuality was a form of mental illness. Not only was a book that advocated for gay relationships seen as deeply offensive, it was also a book that experts could describe as harmful to the public. Should they have adopted your standards and banned it because society needs to be protected from harmful speech?

Left-wing Danyl: That is classic victim inversion. Gay people were the victims of prejudice, and still are, and Nazis are the oppressors.

Liberal Danyl: You’re still locked into this mentality in which there are good guys and bad guys-

Left-wing Danyl: That’s because there are groups that are oppressed and groups that want to oppress them and I side with the oppressed!

Liberal Danyl: But that’s the exact same way Southern and Molyneaux see the world! They see themselves as victims of state oppression.

Conservative Danyl: That’s a key point. Southern and Molyneaux didn’t want to come to New Zealand. They wanted to get banned. They cancelled the instant Goff denied them a venue because they want to present themselves as the victims of censorship and oppression. That’s their PR strategy. When Milo goes to campuses in the US and announces that ‘feminism is cancer’, that doesn’t mean anything: there’s no message there, there’s no argument; it’s purely designed to trigger the libs and provoke the exact response that you’re delivering and make him a martyr of ‘free speech’. You’re part of their business model. None of this works without you responding the exact way they want you to. If you really want to minimise the social harm they’re causing why are you playing their game? Aren’t we all better off ignoring them?

Left-wing Danyl: Easy for you to say when they’re no threat to you. What if ignoring the Nazis is what allows them to seize power?

Conservative Danyl: And what if a toxic media ecosystem that empowers both the radical right and the radical left but precludes the sane 99.9% of the population is what empowers them?

Liberal Danyl: If only there was a pre-existing institution separate from that ecosystem that protected minorities from populist politicians while balancing conflicts between individual rights.

Left-wing Danyl: You’re talking about the legal system? You think free speech issues should be resolved through the courts?

Liberal Danyl: Duh.

Left-wing Danyl: The legal system is expensive. It privileges the wealthy and powerful over the poor and marginalised.

Liberal Danyl: It found in favour of the aged care workers just last year.

Left-wing Danyl: Because a progressive organisation spent enormous time and effort fighting an extensive legal battle.

Liberal Danyl: Isn’t that how it should work if you want to take away someone’s rights? If someone’s speech is so harmful it can bring about loss of life, shouldn’t that be quite easy for you to demonstrate in court? Wouldn’t you welcome that opportunity, if you’re so confident that this harm is real? Maybe white nationalist ideas can be shown to be so dangerous that they should be banned. Or maybe there is no danger, the speech should be allowed and you could do something more constructive with your time?

Conservative Danyl: And, if Southern and Molyneaux really feel their message is important they can defend their right to be heard. Which, because they’re almost certainly just scam artists and grifters, they’d be unlikely to do. Conversely, if someone wanted to take away your own right to speech because they’d decided you were offensive and harmful, wouldn’t you want a more robust system protecting your rights than whatever Phil Goff thought would play well in the media? Or do you not actually care about these issues and just want to rant about them on the internet for clicks and likes?

Left-wing Danyl: So you guys are advocating some kind of expansion of the legal system that could make blanket decisions about what speech is and isn’t permissible.

Conservative Danyl: Seems like that’s where we’ve ended up.

Liberal Danyl: Pretty much.

Left-wing Danyl: Did you ever notice how conservatives and liberals often start out by advocating for individual freedom and end up endorsing the expansion and intrusion of the state into the public sphere? Goddamn Nazis.


The Bulletin is The Spinoff’s acclaimed, free daily curated digest of all the most important stories from around New Zealand delivered directly to your inbox each morning.

Sign up now


But wait there's more!