‘Shocking’ data suggested that a fifth of the population was considering leaving the country. The National Party even shared it. But how accurate is it?
This story was first published on Stuff.
On July 8, the National Party released a statement. It read: “Shocking data out today which shows more than 1 million New Zealanders are actively considering leaving the country shows the Government is taking Kiwis backwards, National’s Deputy Leader Nicola Willis says.”
It would certainly be shocking if a fifth of New Zealand packed up and left.
The National Party statement was based on a poll from MYOB, an Australian business services company, released on the same day. The “Consumer Snapshot”, according to MYOB’s website, “polled over 500 people from around the country and discovered that four percent of respondents plan to move overseas to live and work”.
“Based on population data from Stats NZ [that is the population of New Zealand], this number extrapolates out to more than 200,000 people likely to be thinking the same way,” a blog post reads. The MYOB story also noted: “An additional 20% of those surveyed (equivalent to approximately 1,025,000 people based on population data), said they are actively considering moving offshore.”
The poll also touched on a number of related issues, for example what was driving those polled to look overseas (money) and what would convince them to stay (more money).
MYOB’s own blog post has the catchy headline: Kiwis take flight: Is NZ about to face a mass exodus? It’s subtitled: “New consumer research from MYOB indicates over one million New Zealanders may be actively considering leaving the country in the next 12 months.”
This particular study was reported on by Stuff (not angled on the one million figure), Newshub (very much angled on the one million figure), and referenced in a column by ACT Party deputy leader Brooke van Velden published by the New Zealand Herald.
'Makings of a real crisis': More than 1 million Kiwis actively considering leaving NZ, data shows https://t.co/H3Kcbwn6nJ
— Newshub (@NewshubNZ) July 7, 2022
It was also shared by the National Party with the headline “ONE MILLION KIWIS SET TO LEAVE”.
Is there anything in all this?
It’s true that more people are leaving New Zealand than entering.
Stats NZ data shows that in the year ending May 2022, there was a net migration loss of 10,700 people – made up of a loss of 8400 non-New Zealand citizens and 2300 Kiwi citizens. The latest May figures are interesting as they suggest more New Zealand citizens are leaving than arriving back – a major turnaround from the days of shut borders. In the 12 months to May 2021 for instance, some 10,400 Kiwi citizens left and 26,400 returned.
(As an aside, another entirely unrelated poll from November 2020 suggested that a million overseas New Zealanders were considering coming home.)
Government officials have suggested that about 50,000 could leave in the next year as the border re-opens. There is significant uncertainty around this prediction, however.
To be fair, it’s not as if National is the only political party to seize on underlying narratives for political purposes. For example, you might recall hearing back in 2020 that it was a “flood of New Zealanders” returning home driving up house prices. Prime minister Jacinda Ardern pushed this narrative when under intense pressure over housing affordability. But this kind of ignored the fact that net migration was way down at the time.
Anyway, concerns about migration are valid; this is a real issue here that is likely to put more pressure on an already tight labour market. But to expect 200,000 (or a million!) to leave is a lot.
Can you chat about the study in detail then?
I thought you’d never ask.
These catchy looking kinds of studies are sent to media outlets all the time. Some are ignored. Some are picked up.
The July 8 media release had little detail on the methodology underlying the poll. It outlined the research was carried out by Dynata (a data and insights company) between June 3 and 14. It said 509 consumers were invited to complete the online survey “which was conducted to achieve a nationally representative sample from across New Zealand”.
I asked for more detail this week, particularly around the sample of 509 New Zealanders and the actual question asked of them.
Let’s start with the key question put to the respondents. The Consumer Snapshot asked all survey respondents: ‘Now that borders are re-opening, have you thought about, or looked into moving overseas to live and work?’
- 4% said “Yes, I’ve decided I’m moving”
- 20% said “Yes, I’ve thought about it but not actioned it yet”
- 37% said “No, I haven’t thought about it”
- 33% said “No, I will never move overseas”
- 6% were undecided
I asked Carl Davidson, director of Christchurch company Research First, for his thoughts on the framing. His view was that the question which prompted 20% to say “Yes, I’ve thought about it but not actioned it yet” was potentially leading and could have been structured differently. For example, it’d have been better to use the following:
Now that borders are re-opening…
- Have you thought about moving overseas to live and work?
- Or have you looked into moving overseas to live and work?
And both of those questions, he points out, had two parts – whether people would live AND work. It’s better to keep survey questions narrowly focused to get good, reliable results. There’s also, of course, a huge difference between what people say and what they will do. I mean, I’ve personally thought about moving overseas, but I’m almost certainly not going to.
But still … 4% said they had decided to leave, right? Well then, is the poll representative of all New Zealanders? Both Dynata and MYOB say yes, it is.
In a statement provided this week, Dynata said the company “conducts surveys that are scripted by clients. As such, we do not make claims, but rather collect data for questions provided by MYOB … We adhere to rigorous standards to ensure the final make-up of survey respondents was in line with nationally representative statistics by age, gender and region.”
MYOB’s response read: “Dynata surveyed 509 adult New Zealanders from around the country. To do so, they drew on a sample of the public selected at random from Dynata’s online panel. Participants in the panel choose to take part in individual surveys, in response to an invitation from Dynata.”
Its statement outlined that polling does not happen unless the sample is representative of the population. It said, according to Dynata, the poll was statistically robust with a 95% confidence interval.
So what did Davidson make of this?
“In science the rule is that ‘extraordinary claims require extraordinary evidence’. Given that the highest external migration New Zealand has ever experienced is about 100,000 people a year, the notion that ‘one million Kiwis are set to leave’ is clearly an extraordinary claim.
“To support this claim you would need some compelling data, and it’s not clear the MYOB survey can provide that. Given how the survey participants were selected and the questions they asked them and – most importantly, the huge difference between what people predict they will do and what they actually end up doing – the sensible response is to stay sceptical.”
And the National Party?
In a written statement, Willis says her language when commenting on the survey matched that from MYOB itself. “The respondents gave reasons for why they were considering leaving, indicating they had given it more than a passing thought,” Willis says.
National, she says, is “deeply concerned” that a significant number of New Zealanders are actively considering leaving.
