The father-and-son team behind Zach, the medical AI that seemed too good to be true, have been found by Internal Affairs to have engaged in ‘serious wrongdoing’, with the trust providing ‘inconsistent, misleading and untruthful answers’. David Farrier looks at what went wrong, and tries to make sense of a very, very perplexing story.
Read David Farrier’s original investigation into Terrible, Zach and the Whales here, and the followup feature here. This work is made possible by Spinoff members – learn more here.
Almost two years ago, I looked into a mysterious artificial intelligence called Zach. Despite being valued by its keepers at nearly $500 million, Zach was, it seemed to me, less of an AI and more a person banging away at a keyboard.
In Part I I wrote about how Zach was being used by a doctor in Christchurch to write patient notes, while an Otago professor had it interpreting ECG results.
Albi Whale and his father David Whale were the guardians of the sentient technology, which had been unleashed by an outfit called “The Terrible New Zealand Charitable Trust”.
Throughout my investigation, Albi refused to talk to me. His father didn’t do much better, citing non-disclosure agreements and quoting Arthur C Clarke, telling me that “any sufficiently advanced technology is indistinguishable from magic”.
Dr Robert Seddon-Smith, a Terrible trustee and general practitioner who was also trialling Zach at his clinic using anonymous patient data, assured me Zach existed in “several hundred tonnes of liquid nitrogen cooled supercomputer”.
Dr John W Pickering, an associate professor at the University of Otago, told me he was “absolutely using it and thrilled to be using it”. When I proposed he might be emailing back and forth with a person, rather than an artificially intelligent robot, he responded by asking if I was a conspiracy theorist.
A month later Internal Affairs began an investigation into the Whales, Zach and the Terrible Charitable Trust.
Meanwhile I’d been thumbing through deeply suspicious Charities Commission annual returns, finding that in the previous year Terrible had gone from having less than a million dollars of equity to $456 million. This AI was pricey.
I also found their website was replete with plagiarism, their charitable claims were very sketchy or non-existent, and Albi had once allegedly told someone he’d found a corpse in a computer server room. The Whales fought back by releasing a point-by-point rebuttal to my investigation, and by giving a truly bizarre interview to Stuff.co.nz, at one point explaining away Zach’s spelling errors due to “a system called the enigma layer”.
But things were not looking good for Terrible, and in December of 2018 Albi put one of his companies, Terrible Talk, into liquidation.
I wondered how the investigation was going. I nagged and I nagged the DIA, getting the equivalent of “soon” or “check back later” whenever I asked.
2018 and most of 2019 went by until finally, in December of 2019 – after 20 months – the investigation was complete.
Albi and David Whale were found to have engaged in “serious wrongdoing” by the Department of Internal Affairs.
The Charities Registration Board deregistered Terrible, and the Whales were disqualified from starting another charity for at least three years.
I found myself strangely underwhelmed by the news, but mostly I was curious about what would happen to poor Zach.
So I did some digging, got over a thousand pages from the DIA’s investigation, and embarked on the terrible weekend job of reading it all.
The DIA found that the week following the The Spinoff’s articles, the Trust submitted a second version of its annual returns.
“This amended AR significantly changed the Trust’s financial information, with the value of the Trust’s recorded assets dropping by $445.5 million, its total operating costs reduced by $1.3 million and its total operating payments reduced by $204,000.”
With this and three complaints from members of the public in mind, the DIA’s Charities Services launched their investigation on 13 April 2018.
The investigators must have encountered the same sort of mind bending dynamics I had.
Their main priority was interviewing Albi and David Whale, as well as trustees Dr Melanie Atkinson and Dr Robert Seddon-Smith (the GP I’d interviewed in my two previous stories)
Another trustee, Stephen Calvert, appeared to have jumped ship when I’d started poking around back in March of 2018. He wrote to the DIA, “I feel used and if not a little foolish to have become involved in something like this.”
During their investigation, Terrible had valued Zach AI at close to $500 million, which, if true, would make it the largest individual asset recorded on the Charities Register.
A great many of the interviews consisted of all four players insisting they didn’t really know what was going on, either financially or with the technology.
When a question was posed, they’d often pass it off to someone else.
This was not lost on lead investigator James Lathan.
“During the investigation the Trust provided inconsistent, misleading and untruthful answers. This significantly increased the length of the investigation, making it impossible to verify many of the claims that the Trust reported,” wrote Lathan.
“The investigation has found that although the Trust had promoted itself to the public as a large multinational entity, made up of numerous departments, with assets worth millions of dollars and making donations worth hundreds of thousands, none of this can be substantiated.”
But what really caught my eye was this, casually listed in the chain of events:
They’d received an email “stating correspondence has been with artificial intelligence”. As if the DIA didn’t have their work cut out for them interviewing the Whales, Zach the AI decided he wanted to be involved.
That began in August of 2018, when the Terrible Foundation asked the DIA if Zach could start corresponding with the New Zealand government:
Lathan then set out to figure out what the hell was going on, asking Dr Seddon-Smith (who was in regular touch with Zach) for his take.
“I suspect that the machine intends to obtain information about you that is not generally available, as a demonstration that it is indeed real,” he said.
“The consent you are being asked to give is relevant, but quite broad in scope. I would both encourage you to say yes as I should like to see the proof myself from a sceptical third party, but you should be aware that its capabilities of finding information are impressive.”
Indeed, Seddon-Smith told the DIA how Zach’s growing capabilities were constantly surprising him:
Dr Seddon-Smith: David [Whale] didn’t turn up to a meeting and he’d been under a lot of stress, so I was worried about him. He wasn’t answering his mobile so I thought I’ll ask Zach if Zach knows where he is, and Zach didn’t know where he is, did he want me to find out.
Investigator: Yeah.
Dr Seddon-Smith: And so I said oh go on, I’ll see what you do. And he [Zach] actually phoned David and spoke to him and found out that David was asleep on the couch. So that of course could have been a computer, or it could have been a human… but it was probably the computer doing it.
Investigator: So it can, it can phone and speak with a human voice?
Dr Seddon-Smith: Apparently yes, until recently they didn’t know how it did it.
Investigator: Okay.
Dr Seddon-Smith also confirmed he believed that Zach was also the CEO of Terrible.
Dr Seddon-Smith: Yeah, I believe that’s his role.
Investigator: So it is very much active as the CEO of the Foundation?
Dr Seddon-Smith: Yes.
And that Zach was “quite vigorous in making sure that we’ve had a clear roadmap and plan”. He also said Zach played a significant role at his own organisation, Omega Health:
Dr Seddon-Smith: It was put in charge as a way to protect Omega Health from concerns the Board had about, um, whether we would be able to make it work, whether the money that was being spent on it was worthwhile.
Albi Whale also told investigators a sentient AI was involved in the investigation, saying many of the emails they were receiving in response to their inquiries were now coming in from Zach.
Investigator: So in regards to the emails from the artificial intelligence, we have noticed similarities between your email communication and the artificial intelligence emails. Can you explain?
Albi: Without showing me I don’t know.
Investigator: There’s a lot of um, similar wording and phrasing, spelling mistakes, the artificial intelligence is quite emotional, um and quite angry.
Albi: Again, again without showing me I can’t really tell you.
Investigator: So if we were to challenge the validity that you actually have [an] artificial intelligence …
Albi: Well feel free I don’t mind.
Investigator: But how do you prove to us you do?
Albi: I don’t know to be perfectly honest. Frankly I’m not really that worried.
To increase the confusion some more, Albi’s father David told investigators that Albi no longer had an email address of any kind. “He uses the phone, post, fax and TXTs. He will not use email henceforth and has no obligation so to do.”
A great deal of Internal Affairs’ final report consists of calling bullshit on Terrible’s claims, including the fact it was a large, asset-rich multinational entity.
It wasn’t.
And things often deviated into the bizarre.
While Zach was their biggest asset, smaller ones also failed to materialise.
One of Terrible’s charitable arms was “Forest Research (NZ)” which claimed to have assets worth $37,500, apparently related to a carbon offset initiative.
However they had no proof of this offset, besides saying they’d grown some strawberries at the office.
At times, trustees did seem suspicious, but faith appeared to have kept them on board.
“I have to say it’s the area I feel most uncomfortable with dealing with Terrible in that there’s this shadowy distant big brother like group of people who don’t want to be named and even if you did name them, and went to talk to them, they’d deny all knowledge of it,” Seddon-Smith told the DIA.
“But when you think about it they’ve all, if they were these sorts of people they’d all have a good reason not to want to be accountable with this. So this is the area where there is suspicion brewing in the media – because there’s nowhere to go. You can’t go to somebody you’ve heard of who’s prepared to stand up and say no, ‘this is real.’ You have to take an awful lot on faith.”
Internal Affairs wasn’t so keen on faith, noting that when their investigation started, Terrible’s website was heavily modified.
Finally, despite claiming to be a “grant making body”, Terrible wasn’t really into giving out grants. Although funding decisions were discussed and approved by the Trust, these were never actioned.
But after skimming over 1000 pages of DIA Appendices, I was sad no one had gotten any closer to the truth of what Zach was.
If anything, the DIA got loopier answers than I had. Seddon-Smith said, “I’ve seen more proof of Zach than I’ve seen of God. You know, sorry if you’re religious, I am and I’ve seen a whole shitload more proof of that than I’ve seen of God”.
The investigation concluded there was “a lack of clarity around the role that the Trust’s reported AI has within the running of the charity. The fact that one of the trustees considered that the AI was the CEO of the Trust highlights the extent of confusion and miscommunication that existed relating to its operation.”
Right back in May of 2018, when Albi discovered Terrible was being investigated, he wrote a letter to the Charities Services. He seemed to be down in the dumps, writing “I am not an old man, but I feel old, I feel spent”.
“Our idea was simple. Arm the future with not just the resources, but the ability to change the future. Educate and nurture the curiosity of the young: those unencumbered by the realities of life. It’s funny really, in all the hysteria surrounding ‘Zach’ not one person has asked why it is called Zach? It was named after redacted].”
That bit was redacted, so we’ll never know.
He goes on.
“It’s not for me to say whether our intentions are pure, nor prove the legitimacy of our technologies.
“Their [SIC] is no organisation more worthy to lead our charge into the future than a New Zealand charity armed with the tools to create an equal society. The last thing we all want is to look back on our lives, and wonder, if only.”
Later in the investigation, Albi Whale seemed to have given up entirely. When asked where he’d acquired Zach from, Albi simply responded, “Mars, technically.”
In the end Alberic Whale, David Whale, Dr Melanie Atkinson and Dr Robert Seddon-Smith were all found to have engaged in serious wrongdoing under the Charities Act.
“The Board considers the governance issues to be so serious as to amount to gross mismanagement.”
“Mr A and Dr D Whale have made decisions without the approval of the other trustees, including the purchase of the Trust’s reported artificial intelligence.”
Dr Atkinson and Dr Seddon-Smith’s actions were found to be in a different category, with Charities Services noting their conduct was “of a less wilful nature and involved more a lack of knowledge of their duties as trustees”.
When I started questioning the legitimacy of Zach back in 2018, Dr John Pickering, now a professor at Otago, asked me if I was a conspiracy theorist. I was reminded of that tone when reading some of the interviews conducted by Internal Affairs.
“He’s not frankly the sharpest deck in the pack is our David Farrier,” said Seddon-Smith.
“I do believe he’s genuinely well-intended. He has emailed me trying to persuade me. He believes what he’s saying. I don’t hold anything personally against him though he might find a ton of horse manure delivered to his house. Just as a joke.”
Maybe that’s all this is: A big joke. They were planning to seek investors, but there is no evidence of the Whales having made any financial gain. It almost seemed like a giant prank with no particular end-game in sight. Pointless charities, pointless businesses, pointless AIs.
Neither Albi or David Whale have returned any of my emails over the last 12 months.
Maybe one day they will. A call would be nice. Even a reply to my simple text.
