spinofflive
The headline of a Stuff article produced by Chat GPT
The headline of a Stuff article produced by Chat GPT

MediaMarch 12, 2024

Oh no! Turns out AI sucks at writing

The headline of a Stuff article produced by Chat GPT
The headline of a Stuff article produced by Chat GPT

Mad Chapman takes a dreary walk through a Stuff article written by AI. 

Last week, the sad saga that is “news about the news” continued with reports of major cuts at TVNZ, including the proposed cancellations of current affairs stalwarts Sunday and Fair Go, as well as a restructuring (read: downsizing) of youth-focused arm Re:. Once again, journalists around the country were left standing in the empty stadium of their industry, kicking empty cups around and wondering where everyone went. 

So at the end of the end, it was all the more tragic (depressing? crack-up?) to see Stuff publishing an article written by AI, or more specifically, “summarised from original Stuff reporting and published member comments using generative AI tool ChatGPT with oversight and editing from Stuff journalists”. In other words, Stuff took data from its reader comments and turned it into an article for those same readers to comment on. 

Technology can be immensely useful, and artificial intelligence has its uses already in journalism (if you and your subjects speak with a Pākehā accent, automatic transcriptions are extremely helpful, for example) but much has been made of a potential future where all journalists will be replaced by AI, because AI can write 800 words in 30 seconds. But can AI write good words?

Stuff’s AI-authored article would suggest the answer is no. In fact, I will say right here that as of March 8, 2024, AI sucks at writing. 

‘New Plymouth not happy about it’

I will give credit where credit is due: ChatGPT has come up with an absolutely cooked headline that made me click. “Stuff poll says Christchurch is NZ’s best place. New Plymouth not happy about it”. Christchurch is NZ’s best place. Not city, not town, not place to live or to visit, just place. New Plymouth not happy about it. This is a story about two characters and they are Christchurch and New Plymouth. The headline defies convention and makes little to no sense. Of course I clicked on it.

‘Ah’

The article opens as any human-written article should – “Ah”.

“Ah, the eternal Kiwi conundrum: Where’s the best slice of our paradise?”

No notes.

“Stuff tossed this question into the ring, and New Zealanders, never shy to back their own patch, came out swinging with opinions as varied as flavours in a Whittaker’s sampler.”

This second sentence is so close to sounding like an actual person wrote it, which is Chat GPT’s style, but it’s go far too much going on, which is also Chat GPT’s style. It’s also a sentence no self-respecting editor would accept from a writer.

There are idioms (tossed into the ring), local jargon (own patch), sports metaphors (came out swinging) and a strange hybrid of two Kiwiana staples (Whittaker’s and Sampler boxes). Reading this sentence is like looking into a funhouse mirror. Technically all your parts are there but you look monstrous.

The article then goes on to note that Hamilton is now the fastest-growing city in New Zealand. But instead of just saying that, Chat GPT has added some more sporting flourishes. “Between mid-2022 and mid-2023, Hamilton puffed out its chest with a 3.4% growth spurt, outpacing Tauranga’s 2.5%.” According to the author ChatGPT, every city is a person with a human body.

An old photo of New Plymouth, presumably still not happy (Photo: David Hallett/Getty Images)

‘The whole country’s playing musical chairs with population stats’

This is just a sentence that I don’t understand. 

‘Surf-ready shores of New Plymouth’

Now to the crux of the story: a Stuff poll asking readers where the best place to live in New Zealand is. Christchurch won, with 24% of the vote. A huge 19% added their own alternative answers in the comments. “Thus, the comments section was a riot of loyalty, with pitches for places from the sun-soaked Kāpiti Coast to the surf-ready shores of New Plymouth.”

‘New Plymouth enthusiasts’

Apparently New Plymouth had the most mentions in the comments. What did those comments say? “New Plymouth enthusiasts’ comments painted a picture of a utopia where you could hit the slopes and surf waves without losing your parking spot.” New Plymouth enthusiasts is a wonderful turn of phrase. As for the utopia where you can hit the slopes and “surf waves” (as opposed to the lesser-known pastime of “surfing”) without losing your parking spot? Only Chat GPT knows. 

The comments

After a whole 269 AI-generated words, the article ends with a collection of poll comments from real people. They’re almost jarring to read in their human-ness. A friendly reminder that real people have tone.

The blurb at the top of Stuff’s AI article does say it was edited by Stuff journalists but I’m going to give those journalists the benefit of the doubt by saying I don’t think they did much editing. Because as much as we can all be wowed by the power and progress of AI, we’re not yet at the stage where a machine can write better than a real journalist, or even convincingly as a human. Maybe one day it will (and boy will it be cost-effective!) but until then, I’d much prefer to read the writing of Stuff’s living, breathing journalists, who I can only hope would never write…

“Stuff tossed this question into the ring, and New Zealanders, never shy to back their own patch, came out swinging with opinions as varied as flavours in a Whittaker’s sampler.”

PS I wrote this article very fast but not nearly as fast as Stuff produced the Frankenstein’s monster of AI reporting on Friday. So perhaps the real loser in all of this is, once again, me. And New Plymouth.

‘Love The Spinoff? Its future depends on your support. Become a member today.’
Madeleine Chapman
— Editor
Keep going!
The image is a repeating pattern of the logo for The Bulletin, the Spinoff's daily morning news email
Image: Archi Banal

MediaMarch 12, 2024

Want to be the next editor of The Bulletin?

The image is a repeating pattern of the logo for The Bulletin, the Spinoff's daily morning news email
Image: Archi Banal

Want to work for The Spinoff? We’re looking for an experienced writer/editor to deliver The Bulletin, one of our most valued and widely read products.

The Bulletin is a flagship product for The Spinoff and one of the most widely read in our stable. It’s a curation of essential news and the best journalism from around Aotearoa, delivered as a newsletter to a large subscriber base each weekday morning at 7am. The lead is also published on site each weekday morning. 

We’re looking for someone who is passionate about the news in this country and is energised by the opportunity to deliver a trusted and elevated source of it each morning for a dedicated audience. The right person needs to like early mornings, stay across news developments and have a healthy and varied media diet. Our ideal editor will be a confident and engaging writer and a quick and balanced thinker. They will be someone readers trust and can build rapport with, capable of delivering a calm and informative read of the political, economic, social and cultural news landscape each morning. 

The Spinoff offices are in Morningside, Auckland, but we would consider a remote working arrangement for the right candidate.  

Responsibilities include:

  • Writing, editing, compiling and sending The Bulletin as an email and site lead five days a week at 7am. This includes all required elements.
  • Ensuring The Bulletin is reactive to the fast-changing current affairs and news environment. 
  • Working with editorial and audience teams to ensure all sections and elements are included each day, including key Spinoff stories, video and podcast content, and commercial and audience revenue placements. 
  • Being in charge of publishing our morning leads to social media and via app push notifications.
  • Functioning as early morning eyes and ears on the site and breaking news.
  • Working with The Spinoff editors to ensure strong lines of communication around news, breaking news events and Spinoff coverage.  
  • Working with The Spinoff’s head of audience to maximise The Bulletin’s role in growing audiences, building deeper engagement with them, and ensuring subscriber growth and open rates remain high.
  • Working with The Spinoff’s commercial department regarding sponsorship obligations. 
  • Working with other staff and heads of department to maintain The Bulletin’s position as a  vehicle for connecting people with The Spinoff’s work.
  • Managing feedback and correspondence with Bulletin readers on a weekly basis.
  • Pitching and completing standalone editorial pieces as capacity allows.  
  • Attending weekly all-hands and editorial meetings (Monday 9am-11.30am), contributing additional story ideas that may be written by someone else in the editorial team.
  • Participating in sharing news and information in relevant meetings and online channels. 

Skills and experience

Candidates should have a minimum of four years’ experience in media. This could be a position for an established editor or an experienced writer ready to take the next step in their career. There is also the opportunity to tailor the role to include other format outputs, for example podcast or video, depending on the interests and skills of the candidate. 

Applicants must have the right to live and work in Aotearoa to apply for this role.

Next steps

Please email your application to jobs@thespinoff.co.nz. All applications require a CV and a cover letter to be addressed to The Spinoff editor, Madeleine Chapman. 

Applications are due by March 29, 2024, but may close earlier if a suitable applicant is found.

But wait there's more!