Hardworking public servants who respond to official information requests live in fear of creating a media storm. It’s time to consider introducing an independent agency earlier in the process, suggests W Ingles*.

For journalists, the Official Information Act is both a vital tool and a constant source of frustration. It is one of the strongest “right to know” laws in the world, and in theory demands a level of openness that would make most governments blush. In reality, however, those using the OIA often face delays, obfuscations, and inexplicable redactions.

Of course, many journalists (and others) have written about this before, pointing the finger of blame at a culture of secrecy in the public service, political interference, the act’s lack enforcement teeth, and even at the proliferation of media teams and public relations specialists.

I’ve spent several years working opposite these journalists, in teams responsible for responding to OIA requests in large government agencies. From this experience it’s clear to me that while all of these are reasonable concerns, they’re really symptoms of something more fundamentally wrong with the OIA, and with the very nature of my job: a conflict of interest lies at the very heart of the act.

In most large agencies, requests for information go to a specialised team responsible for OIAs and a couple of similar functions (this is where I worked). This team will then forward the request to whichever part of the agency is responsible for the subject of the request, such as a policy or data team.

The OIA team and the subject experts will then work together to find the information requested, assess the information for the risks of releasing it, and write a short letter to explain what’s provided and, if applicable, why some information is being redacted. Finally, the response is signed by a manager and sent out.

For smaller agencies, the process is much the same, although they may not have a specialised OIA team to assist the subject experts.

So far so good, right?

The problem is that none of these people, from the OIA team to the subject experts to the manager signing it out, can ever be truly impartial. Public servants have learned, through painful experience, what can happen if an OIA goes out with something embarrassing in it that ends up in the media.

Many of us have seen important work grind to a halt and friends working 12-hour days trying to keep on top of an influx of letters from the public, requests from ministers, and more and more information requests from journalists to fuel a media storm. Other times, entire pieces of work have been halted altogether, evidence ignored and solutions unimplemented as misleading and clickbaity media stories make it too politically unpalatable. We’ve seen our own ministers publicly criticise public servants to deflect blame. Occasionally, we’ve even seen respected colleagues have their careers derailed as they’re made to take the fall.

Unsurprisingly, these experiences haunt the OIA process, and ensure that everyone involved has a potential incentive to keep embarrassing information under wraps. As a result, agencies have become very good at stretching the OIA’s withholding grounds to their limits.

A rule that aims to give ministers and officials a chance to consider advice before being subject to public pressure, for example, is now seen as a blanket rule to hide anything that might be thought about in the future, even years ahead. The lacklustre outcomes of a contract, or eye-watering fees of a consultant, meanwhile, are slapped with a “commercial sensitivity” redaction. If all else fails, awkward information is declared free and frank advice, and safely kept out of the public eye.

After every bit of media criticism, agencies also have a habit of adding more layers of sign-out. This means that before a response can be sent, it needs the go-ahead from more and more people, including managers, general managers, deputy chief executives, media teams, chief executives’ offices, and ministers’ offices. This not only adds to the delays that plague the OIA, but also gives ample opportunity for someone to lose their nerve and decide they’d rather just redact that awkward sentence than let it go out.

Giving the responsibility for administering the OIA to people who tend to see the OIA as a nuisance and a threat inevitably leads to bad practice. As well as the over-redacting and delays described above, there’s plenty of room for other “quirks” to develop.

I’ve heard that one former minister’s office, for example, decided that aide-memoires, a type of short report, are exempt from the OIA. This means that these documents weren’t simply withheld, or redacted in full under the act, but were excluded without mention. The responses to requests for lists of documents (a tactic used by some journalists to identify interesting documents worth requesting) therefore seemed to show that the minister hadn’t received any aide-memoires. Unless you knew that these documents existed, you would never even know anything was missing.

Another minister’s office apparently at one point decided that the OIA only applied to formal documents and excluded emails. Not only are emails included under the act, but so is official information of all kinds, right down to people’s own recollections of events.

All of this is a direct result of giving the responsibility for answering OIA requests to the people linked to the subjects of those requests. Just as we don’t give the prosecutor the job of judge in court, or let mining executives act as their own labour inspectors, we can’t expect agencies to apply the OIA fairly to themselves.

The law might set out clear rules, but there’s always room for interpretation, and we all recognise that this needs to be done by someone who is unequivocally impartial.

I want to be clear that this isn’t the fault of malicious bureaucrats and their sinister authoritarian schemes. In my experience, the public sector is full of people who truly care about their work. They often know they could earn more in the private sector, but they believe in what they do, and are motivated to help our people. But we also know that bias and self-interest is a subtle thing, and it doesn’t take a moustache-twirling villain to let your own interests affect your decision-making.

If we truly want to restore the promise of the OIA, and almost as importantly, the public’s trust in the OIA, we need to address this. The Ombudsman offers some oversight, by reviewing complaints, but they can’t review every decision, and nor should they have to. Instead, decisions should be made from the start by an independent body.

This is, of course, much easier said than done. It’s the agencies that hold the information being requested. The agencies also have all the knowledge about what the information means, what else is happening, and any other context that might be needed to make decisions. It’s clear that we can’t cut the agencies out altogether.

But it isn’t impossible. Our systems of justice, mediation, oversight, and other decision-making have countless examples of ways to get information from one party to make an independent decision. A new independent body could maintain relationships with the experts, and with ministers, while staying impartial, and keeping the trust of the public.

This will, of course, take political will. No government wants to give up the power to influence what goes out. But if Robert Muldoon, better known for breaching the 1688 Bill of Rights and calling an election while sloshed, could be compelled to pass the original Act, surely we can today hope for better.

* W Ingles is a pseudonym.