Here is a picture of somebody’s hands dong some coding. May not in fact be Auckland hands.
Here is a picture of somebody’s hands dong some coding. May not in fact be Auckland hands.

OPINIONPoliticsMarch 8, 2016

Public IT projects are a disaster zone. Here’s some free advice

Here is a picture of somebody’s hands dong some coding. May not in fact be Auckland hands.
Here is a picture of somebody’s hands dong some coding. May not in fact be Auckland hands.

Reports of a blowout in the Auckland Council information technology budget are as dismal as they are unsurprising. Nigel McNie, a software developer who has been building complex, bespoke IT systems for many years, offers some free advice.

Eyes have been rolling up and down New Zealand at news of a Super City IT cost blowout, and the tediously predictable way in which it happened.

With $1.2 billion spent and counting, this is shaping up, notwithstanding council efforts to play it down, as one of the biggest overruns ever for a country all too familiar with them.

Public IT projects are a notoriously problematic. Here’s some advice for anyone planning one.

1. Half of IT projects costing more than $15m massively blow their budgets

“Massively” for software projects means an average 66% blow out. These project also overrun their time by 33% and deliver 17% less benefit than they were predicted to. Ouch. (Source: McKinsey-Oxford, 2012.)

Here is a picture of somebody's hands dong some coding. May not in fact be Auckland hands.
Here is a picture of somebody’s hands doing some sort of coding. May not in fact be Auckland hands.

2. 17% of IT projects go so badly that they threaten the very existence of the company

The same study provides us with this quite incredible statistic.

Such overruns match or surpass those experienced by black swans among complex construction projects such as tunnels and bridges. One large retailer started a $1.4 billion effort to modernise its IT systems, but the project was eventually abandoned. As the company fell behind its competitors, it initiated another project  —  a new system for supply-chain management  —  to the tune of $600 million. When that effort failed, too, the retailer had to file for bankruptcy.

The Auckland Council has spent $1.24 billion on IT since 2010. The IRD transformation project has over $1 billion allocated to it. Just fixing Novopay has cost around $50 million. INCIS (the Integrated National Crime Information System)  was budgeted at around $85 million. Do you sense a pattern here?

3. There is no such thing as an off-the-shelf solution for your problem

The Standard’s article on the Auckland Council IT blowout says that the former head of information services “threw out the existing software used for property, consents and LIM’s (Pathway), in favour of heavily customising the expensive off the shelf SAP package”.

Read that again. The strategy was to start with something off-the-shelf, and then … heavily customise it.

For those unfamiliar with software development, this is like trying to build a five-storey house by starting with a one-storey house, and then attempting to build upward. At some point, you realise that you need to go back and change the foundations to get results, but by then you’re three storeys up. It will either cost you big time, or you’ll have to start again, or you’ll have to live with it.

4. With your complexity, you need to employ, or contract for the long term, the people who will build the systems

The art of building a great IT system is to truly understand the problem you are solving. This is easiest if your IT team is either in house, or you have a long, good relationship with them.

The crux of it is this: At some point, somebody has to take the problem and explain it to the computer. You will never get this right the first time. Your best chance for success is to have the team close enough to the problem, and able to move fast enough, that they can clarify the explanation given to the computer before you run out of money.

This feedback loop works best when you have a team that understands both software development and your requirements. Think of the institutional knowledge that a public department has. How can it be easier for an outside IT team to understand the nuance, the rules, the laws, the exceptions, the changing environment within your organisation, when the project is so large?

5. The way to spend more than $10m on an IT project …

… is to first spend $100K, and if it’s working, slowly spend more.

Code is great. Creating and deleting it is easy. Creating the right code is hard, but creating code that tests out approaches quickly is very cheap. Open Source software can give you huge assistance to get started, and New Zealand now has a significant pool of IT talent that knows how to build projects of this size.

Consider how startups turn into successful businesses. They spend a small amount (by necessity), and if it works, they get to spend more, until they succeed. Xero is an IT project that would cost more than $10 million if built from the ground up, but they didn’t do it all at once. They started with a small slice of the problem, and proved it would work as a business first.

6. Enterprise providers will screw you over and leave you with the bag …

… even you don’t know the full extent of your requirements right now. A huge project to work out what they might be, right now, so that we can spend years building systems to match it, is a comically bad idea.

Software and hardware are not like skyscrapers. You can’t design one up front, build it, and have it sit there for 80 years. Your requirements will change. New governments pass laws. Hardware becomes obsolete every three years. Security is a huge problem requiring ongoing investment to fix. What’s more, if you design up front, you make it so much harder to adapt to an improved understanding of the situation as you build.

You must embrace uncertainty. The requirements will not be fully understood up front, but if you keep your projects small and fast, there is room to adjust them on the fly. The software development community even has entire models of work built around this concept (for example: agile). You would do well to use them.

7. Open source is not a panacea

It’s incredibly helpful, but you must be careful how you do it.

If your project gets too large, you get stuck with many of the same problems that a “customised off-the-shelf” closed-source solution has. In particular, your foundations will move, and you need to accommodate this.

If you’re not prepared to keep the WordPress you built on fully patched, if you don’t appreciate that sometimes an open source project will make backward-incompatible changes, or if you simply make too many changes yourself, you are nearly as doomed as if you used closed source. At least your failure will be cheaper.

Keeping your project small really helps. The software development industry uses open source software all the time at small scale, with great success. Don’t make the mistake of thinking you can build larger projects on its back.

8. A path forward: develop your own in-house IT expertise

You have so much complexity and nuance inside your operation. Your IT projects will last far longer than the first hardware you buy. They will even outlast you.

Build a team inside your organisation that understands IT, that can begin to understand your organisation. Let them build small projects. You can bring contractors in to provide extra speed or expertise, but their projects must also be small.

IT is critical. You won’t function without it. You’re IT-driven whether you like it or not. Own it.

Keep going!
header

PoliticsMarch 8, 2016

Tea, pee and pecuniary gains: Amid the clowns at the trade deal roadshow

header

The TPP roadshow kicked off in Auckland yesterday with luxury muffins and pointed questions at an inner city Auckland hotel. Former Herald editor-in-chief Tim Murphy takes in the explanatory slides and the giggling infiltrators.

The real TPP roadshow occurred on intersections and motorway on-ramps about four weeks ago. There was noise, blood and guts. The public noticed.

Today’s is the sanitised version, behind four security checks in the fading charm of the Rendezvous Hotel on Mayoral Drive. Some of the same protesters are here, on the street corner and on the hotel driveway looking for the CR number plate of a Crown car carrying Trade Negotiations Minister Todd McClay.

They’re too late. He’s already inside, talking through the day’s runsheet with MC Sean Plunket.

It’s all very low key. It’s muggy and the week hasn’t started yet.  

One placard holder, a teacher, has slipped away from school and down the road to make that point: “Let’s consult widely with the public. How about on a Monday morning when everyone is at work? Yeah right!”

Dr Fiona McLean
The author’s cousin, Dr Fiona McLean (photo: Tim Murphy)

Standing with her is, as it happens, my cousin, Dr Fiona McLean (fitting her anti-TPP duties in before a family one and her GP job) and waving a sign saying: “TPPA ‘Spin’ Show NOT Road Show”. Fiona was on an intersection a few weeks ago. She asks if I think the government has ‘”bought off the media” on this issue. I think not. For the most part the TPP is a turn-off all on its own.

Two Canadian joggers, from Nova Scotia going by their shirts, stop to chat. “Bless your hearts for being out here,” the guy says, adding that the TPP serves big business and a country “that shall not be named”.

Inside the hotel, about 150 people assemble for the morning session in which McClay and the chief New Zealand negotiator, David Walker, will speak and then take questions. It’s a mixed group – bureaucrats, Penny Bright, real estate guys, accountants, a timber man and several smiling young people.

When McClay, who seemed to come out of nowhere to replace Tim Groser as our trade negotiations minister, ends his broad brush pep talk someone gives out a little “woo hoo”. Who would have thought? Todd, son of the Bolger-era National MP Roger McClay and assumed to be a bit goofy like his Dad, getting a cheer at his first roadshow.

As Walker gets up to speak, he, too, gets a little yelp of support. Heads turn. Tall, bearded and grey-suited trade negotiators do not win yelps. Undeterred, Walker moves into his work, outlining the 12 nations of the TPP, keen to clear up what TPP “is, and what it is not”. He traces on a graph with a series of growing circles the history of the trade deal, starting as a CEP with Singapore, changing into the P4 with two others, and finally the TPP.

shonkey
(photo: Tim Murphy)

To prick his big TPP bubble, though, all of a sudden there comes a weird little giggling from the audience. The kind of suppressed laughter of someone who can’t stop or has some kind of condition. A second person starts laughing as Walker talks Investor State Dispute Resolution. Then a third.

Infiltrators.  

McClay and Walker, security and event organisers all play it cool. The slides continue. So do the sniggers. Soon, though their laughter morphs into wigs, red noses, bubble blowing and then balloons inflate and pop.  

Bang. Up shoots MC Plunket, calling for a vote on whether the clowns should stay.

They ignore the democratic will and start dancing off anyway. “This is a joke,” says a lone cloth sign which had somehow made it through the security checkpoints. Out they go, trespass letters in hand.

McClay must be pleased with the optics. No imagery of a minister requiring the removal of a passionate, peaceful young idealist. That’s what you pay the MC the big bucks for.

One of the giggling infiltrators dances out  (photo: Tim Murphy)
One of the giggling infiltrators dances out (photo: Tim Murphy)

Later the people sitting either side of one of the gigglers reckoned she was obvious from early on: “Shall we say she hadn’t showered,” said one. “And we should’ve known from her tattoos and that brick of a phone,” said another.

Walker goes on through his show. Professor Jane Kelsey, the leading academic critic of the TPP, didn’t expect much new in the presentations. She’s been sitting reading a document, presumably for the Treaty of Waitangi case over the TPP, for which she is an expert witness in Wellington next week.

Question time, though, is what has brought this crowd. Three rows form and people are quick to queue to quiz McClay and Walker. The first man is cross about the lack of consultation. “I’ll leave it to David to go through some of the detail,” the minister offers. But people had been consulted and consulted; stories had appeared in the press. “Unless you are suggesting we write to every single New Zealander, we will end up having to use the media, I suppose.”

Questioner two, in export services, doubts the Foreign Affairs worksheet’s expected dollar savings for the fruit industry. “Will an apple a day keep the spindoctor away?” he asks.  

Auckland Mayoral candidate, Her Warship Penny Bright, presses McClay on the term “free trade” when tariffs are indeed not eliminated on dairy. His answer is an oldie but a goodie: “It is a free trade agreement in that it frees up trade.”

Protestor 'Raven' (photo: Tim Murphy)
Protestor ‘Raven’ (photo: Tim Murphy)

The mix of issues raised, and the backgrounds of those present, is encouragingly diverse. A London School of Economics and Australian National University academic, Dr George Barker, surprises McClay by claiming the government is wrong in thinking copyright changes will cost the country $55m. Instead it will be a net benefit to the country.

Jane Kelsey gets to ask two long, detailed questions and it seems pretty clear McClay didn’t recognise the term “certification process” – a means by which she thinks the Americans can make us do even more trade dealing at the end of the whole thing if they’re not satisfied with TPP as it is.

McClay assures the roadshow a select committee will still take TPP submissions, and it’s clear here that no one is going to shut down questions. He leaves, though, after the first segment. As does Kelsey. They had other things to do and missed the bounteous Rendezvous muffins and tea.

Barry Coates, ex Oxfam and now of Sustainability Initiatives Aotearoa  (photo: Tim Murphy)
Barry Coates, ex Oxfam and now of Sustainability Initiatives Aotearoa (photo: Tim Murphy)

Much later, Barry Coates, once of Oxfam and now with Sustainability Initiatives Aotearoa, gets good air time to point out that the $2.7b a year annual gains from TPP by 2030 equate to what will then be 1% of Gross National Product. Because the $2.7b figure, which is not great now, will be even smaller by then as our economy will have grown by 47 per cent in the next 14 years.

“Who benefits?” Coates asks.  

Walker responds: “The answer to who benefits is whoever seeks to take advantage of the opportunities this agreement will create for investments, services, and in tariff and non-tariff ways.”

He says the $2.7b figure is a deliberate underestimate by officials not wanting to oversell the benefits. In any case, “an additional one per cent of GDP, I would argue, is not to be sneezed at. It can buy quite a lot of prosperity. One per cent positive is versus a negative if we do not join.”

Still the questioners queue.

An Auckland University researcher tries to speed things up when she gets to the mic. “I’ve got two questions but I’ve really got to pee so they’ll be quick.”  

They were, too, and tricky: “Who are the Treaty partners referred to in the presentation?” and an observation that the “language in this agreement has no basis in Tikanga Māori – the neoliberal agenda it promotes is in direct contradiction to what it means to be Māori.”

Then she was off to pee.

Topics abound: overlapping trade deals; Monsanto’s lobbying in the States; software patents; a guy with a man bun who opened with “we take a dim view of everything”; worries about vineyards being bought out by Australians, Americans and Chileans; a query on whether the TPP is a snub to China and finally a man wanting to know about technology innovation.

Finally Plunket had to cut him off. “We’re already 10 minutes over time,” he sighed.  

The questioner sighed, too.

“But we’ve taken five years to get to this point!”

The next roadshow is in Christchurch on Friday. Get along if you’d like a lesson in certification processes. Or a muffin. Or a laugh.