Build or buy: commercial off-the-shelf systems in small island states

Every government that needs a new tax, benefits or permits system faces the same decision. Build something bespoke, or buy a system that already exists and configure it.

The arguments are usually made in terms of cost, and cost is the least reliable way to decide. Both options are quoted before anyone knows what the real number will be, and both quotes are usually wrong.

This article sets out what the evidence on large system projects actually shows, why the decisive constraint in a small state is rarely the price, and what each option costs beyond the invoice.

What the failure data actually says

The strongest evidence in this area is not about build versus buy. It is about scale.

Research by McKinsey with the BT Centre for Major Programme Management at the University of Oxford examined more than 5,400 IT projects with initial budgets above 15 million dollars. On average those projects ran 45 per cent over budget and 7 per cent over time, while delivering 56 per cent less value than predicted.[1]

The tail is worse than the average. Around 17 per cent of large IT projects went so badly that they threatened the organisation’s existence, with cost overruns between 200 and 400 per cent.[1]

Findings like these are often quoted as though they settle the build versus buy question. They do not. They describe large software projects in general, whatever the procurement route.

What they do establish is that the risk on a large system project is high either way, and that anything which reduces scope, shortens timelines or removes unknowns is worth paying for.

The case that gets cited most often, and what it actually shows

The NHS National Programme for IT is the example everyone reaches for. It is worth using accurately.

Launched in England in 2002 to create integrated electronic patient records, the programme was dismantled in 2011 following a review by the Major Projects Authority. The Public Accounts Committee, drawing on National Audit Office findings, reported that an estimated 9.8 billion pounds had been spent without delivering key benefits.[2]

The detail usually left out is that NPfIT was not an in-house build. It was delivered by major commercial suppliers under large contracts, and some components were delivered and remained in use.

So the lesson is not that building fails and buying succeeds. The account points at a centralised programme that did not sufficiently understand or carry the people who had to use it, and that treated a change in working practice as a technology delivery.[2]

That failure mode is available under either procurement route.

The constraint that usually decides it in a small state

Cost comparisons dominate the debate and rarely decide the outcome. The binding constraint in most small administrations is people.

An IMF review of tax administration across the Caribbean found that eight of the twenty administrations examined operated with fewer than one hundred staff in total, across every tax, every function and every process.[3]

A permanent software development capability cannot be carved out of that. It is not primarily a budget question. There is no headcount to allocate, and competition for the specialists involved is not local.

This matters after go-live more than before it. A bespoke system requires people who understand it to remain available for its entire life, which for a tax or benefits system is measured in decades. When two or three of them leave, the administration can find itself unable to change its own system.

Buying does not remove that dependency. It moves it from individuals inside the administration to an organisation outside it, which is a different risk requiring different management.

What buying an existing system actually gives you

Four things.

The system exists before you pay for it. Scope, function and behaviour can be inspected rather than described, which is a different kind of assurance from a specification.

Other people’s failures have already been found. A system running across multiple administrations has encountered edge cases yours has not reached yet, and those have been fixed on someone else’s timeline.

Regulatory change is absorbed across users. Tax and benefits legislation changes constantly. Under a bespoke arrangement the administration commissions and pays for every amendment. Under a shared product the cost of common changes is spread.

Implementation is configuration rather than construction. That shortens timelines, and the failure data above is unambiguous that longer projects fail more often.

What buying actually costs you

There are four costs on the other side of that, and they are rarely set out alongside the benefits.

Fit gaps are real. No product matches national legislation exactly. Every gap resolves into one of three outcomes: configure around it, pay for a change, or change the working practice. All three have a cost and the third is often the largest.

Customisation can erase the saving. A heavily modified product acquires the maintenance profile of a bespoke system while retaining licence costs. This is the most common way a buy decision quietly becomes a build decision.

You are dependent on a supplier for a long time. A tax system procured today may still be running in fifteen years. That makes the supplier’s viability, ownership stability and continuity part of the decision rather than a footnote to it.

Exit is expensive and rarely tested. Data ownership, export formats and transition terms are cheap to negotiate before signature and very expensive to negotiate afterwards.

The questions that actually settle it

Neither route carries less risk on its own. In practice what separates a decision that holds from one that unravels is the quality of the questions asked before signature. The following are not a standard, but they are the ones that tend to expose the difference between suppliers.

Is our legislation configured or coded? Configured rules can be changed by the administration when the law changes. Coded rules require a development cycle and a supplier. This single distinction determines the cost of every future amendment.

Who owns the data, and in what format can we take it out? The answer should be in writing and specific about format, not just about principle.

What happens if the supplier is acquired, restructured or fails? Continuity of ownership over a fifteen-year horizon is a fair question and a revealing one.

What is the fee structure, in full? Licence, implementation, support, and the cost of a legislative change. If any of those four is unavailable before signature, it will not become cheaper afterwards.

Which administrations are running this today, and may we speak to them directly? A conversation without the supplier present, rather than a reference document.

On what basis are the stated results measured? A supplier claiming a revenue improvement should be able to say who measured it, over what period, and against what baseline.

Every one of those questions applies equally to a bespoke build, with the supplier replaced by the internal team.

What this means for small administrations

Three conclusions follow from the evidence rather than from the sales case.

Scope is the variable most under your control. The failure data is about large projects. Reducing what a project has to achieve in its first phase does more for its odds than any procurement decision.

The decisive question is who maintains this in year eight. Not who builds it in year one. Both routes create a dependency, and the responsible comparison is between two named dependencies rather than between a dependency and an imagined self-sufficiency.

Ask suppliers the questions above, including the ones that are inconvenient for them. A supplier who cannot answer them before signature will not answer them afterwards either.

For most small administrations, buying an existing system is the more practical route, for reasons that have more to do with capacity and continuity than with headline cost. That case does not need overstating, and it is weakened rather than strengthened by claims that cannot be traced to a source.