Unlock the Editor’s Digest for free
Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.
The British government demonstrated rare imagination and initiative when it held an international summit in November to discuss the existential risks posed by artificial intelligence. Tragically, successive British governments have shown a shameful failure to protect their citizens from more basic misfiring technology.
Take the case of Seema Misra, pregnant and manager of a sub-Post Office, who was in 2010 wrongfully jailed for 15 months on her son’s tenth birthday for theft and false accounting, flagged by Fujitsu’s flawed Horizon computer system. It took 11 years for her conviction, as well as those of 38 colleagues, to be quashed by the Court of Appeal. The judgment found that the Post Office’s “failures of investigation and disclosure were so egregious” as to make the prosecution of these sub-postmasters “an affront to the conscience of the court”.
Last week’s heart-rending ITV television drama shone a light on the one of the greatest miscarriages of justice in British history. It has rightly triggered public fury and forced the government into fresh action. To date, only 93 of the more than 700 people convicted in the affair have seen their convictions overturned.
Belatedly, the government announced this week it would propose primary legislation to exonerate and compensate those convicted. But it is only thanks to the courageous campaigning of the sub-postmasters themselves over two decades, as well as the cussed reporting on the affair by Computer Weekly, Private Eye and the writer Nick Wallis, that justice will now be done.
At the heart of the scandal lies the spectacular mismanagement of an IT automation project, which should serve as a warning to many other companies and government departments. Hailed at the time by Fujitsu as the biggest non-military IT contract in Europe, the more than £1bn Horizon system was launched by the Post Office in 1999-2000 to improve efficiency and combat fraud. By 2013 Horizon had been rolled out to more than 11,500 branches, processing some 6mn transactions a day. But complaints arose about its errors almost from the start.
As with all software, Horizon contained flaws. With good judgment and focused effort, most flaws should be routinely identified and corrected. There is a saying in the software industry, known as Linus’s law, that “given enough eyeballs, all bugs are shallow”. But that clearly does not apply when the companies running the software are wilfully blind. Horizon’s bugs appear to have been deliberately obscured, rather than exposed.
In a letter to a parliamentary select committee in 2020, Paula Vennells, chief executive of the Post Office from 2012 to 2019, said she was constantly assured by Fujitsu that the Horizon system was robust and as secure as “Fort Knox”. But faced with glaring evidence to the contrary from the sub-postmasters, Post Office management should have exercised far more critical scrutiny. Vennells’ defence also fails to explain why the Post Office pursued the accused sub-postmasters so aggressively for so long and frustrated their legal appeals.
In turn, the government failed to exercise effective oversight as the Post Office’s shareholder. Whitehall’s revolving door ensured that no responsible minister was in place long enough to get to grips with the issues. Ed Davey, now leader of the Liberal Democrats, was one of 17 postal affairs ministers to have held the role while the scandal unfolded. Davey now accuses the Post Office of a “conspiracy of lies”.
But that defence also wears thin.As Edward Henry, a barrister representing former sub-postmasters, told a public inquiry last year: “The government is like the owner of a dangerous dog mauling a defenceless child, saying, ‘Sorry, so sorry, but it has nothing to do with me’.”
It is encouraging that the government has finally taken responsibility for fixing the mess. The UK postal affairs minister Kevin Hollinrake told the BBC that almost two-thirds of the 3,500 people who were affected by the scandal have received full and final settlement at a cost of £148mn. The priority now is to exonerate and compensate the remaining victims and investigate whether managers at Fujitsu and the Post Office themselves broke the law.
One bitter lesson to be drawn from the affair is: listen to the humans on the frontline of technology, more than those in the back office. Software fails. Governance matters. This will become all the more important as AI systems increasingly intrude. But with so many lives shattered, it is hard to argue that the wronged sub-postmasters did not face existential risks of their own.