Whitehall on trials

Appendix
Home Secretary, somewhat late in the day, herewith the appendix promised in my open letter to you dated 8 November 2011.

The fourth enquiry – into the efficacy of the biometrics used by UKBA and the Home Office generally – has at its disposal a lot of evidence in the form of correspondence with the Home Office, the UK Border Agency, the Identity & Passport Service, the Home Office Scientific Development Branch, the Information Commissioner and the Information Rights Tribunal available here, here, here and here. The enquiry may also be assisted by reading the reports on biometrics here, here and here.

The hypothesis that the enquiry needs to test is that:

For 10 years the Home Office have been investing public money unwisely in projects which depend for their success on mass consumer biometrics technology being reliable – it isn't.

There is a lot of respectable evidence suggesting that the technology chosen by the Home Office is not reliable and no respectable evidence suggesting that it is. The enquiry may conclude that for 10 years public money has been wasted.

In the matter of Brodie Clark, the implication is that the Home Office's chosen biometrics cannot enhance border security. It is therefore inept to fire the man for not using it.

The further implication is that you, Home Secretary, have been lured by your officials into talking nonsense about strengthening and relaxing border controls – to the extent that those controls depend on biometrics, the technology available cannot enhance border security and can only weaken it by diverting UKBA staff into useless procedures.

There is so much evidence available that the enquiry may welcome some guidance on the best routes to take as they travel through it. It is suggested that the first route they take should be as shown in the timeline below. It's hard to stay awake as you read through it but there is a dénouement to look forward to, so please persevere.

Facial recognition biometrics and smart gates at UK airports
August 2008	Back in August 2008, the UK Border Agency started a trial of so-called "smart gates" at Manchester Airport. UKBA issued a press release about the trial here. Don't bother clicking on the link, the press release has been deleted. With smart gates, travellers walk into one end of a booth, stick their ePassport (electronic passport) in a reader and stand in front of a camera. Face recognition software compares the face on camera with the "template" stored on a chip in the ePassport. If the two images match, according to the computerised threshold tests, then the exit gate opens and that's the traveller done, successfully through passport control. No UKBA passport control staff needed. They can be laid off. There will be considerable cost savings and – the matching process having been performed by computers – it will be more reliable than mere human beings, the security of the border will have been enhanced. That was the idea. In the event, there was some adverse coverage of the equipment in the Daily Telegraph and on the BBC News website: 19 August 2008 – Machines to scan faces of travellers at UK airports 19 August 2008 – Passengers test new face scanners 4 October 2008 – Security fear over airport face scanners 5 April 2009 – Airport face scanners 'cannot tell the difference between Osama bin Laden and Winona Ryder' But that was just unionised UKBA staff moaning about losing their jobs. Wasn't it?
So far so simple. UKBA, playing it by the book, have got this new equipment that takes advantage of the facilities offered by ePassports. Does the equipment work? They don't know in advance. So they conduct a trial. Depending on the result of the trial, either the idea can be dropped, because the equipment doesn't work, or it works well and UKBA can start to deploy these smart gates at airports elsewhere.
24 February 2009	Six months after the start of the Manchester Airport trial, UKBA announced a 10-point delivery plan, which comprised 10 "pledges". Pledge no.7 was, by August 2009, to "have completed delivery of new facial recognition technology in 10 terminals, giving British passengers a faster, secure route through the border". (UKBA's 10-point delivery plan used to be available here, on the bia.homeoffice.gov.uk domain. Don't bother clicking. The domain has long since disappeared. There is what looks like an accurate copy available here down at the bottom of the page.)
Presumably the Manchester Airport trial must have been a success. Presumably the equipment was found to work, presumably it was established that it was a wise investment of public money to deploy smart gates equipment at 10 airport terminals around the country, and presumably it was accurate to assure the public that their experience of automated passport control would be "faster" and "secure".
16 April 2009	Always worth checking these things. Someone wrote to Sir David Normington, permanent secretary at the Home Office, reminding him of the uninterrupted history of failure of biometrics based on facial recognition, asserting that the public would be sceptical about smart gates, and saying: I suggest that the way to overcome that scepticism is to place the matter in the hands of the Office of National Statistics. The use of mass consumer biometrics in public services, I suggest, should be based on official statistics. If rigorous academic evaluation suggests that mass consumer biometrics have a part to play, well and good. If not, then don't let's waste our time and money on them. There was no answer from Sir David.
26 June 2009	But a couple of months later, an answer came through from Brodie Clark, Head of the Border Force at UKBA. "Your letter has been passed to me to respond", said Mr Clark, and: UKBA commenced testing our Automated Clearance System (ACS) at Manchester and Stansted in August and December last year, to assess the accuracy and reliability of the technology. The Home Secretary’s pledge to introduce gates at a total of 10 UK airport terminals by August, includes the two current sites at Manchester and Stansted. It will provide a further opportunity to test the technology on larger numbers of passengers, across a broader range of locations. It also means that the gates will be available to British and EEA citizens throughout the busy summer holiday period. Asked to confirm that smart gates would be "faster" and "secure", was Mr Clark, on behalf of Sir David, going to try to get away with saying only that they were "available"? Asked to confirm that the Manchester Airport trial had been a success, was he going to say simply that the technology would benefit from further testing? No. Mr Clark adds: The test’s findings demonstrated considerable improvement in this field [facial recognition], and confirmed that the technology could be applied successfully in a one-to-one (verification) mode* and We recognise that the vast majority of the travelling public are legitimate, law-abiding passengers and believe that the gates will deliver an improved service† to our customers whilst allowing us to deploy our staff intelligently to areas of greater risk. ---------- * Some of us harbour the suspicion that Mr Clark had to be leant on to write that. † As we now know, the only way to deliver an improved service is to abandon use of the smart gates.
3 February 2010	The history of biometrics based on facial recognition really is a history of failure. If UKBA now had reliable facial recognition equipment, then some sort of a technology revolution must have taken place. Someone, in a letter dated 4 August 2009, asked Brodie Clark to publish the revolutionary Manchester Airport trial results. The same request was made of his boss, Lin Homer, chief executive of UKBA, in a letter dated 8 August 2009. The trial results were not published then and still haven't been published. There is still no respectable evidence in the public domain that UKBA's facial recognition technology works. On 3 February 2010, Lin Homer wrote: UKBA is currently trialling the use of automated gates using facial recognition technology at 10 sites across the UK ... The technology used has proved reliable within the operational environment ... and Evaluation of Manchester gave us enough confidence to proceed to expand the trial.
23 February 2010	Lin Homer kindly arranged a meeting, which took place at the Home Office on 23 February 2010. The meeting was attended by someone and by Karen Kyle (UK Border Agency), Marek Rejman-Greene (Home Office Scientific Development Branch), Alex Lahood (UK Border Agency), Henry Bloomfield (Identity & Passport Service), and Mike Franklin (UK Border Agency). No useful information about the reliability of UKBA's facial recognition technology was imparted at the meeting. In his informal minutes of the meeting, Alex Lahood wrote: Marek Rejman-Greene explained that we are under no illusion that the systems are 100% accurate but that there is adequate evidence/information about the level of performance to warrant embarking on a trial. A year after the 10 "pledges" of the UKBA delivery plan assured the public that the technology works, Lin Homer and Alex Lahood are still talking about trials. Why?
It is normal to publish the results of trials. It is suspicious when trial results are not published. The efficacy of biometrics has been asserted ever since 9/11, over 10 years ago. Public money has been spent throughout that period and continues to be spent on projects which depend for their success on biometrics being reliable. Not once have the Home Office supplied any trial results proving that they are investing public money wisely. For all we the public know, our money is being wasted on technology that doesn't work. If it doesn't work, using that technology cannot improve the security of the UK border. In which case the Home Secretary and the Shadow Home Secretary are talking nonsense when they say that failing to do biometric checks impairs the security of the border. And it is nonsensical to pillory Brodie Clark and force him out of UKBA for not using technology that doesn't work.
5-7 May 2010	By this stage we have had it confirmed to us by UKBA press releases, newspaper articles, letters from Brodie Clark and Lin Homer and the informal minutes produced by Alex Lahood that UKBA conducted trials of facial recognition technology at Manchester Airport. Mr John Vine CBE QPM is the Independent Chief Inspector of the UK Border Agency. He conducted an inspection of Manchester Airport between 5 and 7 May 2010. In his report, Mr Vine lists a number of problems with the smart gates in use there. And then, at para.5.29, he writes: We could find no overall plan to evaluate the success or otherwise of the facial recognition gates at Manchester Airport and would urge the Agency to do so [as] soon as possible.
It is hard to believe that the Home Office have been wasting public money on biometrics. But if we must entertain that thought, must we also make room for the thought that the Home Office haven't even been conducting the trials they keep talking about? How else are we to understand the Independent Chief Inspector's words, "We could find no overall plan to evaluate the success or otherwise of the facial recognition gates at Manchester Airport"?