These user satisfaction figures are taken from the GOV.UK Verify (RIP) dashboard on the Government Digital Service's GOV.UK performance platform. They are based on 11,687 responses to the ease of use question, 11,623 to the security question and 11,552 on "identity providers".
DMossEsq and other critics can carp all they like. They're wrong, GDS may say. Just look at those user satisfaction ratings. Over 80% of people are satisfied with GOV.UK Verify (RIP) or very satisfied with it. That's what counts. And "counts" is the right word. We're dealing with numbers here. And you can't argue with numbers ...
... or can you?
The user satisfaction statistics haven't been updated on the GOV.UK Verify (RIP) dashboard since 19 September 2016. There's no point guessing why but they're two months out of date.
As at 19 September 2016 847,433 GOV.UK Verify (RIP) accounts had been created and they had been used 881,914 times. That means that 1,729,347 identity verification transactions had taken place. Which means that the user satisfaction ratings are based on responses to about 0.68% of transactions.
"84.5% of 0.68% of people feel secure registering with the system" doesn't make for a knock-down cogent conclusion in favour of GOV.UK Verify (RIP).
Do the views of 0.68% of the population indicate the views of the other 99.32%? We have learned in the UK to be sceptical of polling results like this. The pollsters got the 2014 Scottish independence referendum wrong and the 2015 general election and the 2016 EU referendum.
They still get their forecasts wrong but at least the pollsters have become very careful over the decades about the wording of survey questions and the way in which they are asked. GDS don't have that professional hinterland ...
... and it shows. GDS ask how secure people feel, registering with GOV.UK Verify (RIP) – people may feel secure and yet not be. People may only be comfortable with using "identity providers" because they are misguided.
What we're left with is a question. How useful are GDS's user satisfaction statistics for GOV.UK Verify (RIP)? And we don't know the answer.
It's not just GOV.UK Verify (RIP). HMRC were talking about their on-line Personal Tax Account service the other day (PTA): "It’s still less than 12 months since the PTA was launched but, with 6.7m users and a customer satisfaction rating of 77%, it’s already transforming the way our customers can deal with us".
What does "a customer satisfaction rating of 77%" mean? We don't know. Not least because there is no PTA dashboard on GDS's performance platform. That's a serious gap in coverage.
HMRC have plenty of figures on the performance platform for other services of theirs. The figures haven't been updated for over a year, since September 2015. That's another serious gap in the coverage. A gap that the ONS, for example, or data.gov.uk with their years of expertise would not condone.
GOV.UK Verify (RIP) is ignored by most of PTA's 6.7 million users. And by the tens of millions of users of HMRC's other on-line services with billions of transactions to their name every year. They prefer to use the alternative Government Gateway.
That's the system the UK depends on to raise revenue to fund public services. What are the user satisfaction measurements for the Government Gateway? How many accounts are there? How much revenue has been raised through the Government Gateway over the course of its 15 years of operation to date? How much does it cost to operate the Government Gateway? We don't know. Once again, there's no dashboard for it on GDS's performance platform.
We don't know how much it costs to operate GOV.UK Verify (RIP) either. Another serious gap in the coverage of the performance platform.
We used to know that only about 70% of people who make the attempt can create a GOV.UK Verify (RIP) account. The target was – and still is – 90%. But we don't know how much progress GOV.UK Verify (RIP) is making towards that target because GDS stopped reporting the figures, saying that they didn't tell us much. Yes they did.
Missing dashboards, missing data, suppressed data, unhelpful questions about feeling comfortable, ..., GDS's performance platform is not a good advertisement for data analytics.
GDS still make predictions about the account creation success rate: "The data we’ve gathered shows that if activity from [social media] accounts could be used for activity history, GOV.UK Verify [RIP]’s demographic coverage of the adult population overall could increase by 9%, and for the 16-25 demographic could see a potential increase of up to 38%".
But who believes that now? Probably not even GDS.
It was never clear why Mike Bracken, executive director at the time of GDS, was appointed the government's chief data officer back in March 2015. And it still isn't clear why data analytics is supposed to be one of GDS's specialities. The data simply doesn't support that claim.
Mr Bracken was ejected from Whitehall in September 2015 having made no progress on GDS's data skills in the intervening six months. His successor as executive director, Stephen Foreshew-Cain, lasted 10 months, ditto. Since 1 August 2016 GDS has had a director general, Kevin Cunnington, who said hello on 4 August 2016 and also:
It is not clear why GDS has this mandate. Just because GDS know how to design a pretty front-end system it doesn't follow that they have the first idea how to analyse the data maintained by the back end.
I want to strengthen and accelerate the pace of change. I’ve read many times about the end of GDS, but it has always come back stronger than before. I want to tackle one thing head on: GDS will not be broken up. We remain part of the Cabinet Office with a clear mandate to lead digital, technology and data across government.
GDS and data analytics? An odd couple.
No comments:
Post a Comment