Once a system is designed and put live, and data is collected and published, the focus of engagement should be on gaining feedback on the live system. This will ensure that the data that is disclosed can be effectively used to achieve the policy goals.
Any user-centred design process will naturally be limited by the amount of engagement and testing that is possible within the time and resource constraints of a particular implementation. It is not until a system is made live and all users have interacted with it that the quality of the system can be properly tested.
To ensure a good feedback loop, some basic approaches should be followed:
- Retain budget for further changes and modi!cations to the system to allow for changes based on user feedback. Earlier consultation with government agencies that influence budget decisions, and that bene!t from using the data, will increase buy-in for sustaining reform after the initial implementation
- Maintain user-research teams during the live phase to ensure you have the capacity to keep testing the system when it is live.
- Maintain a users-group who will feedback their experiences of using the system through short questionnaires, further interviews or focus groups
- To the greatest extent possible, ensure that the system for collection and publication has analytical capabilities built in to understand how users are interacting with it, where common problems are occurring and where users are terminating a transaction
A live digital service will always require testing, maintenance and improvement. Ensuring the responsible department and agency has a budget for this long term, or a clear plan to ensure budget is available for monitoring and improvement, will be essential to the sustainability of the work.
Civil society can play an active role in assessing early releases of data and driving future improvements, and experience from the UK has shown that convening groups of users to perform analysis on early data release can lead to the identification of easily solvable problems that increase the usability of data and the overall policy impact.
Following the release of UK beneficial ownership data in 2016, DataKind UK, OpenCorporates, Spend Network and OCCRP brought together thirty volunteer data scientists to analyse the initial data disclosed on the register. One key problem identified was that the data entry system allowed for free text entry for a number of fields. This had led to over 500 different ways of recording “British” as the nationality of the person filing. A simple fix – inserting a pre-formatted list of nationalities – provided the clear remedy and UK Companies House was able to amend the system quickly to include this change. This markedly improved the usability of the data.
Key outcomes from the ‘publish’ stage
- Published data is being used by people internal and external to governments in ways that enable policy impact
- There are mechanisms or systems for data users to feedback ideas to improve the system
- Statistics on user access to published data is being collected