Enabling Data-driven digital transformation in the public sector

Data is a major driving force in today’s modern economy and fuels innovation in businesses. We are seeing a fundamental shift in the use of data as organisations strive to utilise the power of cloud for data management while remaining compliant with legislative and sectoral rules. The use of data plays an increasing role in designing, delivering and transforming public services to improve outcomes and drive efficiencies within current financial constraints but there are still many organisations, particularly in the public sector, who have substantial amounts of data that is not being utilised to its full potential.

Data is Everywhere

It cannot be denied that data is everywhere and that volume is increasing exponentially. The “IDC & Seagate Data Age 2025” report predicts that the global data sphere will reach 175 Zettabytes by 2025 where 49% of data will be in the public cloud and 30% will need to be processed in real-time. Many organisations are not yet equipped to deal with the ever-increasing data volumes and still attempt to utilise outdated approaches to data quality and the management of data in the cloud. The UK government has stated that it believes “organisations spend between 10-30% of revenue on handling data quality issues”. This clearly shows a massive opportunity for cost saving on data quality alone.

This increase in data volume is happening at a time when increased compliance requirements are being enforced at nation state and industry governance levels. At the end of last year, Ticketmaster and Marriott were fined £1.25M and £18.4M respectively failing to protect customer data. The rough amount of all GDPR fines issued to date is currently over €1,28 billion. And this is not just a private sector problem, with public sector organisations responsible for over 54% of all data breach fines according to research by SMS works. In 2018, The Royal Borough of Kensington and Chelsea was fined £120,000 after it unlawfully identified 943 people who owned vacant properties in the borough. Just this year, the ICO fined the Conservative Party £10,000 for sending marketing emails to people who did not want to receive them.

In the organisations we interact with, we often see many hidden costs associated with data mismanagement that can have a massive impact on businesses:

• Expensive data handling costs - The same data is manually keyed into different systems.
• Excessive Storage - Data is retained long past its useful life either due to poor or no data archiving/destruction capabilities or the failure to use archiving/destruction capabilities properly.
• Lack of Insights - Data is difficult to extract and integrate across disparate systems meaning that organisations are missing out democratising data. This results in potentially critical data, that could be used to support business decisions, not being available to those who need it.
• Data decisions taken at a project / programme level - Data decisions being embedded within project, programme or service level decisions without consideration on how they affect the wider organisation over the lifespan of the data.

Data Success Stories in the Public Sector

While we often hear more about data management gone wrong, there are many public sector organisations managing to effectively utilise their data to improve services. A senior UK government official used open data to identify and eliminate significant duplicate spending, generating a massive saving of about £4 million. OpenSAFELY, a data analytics platform developed between the DataLab at the University of Oxford and a number of electronic health record providers working on behalf of NHS England, identified patients most at risk of dying from COVID-19 with significantly higher accuracy than other approaches. The Sussex Partnership Foundation Trust developed a data visualisation tool that helped clinicians to provide more effective treatments for their patients suffering from mental health issues.

Data as an Asset

The organisations we see who are utilising data effectively are those who see data as an organisational asset to be used when needed and securely disposed of when not. This will commonly culminate if key behaviours such as:

• Having a strategy for data, supported at the highest level of the organisation, to elevate it above a project/programme concern and ensure perspectives such as cost, security, sustainability, compliance etc. are in place before project/programme.
• Having clear contract conditions for 3rd party suppliers in respect of how data needs to be handled throughout the lifecycle including primary data and metadata.
• Using technology for data management including storage, processing, transformation etc. when you need it rather than trying to guess what will be required for multiple years resulting in vendor lock in and shoehorning of needs to fit technology rather than it being needs driven.
• Leveraging cloud technologies and cost models to allow for the use of just in time, just the right amount of powerful data processing for storage, transformation, sharing and visualisation.
• Driving continual improvement to move data from legacy environments, such as desktop databases, to powerful cloud based solutions allowing much more effective use of the data.
• Knowing (even if only at a high level) where data ingresses, egresses and is generated in the organisation as well as understanding where it is stored and processed. This makes compliance easier/cheaper and facilitates business transformation at pace.
• Driving to use summary data wherever possible for decision making as often the value is in the summarisation, not the raw underlying data, with summarised data easier to keep compliant and utilising less storage and processing.
• Driving use of data visualisation to make data more accessible and usable by the people that need it.

What’s Next?

If you are a senior manager responsible for overseeing multiple digital transformation programmes or services in the public sector then ask the hard questions of those delivering for you:

• How do you know what data will be stored/processed and how it is managed from ingress/generation to end of life?
• How do we know we can delete personally identifiable data when requested to do so and how do we know we still have a legal right to store/process it?
• What is the organisational strategy on data and how are my programmes and services helping to deliver to that strategy?
• How do I know all the things we need as an organisation and being done by the supply chain for my programmes and services?

If you are a manager responsible for the delivery of programmes or services then ask yourself:

• Do I know the data and how it flows through the system?
• How can this data, in summary form, support wider organisational decision making and how am I making this visible to decision makers?
• How will this data be sustainably managed throughout the lifecycle of its existence within the organisation (not just the project, programme or service)?
• Have the needs of the organisations strategy on data been absorbed into the functional and non-functional requirements of my programmes or services?

Your journey does not end here and much more must be done before you can truly say you are gaining maximum value from your data. However, by understanding what questions need to be asked and then encouraging those responsible for programme delivery to think about data as a fundamental part of the project is a good start. The phrase “Well begun is half done”, often attributed to Aristotle, remains a cornerstone for effective transformation with many changes derailed or stalled by decisions made in the early stages.