Serverless technology: the key to DWQR’s innovative cloud native data solution

DWQR background

Implementing complex regulations requires the need to verify and validate large datasets - a challenge not lost on the Drinking Water Quality Regulator (DWQR). For the past 15 years, its former legacy system was holding it back.

The previous solution presented significant barriers to further redevelopment and was inefficient, inflexible and had high operational costs.

Who is DWQR? 

The Drinking Water Quality Regulator (DWQR) ensures drinking water in Scotland is safe to drink. This is achieved by ensuring Scottish Water safeguards the quality of the public supply, through a process of inspections and monitoring. 

The challenge: transforming data management

DWQR's existing systems used outdated, unsupportable legacy technology at high operational and resourcing costs. The technology did not meet their organisational need for efficient data management; it also no longer kept pace with the demands of two core functional requirements:

1. Better Incident and Event Management to enable DWQR to provide rapid and efficient risk analysis, so it can identify the required next steps for water quality improvement.  
2. Enhanced data analysis and management for better reporting and regulatory submission of data to and from Scottish Water and Local Authorities that manage private water supplies.

To supply regulatory data to DWQR, data was received annually or monthly. The regulator needed to store, processes and analyse data more efficiently and in a timely manner to meet regulatory requirements and react promptly to any issues. DWQR had to make recommendations to Scottish Water using easily-accessible data.

The problem with the legacy data system was the regulator had to perform many manual operations that incurred lengthy time and high resource input. This approach involved a proliferation of spreadsheets, making the analysis and reporting process cumbersome and slow.

Ultimately, DWQR had to upgrade its legacy technology to transform how data was managed. 

The Exception approach - structured, collaborative and flexible

Exception already had a head start by having a successful working relationship with DWQR. Its work on a previous DWQR project involved analysing and reviewing systems and the infrastructure in order to move them to the cloud.

Exception used a combination of Agile Ceremonies and Waterfall in the Development and Delivery approach. This enabled better understanding of DWQR’s user needs to build cloud prototypes, developed through a Beta phase of the project.

The tech company’s Agile team worked closely with the internal DWQR team, providing prototypes via UX wireframes to bring the system to life at an early stage. It subsequently gathered important user feedback and testing to continually iterate the technology.

The new cloud native system enables Scottish Water and the 32 Scotland local authorities to upload data which is now automatically validated with no more manual intervention. 

The power of AWS cloud serverless computing

By deploying AWS cloud serverless computing, the customer also benefits from having access to serverless data analytics. To give users a better experience, the cloud technology incorporated the Scottish Government’s federated identity service, using a single sign on.

Exception’s Technical Team were also able to leverage AWS’s Management and Security Infrastructure to enable DWQR to take advantage of the low operating cost serverless technology, supported with a high degree of security.

The cloud-native technology aligns with many aspects of the Scottish Government’s Cloud First programme. This programme is intended to encourage and support public sector bodies towards using cloud services and realise the benefits of cloud computing. It also focusses on sharing both information and knowledge, which organisations can use when designing services in the cloud. Going cloud native using serverless technology was the way ahead for its data innovation. 

Neil Campbell, Exception's Head of Cloud and Cyber says:

“The technology has been truly transformational in innovating DWQR's data management, setting a new benchmark in operational efficiency and usability that other public sector organisations can adopt and benefit from,” he adds: “Instead of just taking on-premise workload and putting it on the cloud, the technology really leverages the power of core functions. These include analytics, identity login and serverless computing, which are only used and paid for on demand when needed, cutting significant running costs.”

From an environmental perspective, serverless enables DWQR to save energy when not using computing power. This will enable the customer to conserve energy on the strength of only needing to process data during a short period every year. Neil describes the old legacy technology as a “high watermark” system that was “over powerful” and too costly to process data, running for long periods.

Importantly, the new AWS Lambda serverless computing service processes data, more leaner and greener, in very a short window as and when it’s required, on demand.

This solution has transformed the operating costs, moving away from a large legacy Oracle solution to a low-cost pay-per-use cloud native application, running in just the £100s a month.

Valued delivered for DWQR:

• Greater functionality - a Virtual Data Lake is used for functions such as Data Analytics and Trend Analysis
• Web Interface - to enable users to access asset and event data supplied by Scottish Water
• Easy assessment of data - a better way to augment case management and add information to the system
• Improved communication – through event notifications and classification back to Scottish Water and local authorities – e.g. information requests for reports and categorisation of active incidents

Valued delivered for Scottish Water:

• Automatic rejection of data back to source if deemed invalid
• Clear and user-friendly UX/UI - front-end visible to Scottish Water and local authorities to upload data, confirming if data validation has been successful 

Why did DWQR choose Exception?

To reiterate, Exception already had a successful relationship and project delivery with DWQR. This is why the tech company was chosen as a natural fit for this data-led, cloud native project; and not least because its scope for the work was cost competitive.

Ready for next level innovation in your systems, products and services? 

Discover how we boost digital innovation with AWS Serverless Technology