North West Boroughs Healthcare NHS Foundation Trust
The secure and timely data migration of over 2.6 million records
North West Boroughs Healthcare NHS Foundation Trust (NWBH) is a large NHS Trust located in the North West of England, supporting a population of more than 3.5 million people within Cheshire, Merseyside and Greater Manchester.
The Challenge
One common challenge facing large NHS Trust’s across the UK is the management of clinical records for all patients under their care, especially with no uniform solution across the NHS.
In the case of NWBH, a legacy clinical system which had served the Trust well for years, was no longer fit for purpose. The majority of the structured clinical records data had been migrated across to the Trust’s newly procured Rio system, however the remaining challenge was to migrate the unstructured notes data.
Further challenge was added as the legacy system servers were no longer supported, deemed end of life and had reached a point where failure was likely, creating a risk of clinical data loss.
The task of extracting data from their old clinical system was made more difficult as the Trust was racing against the clock. It was essential to complete the data extraction of 2.6 million individual records on time, without putting strain on an unstable system infrastructure.
Each record could consist of unstructured data and documents, which further increased the complexity of the task. The records included approximately 300 different types of case notes and assessment forms for around 30,000 patients. To complete this task manually would take around 1-2 minutes per form.
NWBH had experimented with extracting the data using an in-house method of SQL queries, however due to the varying format of the documents, they had mixed success. The Trust was unable to extract headings from the forms and without this, the data would have been un-usable.
The Solution
A neighbouring NHS Trust recommended NDL’s low code Robotic Process Automation (RPA) platform, Automate. It was established that Automate could provide the answer to NWBH’s migration difficulties.
Automate allowed NWBH to configure software bots to automate the extraction of the forms and data. Given the urgency of the project, creating a process that was fast, accurate, and consistent was paramount. To achieve this, the Trust’s Informatics Systems team worked closely with NDL to make sure that the process could be deployed in a timely manner.
An effective RPA process was created which interacted with the legacy system, the extract was run initially by a single bot. By scaling up to 10 bots that worked 24/7, the Trust was able to securely complete the extraction in 3 months. NDL’s orchestration layer was able to manage and prioritise the flow of records for extraction.
In addition, a dashboard was used by the Trust to visually monitor the progress of the data extraction project. The dashboard permitted users to see at-a-glance progress, identifying how many records had been migrated and how many were left.
The Benefits
With 2.6 million records needing to be processed, it was important that the rate of completion was fast, while not compromising the stability of the servers.
The migration started in July 2020 and by September 2020, over 75% of the extraction was complete, 30,000 records per day. The final files were completed in October 2020, allowing the legacy system to be decommissioned. It had been estimated that manually completing this process would have taken two man-years.
Once the migration was complete, all data was available for clinicians in the new system.
The legacy system servers could finally be switched off, removing the risk of unsupported hardware, server failure and data loss.
What's Next?
This first RPA implementation for NWBH was ambitious. As the project was a success, the Trust has embraced the technology and is planning for further implementations moving forwards. The next phase of this project is to place the extracted data into Rio.
On a wider scale, the Trust is looking into how they can automate many different back-office tasks which are heavily manual and repetitive – making them ideal candidates for RPA.
"RPA has been an extremely useful digital tool which has transformed data extraction, without which, we may have had to admit defeat and extract what data we could in a less efficient manner. On that note, RPA has truly saved the day!"
Laura Bow, Information Security Coordinator