Public Sector / Modern Data Platform

Harnessing Cloud-scale COVID-19 Data Speeds Hope, Informs Public Healthcare



As the extraordinary, rapidly evolving COVID-19 pandemic took hold in the United States, one of the first regions to feel its devastating toll was also one of the first to seek solid strategies to manage disease transmission and slow infection rates. With no time to lose and everything at stake, this state’s heavily burdened Department of Health prioritized development of an innovative, tactical COVID-19 action response – employing technology to prevent countless deaths, lessen the strain on hospitals, enable contact tracing, and reduce economic shocks.

In collaboration with Microsoft and the state health department, BlueGranite – a Microsoft Gold partner, employed its Data and AI proficiency to help speed response tools into existence. Working together, we quickly facilitated a plan of action – clarifying technology goals around the region’s pandemic response – then embarked on a full-scale implementation effort.

The Challenge

Analytic epidemiology uses case data to try to understand health risks, uncover vulnerable populations, and highlight transmission methods. State public health and epidemiology experts rely on these insights to inform responses and policies, like stay-at-home orders, social distancing recommendations, and face mask requirements.

But public health funding cuts across the U.S. mean state health departments, like this one, are often relying antiquated, on-premises data systems to collect and analyze COVID-19 case data; the ability to fully scale and analyze the influx of COVID-19 cases here was infeasible using the legacy architecture.

Collaborating with Microsoft, and a Microsoft state and local government infrastructure partner, BlueGranite quickly acted to modernize the health department’s data and analytics architecture in Azure.


The Solution


We partnered to migrate on-premises systems, such as legacy disease reporting applications, to the cloud, allowing for dynamic scaling to accommodate the influx of testing and case reporting.

To reduce operational system strain and offload much of the data processing, BlueGranite focused on creating a hybrid architecture that integrates on-premises system data, and data hosted on Azure Infrastructure-as-a-Service virtual machines and based in Platforms-as-a-Service. A hybrid approach that includes IaaS and PaaS solutions can offer rapid time to value, pay-as-you-go cloud-scale development and deployment environments, and can reduce or eliminate software licensing and infrastructure costs.

We designed the rapid response architecture to replicate on-premises capabilities while allowing for ancillary processes – such as reporting – to continue with little interruption. We will enhance the hybrid architecture in a future phase, using a fully cloud-based approach.


Design Tools

Our phase one solution incorporates a multizone Azure Data Lake, which houses data from the constituent operational systems, and employs Azure Databricks for hyper-scalable data processing. Azure Data Factory orchestrates the integration of Databricks and the Data Lake.

The modern cloud data platform allows rapid integration of new data sources – such as REDCap and Microsoft Dynamics for data capture and contact tracing and MITRE’s Sara for its automated symptom monitoring capabilities.

Enabling the complex process of contract tracing required matching data across various systems – enriching sensitive, on-premises personal healthcare data with that from secure external databases. The process also required integration of symptom monitoring and testing referral data. Identity matching and de-duplication were also key. Once this was accomplished everything was re-integrated with core healthcare system and laboratory data to accurately capture provider encounters, hospital admissions, and test status.

These processes were performed using a mixture of R programming scripts and manual epidemiology team intervention. We then translated the logic to Databricks to modernize and automate the processes, allowing for more performant and scalable data integration processing to keep pace with growing COVID-19 cases.

The Results

The new data analytics platform enables in-depth insights, critical to data-driven, intelligent policy making. The state health department can easily plug in additional data sources as they become available, allowing even deeper analysis and more informed responses. These cloud-scale capabilities are sparking interest in further modernizing other systems and processes.

To discuss the many ways BlueGranite can speed Health and Life Sciences insights and improve outcomes – including scalable tools to inform COVID-19 policies at every level of government –contact us today.


Kickstart Your Digital Transformation With BlueGranite's Business Intelligence Solutions