Automating adjudication of events in cardiovascular trials
Clinical trials for cardiovascular (CV) disease are time consuming and expensive to run, often requiring large patient populations to meet the statistical requirements needed to demonstrate efficacy.1 These trials require robust data packages to achieve market approvals and registrations in order to get essential new medicines to patients.
At AstraZeneca, we have a bold ambition to revolutionise how we conduct clinical trials. We are focussing on how we can do things differently and bring innovation into every aspect of clinical development. The application of artificial intelligence (AI) and data science to running clinical trials has the potential to enable us to optimise the process at different stages with the intent of reducing the time overall.
One of the many ongoing projects that is part of our larger initiative is called Automating Identification Detection Adjudication (AIDA), which is made up of three components.
Firstly, in an ongoing CV trial, we want to find out the moment an event happens. The event sniffer is exploring different ways to detect events faster, exploring the use of home monitoring using connected devices, to geofencing to signal when a patient has been physically located at a hospital for some time. Recently in one of our trials, we also incorporated patient self-reporting through our new clinical study support platform called Unify.
The event harmonizer aims to automate data collection and processing, integrating data from a variety of different sources. This includes structured data, such as age or gender, and unstructured data, such as a hospital discharge summary. Unstructured data tends to be messy with less consistency but by training a deep learning algorithm to extract biomedical entities from text like a death certificate or a discharge summary, the necessary data can be processed. The text is then vectorized, meaning it is turned into numbers and it can then be correlated back with the structured data and run through a machine learning algorithm to be trained. By applying AI and machine learning techniques, we aim to build analytic ready data sets to accelerate the clinical trial process.
And finally, the event classifier is where we use machine learning and AI techniques to identify and classify the events. Adjudications in clinical trials, especially when it comes to CV events, can be a very laborious process as you have to verify that the patient did experience a specific event. Added to that is that in an outcome study, for the reliability of the results, capturing every event, and knowing exactly what has happened to the patients is key for the quality and reliability of the results.
Currently, the process uses external independent adjudication committees – namely human experts. However, this is resource intensive as it is a very manual and iterative process. While the study is ongoing, it can take more than a couple of months between when an event occurs, and when it can be verified.
Now, for the first time, we are presenting data at a scientific congress comparing the consistency between machine learning algorithms and human expert adjudicators to classify major adverse cardiovascular events in an outcomes trial.
In a poster presented at ESC Congress 2021, organised by the European Society of Cardiology (ESC), our results found that there was high consistency (>95% AUC-ROC) between automated and expert adjudication, demonstrating that machine-learning based adjudication of CV events (ischemic stroke, transient ischemic attack) has the potential for adoption in outcome trials.2
Next, our research will aim to enhance the machine learning models and pursue machine learning approaches to adjudicate other outcome and safety events.
1. Moore TJ, Zhang H, Anderson G, Alexander GC. Estimated Costs of Pivotal Trials for Novel Therapeutic Agents Approved by the US Food and Drug Administration, 2015-2016. JAMA Intern Med. 2018 Nov 1;178(11):1451-1457. doi: 10.1001/jamainternmed.2018.3931.
2. Lea H, Meeson A, Nampally S, et al. Can machine learning augment clinician adjudication of events in cardiovascular trials? A case study of major adverse cardiovascular events (MACE) across CVRM trials. ESC Congress 2021.