The model developed accurately predicted, sustained events with more than 97% sensitivity, and the false-alert rate was kept to less than 25%, compared to around 80% when focused on all hypoglycemic events. |
Image: Getty Images
Over 30 million people in the United States have diabetes, approximately 10.5% of the country’s population, with 1.5 million new diagnoses made each year and an estimated annual cost exceeding $300 billion.
Because of the acute and severe consequences of hypoglycemia (low blood sugar), the fear of it often leads to less aggressive insulin therapy, ultimately increasing the long-term exposure of hyperglycemia (high blood sugar).
In order to optimize the management of diabetes, the long-term risks associated with hyperglycemia must be balanced with the risks for hypoglycemia.
Dr. Madhav Erraguntla, associate professor of practice in the Wm Michael Barnes ’64 Department of Industrial and Systems Engineering, and Dr. Balakrishna Haridas, professor of practice in the Department of Biomedical Engineering, are working to develop a prediction model for hypoglycemic events that has a low false-alert rate, is highly sensitive and specific, and can be applied to new patients and to new time periods.
The main motivation for this research is to explore ways to further improve predictive performance of hypoglycemia to reduce the rate of false alarms.
“Early detection of impending hypoglycemic events based on continuous glucose monitoring (CGM) readings — with high sensitivity and specificity and low false alarm rates — can help patients with diabetes better manage hypoglycemia and their overall health,” Erraguntla said.
The accurate hypoglycemia alerts proposed by the team will allow patients to better manage diabetes by reducing the fear of hypoglycemia.
The research was performed on de-identified, retrospective data, i.e., data already in existence.
The research team obtained CGM datasets from 110 pediatric patients with Type 1 diabetes over the course of 30 to 90 days. The data was comprised of over 1.6 million CGM values under normal living conditions and analyzed using machine-learning algorithms.
Instead of focusing on all hypoglycemic events, the team focused on sustained hypoglycemic events, defined as glucose values less than 70 mg/deciliter, for at least 15 minutes. Focusing on persistent events rather than transient events led to better model performance.
In prior research, the team developed a machine learning algorithm that detects hypoglycemia 30 minutes in advance using CGM readings. The team was able to achieve sensitivity (true positive rate) and specificity rates (true negative rate) of more than 90% accuracy. However, due to a small number of hypoglycemic events compared with non-hypoglycemic events, even this relatively high sensitivity and specificity can lead to false-alert rates of around 80%, which can result in patients taking unnecessary actions to avoid hypoglycemia.
Improving specificity in such highly imbalanced situations will reduce the false-alert rate and therefore improve user experience and trust in the alerts.
The machine-learning model ultimately developed accurately predicted, sustained events with more than 97% sensitivity and specificity for time frames of both 30 and 60 minutes into the future. The false-alert rate was kept to less than 25%, compared to around 80% when focused on all hypoglycemic events. In other words, only one out of four alerts are potentially false, compared to four out of five alerts being false before.
An additional benefit is that while the model focused on sustained hypoglycemic events, more than 61% of the transient hypoglycemic events are still detected as false positives by the model.
“The results of this research take us one more step closer to effectively managing diabetes and the huge burden it places on our health care,” Erraguntla said.
Because of the acute and severe consequences of hypoglycemia (low blood sugar), the fear of it often leads to less aggressive insulin therapy, ultimately increasing the long-term exposure of hyperglycemia (high blood sugar).
In order to optimize the management of diabetes, the long-term risks associated with hyperglycemia must be balanced with the risks for hypoglycemia.
Dr. Madhav Erraguntla, associate professor of practice in the Wm Michael Barnes ’64 Department of Industrial and Systems Engineering, and Dr. Balakrishna Haridas, professor of practice in the Department of Biomedical Engineering, are working to develop a prediction model for hypoglycemic events that has a low false-alert rate, is highly sensitive and specific, and can be applied to new patients and to new time periods.
The main motivation for this research is to explore ways to further improve predictive performance of hypoglycemia to reduce the rate of false alarms.
“Early detection of impending hypoglycemic events based on continuous glucose monitoring (CGM) readings — with high sensitivity and specificity and low false alarm rates — can help patients with diabetes better manage hypoglycemia and their overall health,” Erraguntla said.
The accurate hypoglycemia alerts proposed by the team will allow patients to better manage diabetes by reducing the fear of hypoglycemia.
The research was performed on de-identified, retrospective data, i.e., data already in existence.
The research team obtained CGM datasets from 110 pediatric patients with Type 1 diabetes over the course of 30 to 90 days. The data was comprised of over 1.6 million CGM values under normal living conditions and analyzed using machine-learning algorithms.
Instead of focusing on all hypoglycemic events, the team focused on sustained hypoglycemic events, defined as glucose values less than 70 mg/deciliter, for at least 15 minutes. Focusing on persistent events rather than transient events led to better model performance.
In prior research, the team developed a machine learning algorithm that detects hypoglycemia 30 minutes in advance using CGM readings. The team was able to achieve sensitivity (true positive rate) and specificity rates (true negative rate) of more than 90% accuracy. However, due to a small number of hypoglycemic events compared with non-hypoglycemic events, even this relatively high sensitivity and specificity can lead to false-alert rates of around 80%, which can result in patients taking unnecessary actions to avoid hypoglycemia.
Improving specificity in such highly imbalanced situations will reduce the false-alert rate and therefore improve user experience and trust in the alerts.
The machine-learning model ultimately developed accurately predicted, sustained events with more than 97% sensitivity and specificity for time frames of both 30 and 60 minutes into the future. The false-alert rate was kept to less than 25%, compared to around 80% when focused on all hypoglycemic events. In other words, only one out of four alerts are potentially false, compared to four out of five alerts being false before.
An additional benefit is that while the model focused on sustained hypoglycemic events, more than 61% of the transient hypoglycemic events are still detected as false positives by the model.
“The results of this research take us one more step closer to effectively managing diabetes and the huge burden it places on our health care,” Erraguntla said.