Do anesthesiologists know what they are doing? Mining a surgical time-series database to correlate expert assessment with outcomes

Risa B. Myers, John C Frenzel, Joseph R Ruiz, Christopher M. Jermaine

Research output: Contribution to journalArticle

Abstract

Anesthesiologists are taught to carefully manage patient vital signs during surgery. Unfortunately, there is little empirical evidence that vital sign management, as currently practiced, is correlated with patient outcomes. We seek to validate or repudiate current clinical practice and determine whether or not clinician evaluation of surgical vital signs correlate with outcomes. Using a database of over 90,000 cases, we attempt to determine whether those cases that anesthesiologists would subjectively decide are "low quality" are more likely to result in negative outcomes. The problem reduces to one of multi-dimensional time-series classification. Our approach is to have a set of expert anesthesiologists independently label a small number of training cases, from which we build classifiers and label all 90,000 cases. We then use the labeling to search for correlation with outcomes and compare the prevalence of important 30-day outcomes between providers. To mimic the providers' quality labels, we consider several standard classification methods, such as dynamic time warping in conjunction with a kNN classifier, as well as complexity invariant distance, and a regression based upon the feature extraction methods outlined by Mao et al. 2012 (using features such as time-series mean, standard deviation, skew, etc.). We also propose a new feature selection mechanism that learns a hidden Markov model to segment the time series; the fraction of time that each series spends in each state is used to label the series using a regression-based classifier. In the end, we obtain strong, empirical evidence that current best practice is correlated with reduced negative patient outcomes. We also learn that all of the experts were able to significantly separate cases by outcome, with higher prevalence of negative 30-day outcomes in the cases labeled as "low quality" for almost all of the outcomes investigated.

Original languageEnglish (US)
Article number24
JournalACM Transactions on Knowledge Discovery from Data
Volume10
Issue number3
DOIs
StatePublished - Feb 1 2016

Fingerprint

Labels
Time series
Classifiers
Feature extraction
Hidden Markov models
Surgery
Labeling

Keywords

  • Hidden Markov model (HMM)
  • Ordinal regression
  • Vital signs

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

@article{7ae3e0a417e4405bb07619bc02c20b8c,
title = "Do anesthesiologists know what they are doing? Mining a surgical time-series database to correlate expert assessment with outcomes",
abstract = "Anesthesiologists are taught to carefully manage patient vital signs during surgery. Unfortunately, there is little empirical evidence that vital sign management, as currently practiced, is correlated with patient outcomes. We seek to validate or repudiate current clinical practice and determine whether or not clinician evaluation of surgical vital signs correlate with outcomes. Using a database of over 90,000 cases, we attempt to determine whether those cases that anesthesiologists would subjectively decide are {"}low quality{"} are more likely to result in negative outcomes. The problem reduces to one of multi-dimensional time-series classification. Our approach is to have a set of expert anesthesiologists independently label a small number of training cases, from which we build classifiers and label all 90,000 cases. We then use the labeling to search for correlation with outcomes and compare the prevalence of important 30-day outcomes between providers. To mimic the providers' quality labels, we consider several standard classification methods, such as dynamic time warping in conjunction with a kNN classifier, as well as complexity invariant distance, and a regression based upon the feature extraction methods outlined by Mao et al. 2012 (using features such as time-series mean, standard deviation, skew, etc.). We also propose a new feature selection mechanism that learns a hidden Markov model to segment the time series; the fraction of time that each series spends in each state is used to label the series using a regression-based classifier. In the end, we obtain strong, empirical evidence that current best practice is correlated with reduced negative patient outcomes. We also learn that all of the experts were able to significantly separate cases by outcome, with higher prevalence of negative 30-day outcomes in the cases labeled as {"}low quality{"} for almost all of the outcomes investigated.",
keywords = "Hidden Markov model (HMM), Ordinal regression, Vital signs",
author = "Myers, {Risa B.} and Frenzel, {John C} and Ruiz, {Joseph R} and Jermaine, {Christopher M.}",
year = "2016",
month = "2",
day = "1",
doi = "10.1145/2822897",
language = "English (US)",
volume = "10",
journal = "ACM Transactions on Knowledge Discovery from Data",
issn = "1556-4681",
publisher = "Association for Computing Machinery (ACM)",
number = "3",

}

TY - JOUR

T1 - Do anesthesiologists know what they are doing? Mining a surgical time-series database to correlate expert assessment with outcomes

AU - Myers, Risa B.

AU - Frenzel, John C

AU - Ruiz, Joseph R

AU - Jermaine, Christopher M.

PY - 2016/2/1

Y1 - 2016/2/1

N2 - Anesthesiologists are taught to carefully manage patient vital signs during surgery. Unfortunately, there is little empirical evidence that vital sign management, as currently practiced, is correlated with patient outcomes. We seek to validate or repudiate current clinical practice and determine whether or not clinician evaluation of surgical vital signs correlate with outcomes. Using a database of over 90,000 cases, we attempt to determine whether those cases that anesthesiologists would subjectively decide are "low quality" are more likely to result in negative outcomes. The problem reduces to one of multi-dimensional time-series classification. Our approach is to have a set of expert anesthesiologists independently label a small number of training cases, from which we build classifiers and label all 90,000 cases. We then use the labeling to search for correlation with outcomes and compare the prevalence of important 30-day outcomes between providers. To mimic the providers' quality labels, we consider several standard classification methods, such as dynamic time warping in conjunction with a kNN classifier, as well as complexity invariant distance, and a regression based upon the feature extraction methods outlined by Mao et al. 2012 (using features such as time-series mean, standard deviation, skew, etc.). We also propose a new feature selection mechanism that learns a hidden Markov model to segment the time series; the fraction of time that each series spends in each state is used to label the series using a regression-based classifier. In the end, we obtain strong, empirical evidence that current best practice is correlated with reduced negative patient outcomes. We also learn that all of the experts were able to significantly separate cases by outcome, with higher prevalence of negative 30-day outcomes in the cases labeled as "low quality" for almost all of the outcomes investigated.

AB - Anesthesiologists are taught to carefully manage patient vital signs during surgery. Unfortunately, there is little empirical evidence that vital sign management, as currently practiced, is correlated with patient outcomes. We seek to validate or repudiate current clinical practice and determine whether or not clinician evaluation of surgical vital signs correlate with outcomes. Using a database of over 90,000 cases, we attempt to determine whether those cases that anesthesiologists would subjectively decide are "low quality" are more likely to result in negative outcomes. The problem reduces to one of multi-dimensional time-series classification. Our approach is to have a set of expert anesthesiologists independently label a small number of training cases, from which we build classifiers and label all 90,000 cases. We then use the labeling to search for correlation with outcomes and compare the prevalence of important 30-day outcomes between providers. To mimic the providers' quality labels, we consider several standard classification methods, such as dynamic time warping in conjunction with a kNN classifier, as well as complexity invariant distance, and a regression based upon the feature extraction methods outlined by Mao et al. 2012 (using features such as time-series mean, standard deviation, skew, etc.). We also propose a new feature selection mechanism that learns a hidden Markov model to segment the time series; the fraction of time that each series spends in each state is used to label the series using a regression-based classifier. In the end, we obtain strong, empirical evidence that current best practice is correlated with reduced negative patient outcomes. We also learn that all of the experts were able to significantly separate cases by outcome, with higher prevalence of negative 30-day outcomes in the cases labeled as "low quality" for almost all of the outcomes investigated.

KW - Hidden Markov model (HMM)

KW - Ordinal regression

KW - Vital signs

UR - http://www.scopus.com/inward/record.url?scp=84964598959&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84964598959&partnerID=8YFLogxK

U2 - 10.1145/2822897

DO - 10.1145/2822897

M3 - Article

AN - SCOPUS:84964598959

VL - 10

JO - ACM Transactions on Knowledge Discovery from Data

JF - ACM Transactions on Knowledge Discovery from Data

SN - 1556-4681

IS - 3

M1 - 24

ER -