Información de la revista
Vol. 36. Núm. 9.
Páginas 634-643 (diciembre 2012)
Descargar PDF
Más opciones de artículo
Vol. 36. Núm. 9.
Páginas 634-643 (diciembre 2012)
Acceso a texto completo
Relationship between mortality and first-day events index from routinely gathered physiological variables in ICU patients
Relación entre la mortalidad y el índice de eventos del primer día de estancia a partir de variables fisiológicas recogidas durante la monitorización rutinaria de pacientes de la UCI
R. Rivera-Fernándeza, E. Castillo-Lorenteb,
Autor para correspondencia
, R. Napc, G. Vázquez-Matad, D. Reis Mirandac
a Carlos Haya University Hospital, Málaga, Spain
b University Hospital of Jaén, Neurotrauma Center, Jaén, Spain
c University Hospital, Groningen, The Netherlands
d Medical Education Foundation, International Cooperation, Spain
Este artículo ha recibido
Información del artículo
Texto completo
Descargar PDF
Figuras (1)
Tablas (6)
Table 1. Relation between ICU mortality and mean duration of events (categorized in intervals).
Table 2. Characteristics of study population.
Table 3. Analysis of discrimination capacity for hospital mortality evaluated by area under Roc curve for different models.
Table 4. Events+age+coma+previous location index.
Table 5. Odds ratio of the final model.
Mostrar másMostrar menos

To test the hypothesis that the degree and duration of alterations in physiological variables routinely gathered by intensive care unit (ICU) monitoring systems during the first day of admission to the ICU, together with a few additional routinely recorded data, yield information similar to that obtained by traditional mortality prediction systems.


A prospective observational multicenter study (EURICUS II) was carried out.


Fifty-five European ICUs.


A total of 17,598 consecutive patients admitted to the ICU over a 10-month period.



Main variables of interest

Hourly data were manually gathered on alterations or “events” in systolic blood pressure, heart rate and oxygen saturation throughout ICU stay to construct an events index and mortality prediction models.


The mean first-day events index was 6.37±10.47 points, and was significantly associated to mortality (p<0.001), with a discrimination capacity for hospital mortality of 0.666 (area under the ROC curve). A second index included this first-day events index, age, pre-admission location, and the Glasgow coma score. A model constructed with this second index plus diagnosis upon admission was validated by using the Jackknife method (Hosmer-Lemeshow, H=13.8554, insignificant); the area under ROC curve was 0.818.


A prognostic index with performance very similar to that of habitual systems can be constructed from routine ICU data with only a few patient characteristics. These results may serve as a guide for the possible automated construction of ICU prognostic indexes.

Intensive care
Critical care
Severity of illness index

Comprobar si el grado y duración de las alteraciones en las variables fisiológicas recogidas en la monitorización rutinaria en UCI durante el primer día de estancia, junto con pocos datos adicionales, proporcionan información similar a la obtenida con los sistemas tradicionales de predicción de mortalidad.


Estudio observacional, prospectivo y multicéntrico (EURICUS-II).


55 UCIs de Europa.


17.598 pacientes consecutivos, ingresados durante 10 meses.



Variables de interés principales

se recogieron manualmente datos horarios sobre alteraciones o “eventos” en la presión arterial sistólica, frecuencia cardiaca y saturación de oxígeno, para construir un índice basado en estos eventos y un modelo de predicción de mortalidad.


El índice de eventos el primer día fue 6,37±10,47 puntos y se asoció significativamente con la mortalidad (p<0,001), con una capacidad de discriminación (área bajo la curva ROC) para la mortalidad de 0.666. Se construyó un segundo índice que incluye este índice de eventos en el primer día, la edad, procedencia del ingreso y puntuación de la Escala de Coma de Glasgow. Un modelo construido con este segundo índice más el diagnóstico fue validado mediante el método jackknife (Hosmer-Lemeshow, H=13.8554, no significativo), con un área bajo la curva ROC de 0,818.


Se puede construir un índice pronóstico con rendimiento similar al de los sistemas habituales a partir de los datos de monitorización de los pacientes en la UCI junto a escasas características del paciente. Nuestros resultados pueden servir de guía para la posible construcción automatizada de índices pronósticos.

Palabras clave:
Cuidados Intensivos
Cuidados Críticos
Índices de gravedad
Texto completo

Information on vital signs, e.g., blood pressure (BP), heart rate (HR), and oxygenation (SaO2) is routinely gathered in the ICU by monitoring equipment. Nurses currently transcribe data from monitors onto paper records, a labor-intensive activity.

Mortality prediction systems are widely used in critical care medicine to evaluate the quality of care (predicted versus observed mortality) and predict the mortality/survival of individual patients. Moreover, any study of critical care patients requires the quantification of their severity of illness, and numerous severity scoring systems have been developed, notably the APACHE,1–4 SAPS,5–7 and MPM8,9 systems, scoring patients according to their type of disease, age, history of disease, and degree of alteration of laboratory (hematocrit, leukocytes, etc.) and physiological (SBP, HR, etc.) variables. The score for physiologic variables is usually based on the most severe alteration observed in the patient at admission or during a given time (often 24h), taking no account of the timing or duration of abnormalities.

The use of prognostic indexes imposes a substantial work load. According to Teres and Higgins,10 who used trained data collectors at the bedside, the main issue is keeping up with the flow of patients. The IMPACT project of the American Society of Critical Care Medicine estimated that one full-time data collector is needed for every 20–30 ICU beds.11 For this reason, many centers do not routinely use these indexes. Automation of these tasks would facilitate their generalized routine application.

In the Euricus II study,12 detailed information was gathered throughout the ICU stay of patients on alterations in routinely collected physiological variables ICU (BP, HR, and oxygenation), classifying them as a function of their degree and duration.

The aim of this study was to test the hypothesis that routinely recorded data on the degree and duration of physiological alterations during ICU stay can be synthesized and used to predict mortality, and that first-day information plus very few additional data offer similar information to that provided by traditional mortality prediction systems. The data analyzed in the Euricus II projects were collected manually, the usual method in ICUs. The development of automatic computer systems to construct an index would enormously reduce the high workload currently required.


The data in this study were derived from the EURICUS PROJET,12–18 using the database of EURICUS II,12,13 a multi-center, randomized, and controlled research project designed to determine whether improved collaborative practices in the ICU have beneficial effects on intermediate (physiologic derangement) and final (mortality) outcomes of patients. The project (from 1997 to 1998) included 55 ICUs from different European states and had two components: one involved the training and instruction of nurses and physicians on collaborative practice and the other the daily use of two protocols. The first one, designed to increase the awareness of nurse–physician interactions, consisted of a decision tree related to hypoxia (oxygen saturation), diuresis, pulse, and (systolic) blood pressure for nurses to apply hourly in the titrated therapy cycle (monitoring-comparison-action). The second protocol guided and stimulated the daily exchange of complementary information between nurses and physicians.

ICU performance was measured by recording the clinical outcomes of consecutively admitted patients. During a 10-month period, clinical data were recorded by trained personnel on age, pre-admission location, SAPS-II measured severity of illness during first day of stay,6 admission diagnosis according to APACHE III classifications,3 daily nursing workload using Nine Equivalents of Nursing Manpower Score (NEMS),19 organ dysfunction/failure at ICU admission and at 24h using the Sequential Organ Failure Assessment Score (SOFA),20 events and critical events (see Annex 1), and discharge destination. Patients were followed until hospital discharge, and their survival status was recorded (ICU/hospital mortality). Readmissions were not analyzed.

Trained personnel collected clinical data and applied protocols and questionnaires. All participating staff attended training sessions in each country and had access to a detailed operation manual containing protocols and definitions.

The study was approved by the ethics committees of participating hospitals, who waived informed consent. Results of this randomized and controlled study were published elsewhere.12,13,21


Events were defined as abnormal physiologic values (SBP, HR, and SaO2). More detailed definitions of physiologic events are given in Annex 1.

Events were defined by two characteristics: (a) parameter value outside range of accepted values and (b) persistence of divergence from normality for a given time period. Events were classified as critical or non-critical according to their duration and degree of divergence from normality (Annex 1).

It was recognized that the ranges of normality may vary according to the patient and his/her clinical condition and that a fixed range of normality would often be incompatible with good clinical practice (e.g. SaO2 of >90% in a patient with chronic obstructive pulmonary disease). Consequently, the physician responsible for the patient defined the normality ranges for the parameters in each case. Nevertheless, the study protocol indicated a reference normality range for each parameter (previously indicated by a panel of experts) to reduce bias in the uniform definition of events. The frequency of changes to recommended ranges was recorded: low BP (<90mmHg) was changed on 8.6% of occasions, high BP (>180mmHg) on 1.3%, SaO2 (<90%) on 3.6%, low HR (<60bpm) on 25.3%, and high HR (>120bpm) on 24.7% of occasions.

Events data were collected from all patients. After calculation of the total number and duration of events, their relationship with ICU mortality was studied. An event <1h was considered to last 1h, given that incidents were routinely recorded on an hourly basis.

Before the present study, the relationship of mortality with number and duration of ICU events was analyzed in all patients,13 finding the mortality to be more closely related to the duration than number of events. Consequently, the mean durations of events were categorized into intervals (no events, 0–1h with events, 1–4h, and ≥4h with events), such that their duration was as similar as possible among different events and there was an adequate patient sample for each interval. The aim was to facilitate results interpretation and establish a possible basis for a simple-to-use scale or scales, because the relationship between ICU mortality and mean of duration of events is not linear.

The present study analyzed events on the first, second, and third ICU days.

Index construction and outcomes

The events index was based on the number, type, and duration of events, assigning each patient and each day with a score. The index was quantified as a function of the relationship with ICU mortality shown by each type of alteration (see Table 1). Coefficients were calculated by logistic regression analysis, with ICU mortality as dependent variable and mean duration of events (in intervals of: 0, 0–1, 1–4, and ≥4h) as independent variables. Coefficients were obtained by multiplying beta coefficients of the categories (in the logistic regression analysis) by 10 and rounding decimals to the nearest whole number. The events score was then calculated for each patient and for each day of stay, although the present article only considered the first, second, and third days of ICU stay, primarily focusing on first-day events. The first day was considered to end at midnight on the day of admission, at which time the second day was considered to commence.

Table 1.

Relation between ICU mortality and mean duration of events (categorized in intervals).

Blood pressure events  Frequency  Mortality 
Non-critical events
No events  8774  (4.6%) 
0–1h with events  5269  (17.3%) 
1–4h  2402  (33.4%) 
Four or more hours  308  (53.9%) 
Critical events
No events  11,177  (4.8%) 
0–1h with events  3298  (20.6%) 
1–4h  1675  (40.2%) 
Four or more hours  603  (65.5%) 
Oxygen saturation events
Non-critical events     
No events  12,035  (7.7%) 
0–1h with events  3833  (25.6%) 
1–4773  (40.8%) 
Four or more hours  112  (57.1%) 
Critical events     
No events  13,938  (8.1%) 
0–1h with events  2038  (33.4%) 
1–4576  (56.8%) 
Four or more hours  201  (72.6%) 
Heart rate events     
Non-critical events     
No events  10,433  (6.1%) 
0–1h with events  4642  (23.3%) 
1–41407  (32.1%) 
Four or more hours  271  (43.2%) 
Critical events     
No events  12,202  (6.8%) 
0–1h with events  2635  (26%) 
1–41357  (36.3%) 
Four or more hours  559  (49.7%) 

The events score was constructed by using the ICU mortality as the dependent variable, following a previously applied methodology13 that had explained almost all of the ICU mortality by the combination of ICU events, multiple organ dysfunction, and diagnosis. Accordingly, we used data on the events on each day of stay in the ICU and their relationship with ICU mortality in order to determine the coefficients for calculating events index scores. The hospital mortality, the more usual outcome measure, was considered in the other analyses and was used to construct a second index that integrated the events index with other variables (i.e., first-day events index score, Glasgow Coma Scale, previous location, and age, classified as in SAPS II), designated the “combined events index” (see Results section).

Annex II shows two examples of how to calculate the “combined events index” and the probability of hospital mortality.

Statistical analysis

Data are expressed as mean±standard deviation for continuous variables and absolute frequency and percentage for categorical variables. The Student's t test was used to compare two means. The Analysis of Variance (ANOVA) and Newman–Keuls test for multiple comparisons were applied to compare more than two means. The Pearson coefficient was used to evaluate associations between quantitative variables.

The relationship of hospital mortality with the events score and other variables was studied by logistical regression, creating dummy variables for categorical variables. Based on these results, two indexes were created: events index and “combined events index”. Scores for variables and intervals for their categorization were assigned as a function of the beta coefficients obtained in the logistic regression. The discriminatory ability of the models was evaluated by the area under the ROC curve.22

We analyzed the discrimination of several possible models and studied the improvements in discrimination obtained by the addition of new variables that can be easily introduced into ICU monitoring systems and are known to discriminate between survival and non-survival, being included in APACHE, SAPS, and MPM and other widely used models. The discrimination of the events index was improved by adding age, previous location, and GCS, yielding the “combined events index”. This index was completed by including the admission diagnosis (classified as in APACHE III), which can also be included in the monitoring system at ICU admission. This final model was selected as the main object of the present investigation and for validation studies.

The entire sample was used for the creation of the model, and cross-validation was performed by using a jackknife approach,23 dividing the whole sample into 10 mutually exclusive groups by random method; 10 different regression models were estimated, with each model excluding one group; each model was used to calculate predictions for the excluded group. Predicted risks for individual patients from the excluded groups were then compared with predictions based on the entire sample. In all cases, the same variables with identical codes were included in the equations. The area under the ROC curve was also calculated for the predictions obtained by the cross-validated method, and the Hosmer Lemeshow test was used to compare observed and predicted mortality. The SPSS statistical package and MedCalc statistical software were used for all analyses; p<0.05 was considered significant.


The study included 17,598 patients from 55 ICUs. Data on events were available for 17,022 patients (Table 1). Patients characteristics are shown in Table 2.

Table 2.

Characteristics of study population.

Characteristic  Study population (N=17,598) 
Age (years)  59.11±22.06 
SAPS II (points)  31.17±8.42 
Type of admission   
Medical (%)  48.6 
Elective surgical (%)  24.7 
Emergency surgical (%)  26.7 
ICU length of stay  5.2±9.94 
ICU mortality (%)  13.9 
Hospital mortality (%)  20.6 

The mean events index value for the first day was 6.37±10.47 points (range, 0–70 points). Events index values were significantly associated with ICU and hospital mortality: the mean value was 4.85±8.47 points for the 13,348 patients who survived hospital admission versus 12.47±4.66 points for the 3405 who did not (p<0.001) and was 5.05±8.68 points for the 14,446 who survived ICU stay versus 14.91±15.72 for the 2287 who did not (p<0.001).

The relationship with mortality was further explored by dividing the index into intervals: the 9205 patients with a first-day index score of 0 showed a hospital mortality rate of 12.9%, the 3487 with a score of 1–10 points, mortality of 20.6%; the 1,934 with 11–20 points, mortality of 30.9%; the 983 with 20–30 points, mortality of 38.4%; and the 784 with >30 points, a hospital mortality rate of 58.2% (χ2=1342, df=4, p<0.001). The events index was also significantly associated with age (r=0.118, p<0.001), SAPS II score (r=0.377, p<0.001), and GCS score (r=−0.207, p<0.001).

The index score significantly differed according to patient type, with 7.73±11.68 points for medical patients, 5.53±9.9 for emergency surgical patients, and 4.75±8.11 for elective surgical patients (p<0.001 by ANOVA; significant intergroup differences by Newman–Keuls test). Trauma patients were included as medical or surgical patients according to their need for surgery.

The discrimination capacity for hospital mortality, evaluated by the area under the ROC curve, was 0.666 for the first-day events index score and 0.776 for the SAPS II score. The discrimination capacity of the events index was markedly improved by the addition of some other variables: addition of age increased the area under the ROC curve from 0.66 to 0.709; addition of first-day GCS score increased it to 0.763; addition of previous location to 0.792 and addition of admission diagnosis to 0.816 (Table 3).

Table 3.

Analysis of discrimination capacity for hospital mortality evaluated by area under Roc curve for different models.

Events index (EI)  0.666  (0.655–0.677) 
EI+agea  0.709  (0.7–0.719) 
EI+agea+GCSb  0.763  (0.754–0.772) 
EI+agea+GCSb+previous locationc  0.792  (0.783–0.800) 
EI+agea+GCSb+previous locationc+diagnosisd  0.816  (0.808–0.824) 
SAPS II+EI+agea+GCSb+previous locationc+diagnosisd  0.833  (0.823–0.838) 

There were statistically significant differences in all cases (p<0.05) by Hanley–Mcneil method.


Age classified as in SAPS II score.


Glasgow Coma Scale classified as in SAPS II score.


Previous location classified as in APACHE III: other hospital, emergency area, hospital ward, emergency surgery and elective surgery.


Diagnosis classified as in APACHE III.

Based on the results of logistic regression analysis with hospital mortality as dependent variable, a second index was constructed (“combined events index”) that included the first-day events index, GCS, and age, classified as in SAPS II, but with scores assigned according to the logistic regression results. It also included the previous location of the patient and reason for admission, which were both classified as in APACHE III and scored as a function of the logistic regression results. The score assigned to variables or categories was based on the beta coefficients of the logistic regression analysis such that the events score was similar to the original (because the beta coefficient of the score was 0.04, coefficients of the categories of the other variables were multiplied by 25 and decimals were rounded to the nearest whole number). Table 4 shows the scores assigned to categories of the new index. Selection of additional variables (age, GCS, and previous location) was based on previous knowledge of their importance and on the considerable improvement in the discriminating capacity of the events index produced by their addition, with very little additional data collection workload.

Table 4.

Events+age+coma+previous location index.

Events score
Blood pressure events  Score 
Non-critical events
No events 
0–1h with events 
Four or more hours 
Critical events
No events 
0–1h with events 
1–4h  14 
Four or more hours  23 
Oxygen saturation events
Non-critical events   
No events 
0–1h with events 
Four or more hours 
Critical events   
No events 
0–1h with events 
Four or more hours  17 
Heart rate events
Non-critical events   
No events 
0–1h with events 
Four or more hours 
Critical events   
No events 
0–1h with events 
Four or more hours  10 
<40 years 
40–59 years  10 
60–69 years  17 
70–74 years  24 
75–79 years  27 
>80 years  33 
Glasgow Coma Scale
14–15 points 
6–13 points  16 
<6 points  46 
Previous location
Elective surgery 
Emergency  12 
Emergency surgery  13 
Other Hospital  17 
Other Ward  28 

The “combined events index” showed a significant relationship with hospital mortality. The score was 65.76±28.53 points for the 3352 who died in hospital versus 37.51±21.54 for the 13,152 who survived (p<0.001); the mean score for all 16,504 patients was 43.25±25.77 points. There was also a significant difference in this index score between ICU non-survivors and survivors, with a score of 71.11±29.32 points for the 2250 who died in the ICU versus 38.85±22.16 for the 14,254 who survived (p<0.001).

We evaluated the relationship of the length of ICU stay with the SAPS II, events index, and “combined events index”, finding r=0.221 (p<0.001) with SAPS II, r=0.095 (p<0.001) with events index, and r=0.147 (p<0.001) with “combined events index”.

Finally, a hospital mortality prediction model was constructed and subsequently validated. It included the variables: the “combined events index”, the square of this index, and the admission diagnosis (Table 5). This model was validated by means of a Jackknife approach as explained above, using the same sample for the creation and validation of the model. The Hosmer Lemeshow test was applied to evaluate the discrepancy between the prediction of the cross-validated model and the observed mortality, giving a value of H=13.8554 (df 10), not statistically significant (Fig. 1). The discrimination capacity (area under the ROC curve) was 0.818 for the original model and 0.812 for the cross-validated one. This Jackknife-validated model differs from that shown in Table 3 under the heading “EI+Age+GCS+Previous location+Diagnosis”, because the latter does not include the square of the index score, which was added to improve the calibration.

Table 5.

Odds ratio of the final model.

Index  1.053  (1.046–1.06) 
(Squared index)/100  0.989  (0.984–0.994) 
Most frequent diagnoses
Acute myocardial infarction  0.231  (0.14–0.381) 
Bacterial pneumonia  0.542  (0.343–0.856) 
Other respiratory diseases  0.433  (0.268–0.699) 
Cranial trauma (with/without multiple trauma)  0.252  (0.155–0.409) 
Medication poisoning  0.059  (0.031–0111) 
Surgical diseases     
Lung neoplasm  0.117  (0.065–0.209) 
Gastrointestinal rupture/perforation  1.257  (0.780–2.025) 
Gastrointestinal neoplasm  0.637  (0.333–1.217) 

For the diagnosis variable, cardiogenic shock was used as the reference category.

Figure 1.

Predicted vs. observed mortality for validated model.

Events on first, second, and third day

The information contributed by analysis of second- and third-day events was also studied. The mean events index score of patients was 7.17±11.49 points on the second day of admission (n=14,485) and 6.91±11.37 points on the third day (n=9570). When first-day events data were complemented with second-day events, by using the worst (highest) value for each patient in the first two days (or first-day data for patients with stay of 1 day), the area under the ROC curve increased from 0.666 to 0.716 (0.709–0.723), and when the worst value in the first three days was used it increased to 0.734 (0.728–0.741) (all differences, p<0.05; Hanley–Mcneil method). The area under the ROC curve increased to 0.765 (0.759–0.772) when first-day GCS score was included (in 3-day model), to 0.788 (0.781–0.794) when age was added, to 0.807 (0.801–0.814) when previous location was added; and to 0.831(0.825–0.837) when diagnosis was also added (all differences, p<0.05).


This study showed that a prognostic index that performs as well as the usual systems can be constructed from data already routinely recorded in the ICU complemented with a few characteristics of the patient and disease. One advantage of our index is that laboratory data are not required for its calculation. Although the data in the present study were manually collected, similar results could be expected by using the same data automatically gathered by ICU monitoring equipment. The results of this study may serve as a guide for a future automatization of the collection and use of prognostic indexes in the ICU.

An important component of this instrument is an events index that expresses the degree and duration of alterations in the HR, SBP, and oxygenation of the patient. Although the term “adverse event” is frequently utilized to describe injuries caused by medical management,24 it refers here to alterations in the above physiologic functions. Because numerous false alarms were expected,25,26 a 10-min time limit was established for recording abnormalities in physiological variables (events), reducing the number of false alarms and non-relevant events and excluding the extreme alterations observed when time limits are not imposed.

The variables used to calculate the “events index” (HR, SBP, SaO2) were selected because they were studied in considerable detail in the Euricus Project and because they are routinely recorded by bedside monitoring equipment in ICU patients, opening up the possibility of their automated collection. They were classified as critical or non-critical in the Project and therefore in the present study; the categorization of events and the scoring procedures were previously reported.13 This index evaluates not only the degree of an alteration but also, unlike other indexes, its duration. In fact, the discrimination capacity shown by this model was superior to that of SAPS II and similar to that of other prediction systems. Thus, although the area under the ROC curve was 0.81 for SAPS II in its reference population, it was 0.77 in our series, lower than the 0.82 obtained with our model. The model with “EI+Age+GCS+Previous location+Diagnosis+SAPS II showed a discrimination of 0.83. The difference between 0.81 and 0.83 may be explained by the additional information provided by the analytical variables (e.g., bilirubin, creatinine, platelets, and leucocytes) included in SAPS II but not in our model.

Importantly, our model also showed a good agreement between the predicted and observed mortality in the present series, with no significant difference according to the Hosmer Lemeshow test using a Jackknife approach (Fig. 1).

The manual collection of data from the documentation (charts, laboratory reports, etc.) for each patient, required by the usual prediction systems and indeed for our own index, imposes a heavy workload when systematically carried out in all patients. Even more effort and time must then be spent analyzing the data in order to estimate the likelihood of death for individuals and groups of patients. However, various researchers recently described computer systems for physiological data acquisition and analysis27–31 that offer complete information on the patient's clinical condition in the ICU. Bioengineering advances have led to an impressive amount of reliable data being available at the bedside,27,31,32 but clinical practice has yet to take full advantage of this rich, varied, and continuous stream of information, and the synthesis and interpretation of information is still performed manually in many ICUs, with minimal preprocessing.

Our original events data were gathered manually, before the possibility of constructing an index was conceived. Hence, these results can be generalized for manual but not automatic data collection systems. Moreover, we were unable to compare workloads between manual and automatic methods. Nevertheless, we believe that current bedside systems, with the appropriate settings, are capable of automatically gathering, analyzing, and transforming these data. They can be programmed for this purpose, entering the other variables considered in our study (age, pre-admission location, GCS score, and admission diagnosis [APACHE III classification]), and they can deliver highly valuable information, including the “combined index score” and likelihood of survival/mortality for each patient. A further advantage of an automated index is the reduction in inter-observer variability, which can influence mortality prediction.33

Our intention was not to replace existing indexes,1–9 since the manual collection of data on the degree and duration of these alterations implies a heavier work load. However, these routinely collected data appear to offer similar information to that provided by these systems, and future technological developments may vastly reduce the work involved in constructing this new type of index.

There have been reports on systems that use monitoring equipment software for vital constant analysis and mortality predictions. The Vanderbilt University group demonstrated that HR variability is a good predictor of mortality in trauma patients as early as 12h after admission and throughout ICU stay.34–36 Correlations have also been reported between HR variability and severity of illness and outcome in critically ill and injured children37 and diverse populations of critical ill patients.38 In the present study, mortality was found to be related to the alteration of three vital constants and to the duration of the alteration, allowing all three to be used in a prognostic index with an acceptable discrimination capacity.

Attempts have been made to develop an automated severity of illness and mortality risk system using data available in hospital electronic databases, e.g. the Veterans Affair Healthcare system, a model with a good discrimination and calibration.39 This experience supports the development of automated ICU risk prediction systems.

Tarassenko et al.40 recently published a system based on automated vital sign acquisition for intelligent alerting in patients with redeemable morbidity or avoidable mortality, further indicating the possibility of new prediction targets with this approach.

One study limitation is the possible influence on the results of the ability of the physician-in-charge to change the normality limits established for the physiological variables. In fact, they were respected in the large majority of cases, and changes were at the limits of normality, suggesting that a large proportion of ICUs considered these limits to be clinically useful.

The variables comprising the first-day events index (HR, SBP, SaO2) showed a low predictive value (area under ROC curve of 0.66) in comparison to other variables in the predictive system (age, previous location and Glasgow scale score, etc.). However, the latter also carry considerable weight in the usual severity indexes (SAPS, APACHE), and we do not consider this to be a study limitation. It may indicate that severity indexes that do not include these variables and rely solely on monitoring values may not perform as well as the habitual systems.

It is highly likely that the discrimination capacity of our index can be further improved after a period of implementation and research into the complementary use of other prognostic factors. Our study demonstrates that an automated system can be designed by taking into consideration the selected findings. Nevertheless, the effective functioning and usefulness of this approach can only be established after its application and validation in different populations. It is also necessary to test in practice our claim that it would significantly reduce the workload required by the commonly used prognostic systems and produce an economic saving that justifies its implementation.


This study forms part of the EURICUS project and was supported by European Union grant BMH4-CT96-0817.

Conflicts of interest

The authors have no conflicts of interest to declare.

Annex I
Definition of adverse events

  EVENT (Ev)Critical event (CrEv)
  Suggested range  Continuously out-of-range  Intermittently out-of-range  Continuous event  Discontinuous event  Event anytime 
SBP(a) (mmHg)  90–180  ≥10′  ≥10′ in 30′  ≥60′  ≥60′ in 2<60 
SaO2(b)  ≥90%  ≥10′  ≥10′ in 30′  ≥60′  ≥60′ in 2<80% 
HR(c) (bpm)  60–120  ≥10′  ≥10′ in 30′  ≥60′  ≥60′ in 2<30 or >180 

(a) Blood pressure events

The normal range indicated in the protocol was 90–180mmHg. SBP was recorded hourly, and it was noted whether the measurement was made continuously or discontinuously (e.g., with sphygmometer).

A non-critical blood pressure event was recorded when: (a) SBP remained out of range for ≥10min (hypo-hypertension); (b) SPB was out of range for a total of ≥10min during a 30-min period if SBP was unstable and out of accepted range for periods of <10min. A Critical Blood Pressure event was recorded when: (a) SPB event lasted for ≥60min; (b) SBP event lasted a total of ≥60min during a 2-h period if SBP was unstable and out of range for <60min; or (c) SBP was <60mmHg (any duration).

(b) Oxygen saturation events

The normal range according to the protocol was >90%. SaO2 was recorded when blood gas analysis was performed, or hourly in cases of continuous monitoring (e.g. pulse oximeter). In case of discontinuous monitoring, the occurrence of the event is defined by one measurement of SaO2 below range.

Non-critical oxygen saturation events

In continuous monitoring: (a) SaO2 remained below accepted value (hypoxia) for ≥10min; or (b) was below this value for a total of ≥10min during a 30-min period if SaO2 was unstable and below this value for periods of <10min. During artificial lung ventilation, SaO2 can fall below the accepted value due to care procedures (e.g. endotracheal suction). Therefore, the time factor for a hypoxia event in these cases was only considered after completion of these procedures (e.g., after re-connection of patient to the mechanical ventilator). In discontinuous monitoring (e.g., at blood gas analysis), the time factor was omitted and the event was exclusively defined by a measurement below the accepted value.

Critical oxygen saturation events

In continuous monitoring: (a) The event lasted ≥60min; (b) the event lasted a total of ≥60min during a 2-h period if the SaO2 was unstable and events lasted <60min; or (c) SaO2 was <80% (any duration). In discontinuous monitoring: (a) SaO2 was below the accepted value at two consecutive measurements, or (b) SaO2 was <80% at one measurement.

(c) Heart rate events

The normal range according to the protocol was 60–120bpm. HR was recorded hourly, and it was noted whether the measurement was made continuously (ECG lead) or manually.

A non-critical heart rate event was recorded when: (a) HR remained out of range (brady-tachycardia) for ≥10min; or (b) HR was out of range for a total of ≥10min during a 30-min period if HR was unstable and out of range for periods of <10min. A critical heart rate events was recorded when: (a) event lasted for ≥60min; (b) event lasted for a total of ≥60min during a 2-h period if HR was unstable and out of range for periods <60min; or (c) HR was <30bpm or >180bpm (any duration).

Recovery time

Depending upon their clinical condition, patients may have more than one critical event on the same day. It was therefore necessary to define the recovery period after one critical event was ended and before the next event was registered, as follows: SBP within accepted range for >60min; SaO2 within accepted range for >60min; HR within accepted range for ≥60min.

Annex II

Example 1

As reported in annex 1 and the accompanying table, in a 66-year-old patient under continuous SBP monitoring, an SBP <90 but ≥60mmHg for <10min is not considered an event and the duration is therefore 0; when the duration of this episode is >10min but <60min, it is considered a non-critical event with a duration of 1h and an events scale score of 4 points (see Table 4). The patient shows a fall in SBP to below 90mmHg, the lower limit of the range, at 3h 20min, and then returns to within the accepted range at approximately 4h 10min, still not considered a critical event. The SBP again falls to below 90mmHg at 4h 20min and stays there for approximately 10min, therefore the total time with SBP <90mmHg during a 2-h period is around 1h (period 1+period 2). During the next three hours, the SBP fluctuates and is outside the accepted range for ≥60min. Hence, the events score for this patient is 23 points for a critical event of ≥4h (see Table 4). The patient shows no further events and is admitted from the Emergency Area for traumatic brain injury with a Glasgow Coma Scale of 5 points, being assigned a score of 98 points on the “Events+Age+Coma+Previous location” index, (Events, 23; Age, 17; Coma, 46; Previous location 12).

In order to calculate the probability of hospital mortality, we first compute the logit:

Logit=Constant+[0.051×(event+age+coma+previous location index)]+[−0.114×(events+age+coma+previous location index)×(events+age+coma+previous location index)/100]+diagnosis coefficient.

In this case:


This logit is converted to the probability of hospital mortality by the following equation:

Probability of hospital mortality=exp(logit)/[1+exp(logit)]. Hence, in this case, the probability of hospital mortality is 0.4953 (49.53%).Example 2

A 45-year-old patient is admitted to the ICU after elective surgery (lung neoplasm) with a Glasgow outcome scale of 15 points, oxygen saturation values always >90, SBP always 90–180 and HR always 60–20 except for a 5-min period above 120 before returning to 60–120. The events score for this patient is 0 points and the “Events+age+coma+previous location” index score 10 points (events, 0; age, 10; GCS, 0 points; elective surgery, 0 points).

In this case:


Probability of hospital mortality=exp(−4.295)/[1+exp(−4.295)]=0.0135.

Hence, the probability of hospital mortality is 1.35%.

W.A. Knaus, J.E. Zimmerman, D.P. Wagner, E.A. Draper, D.E. Lawrence.
APACHE-acute physiology and chronic health evaluation: a physiologically based classification system.
Crit Care Med, 9 (1981), pp. 591-597
W.A. Knaus, E.A. Draper, D.P. Wagner, J.E. Zimmerman.
APACHE II: a severity of disease classification system.
Crit Care Med, 13 (1985), pp. 818-829
W.A. Knaus, D.P. Wagner, E.A. Draper, J.E. Zimerman, M. Bergner, P.G. Bastos, et al.
The Apache III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults.
Chest, 100 (1991), pp. 1619-1636
J.E. Zimmerman, A.A. Kramer, D.S. McNair, F.M. Malila.
Acute Physiology and Chronic Health Evaluatio (APACHE) IV: hospital mortality assessment for today's critically ill patientes.
Crit Care Med, 34 (2006), pp. 1297-1310
J.R. Le Gall, P. Loirat, A. Alperovitch, P. Glaser, C. Granthil, D. Mathiue, et al.
A simplified acute physiology score for ICU patients.
Crit Care Med, 12 (1984), pp. 975-977
J.R. Le Gall, S. Lemeshow, F. Saulnier.
A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study.
J Am Med Assoc, 270 (1993), pp. 2957-2963
R.P. Moreno, P.G. Metnitz, E. Almeida, B. Jordan, P. Bauer, R.A. Campos, et al.
SAPS 3-from evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission.
Intensive Care Med, 31 (2005), pp. 1345-1355
S. Lemeshow, D. Teres, J. Klar, J.S. Avrunin, J.S. Steingrub.
Mortality probability models (MPM II) based on an international cohort of Intensive care unit patients.
J Am Med Asooc, 270 (1993), pp. 2478-2486
T.L. Higgins, D. Teres, W.S. Copes, B.H. Nathanson, M. Stark, A.A. Kramer.
Assessing contemporary intensive care unit outcome: an updated Mortality Probability admission Model (MPM0-III).
Crit Care Med, 35 (2007), pp. 827-835
D. Teres, T.L. Higgins.
What if your hospital informatics department could provide severity adjuster.
Crit Care Med, 28 (2000), pp. 3570-3571
T. Higgins, W. McGee, J.S. Steingrub, J. Rapoport, S. Lemeshow, D. Teres.
Early indicators of prolonged intensive care unit stay: impact of illness severity, physician staffing, and pre-intensive care unit length of stay.
V. Fidler, R. Nap, D. Reis Miranda.
The effect of a managerial-based intervention on the occurrence of out-of-range-measurements and mortality in Intensive Care Units.
J Crit Care, 19 (2004), pp. 130-134
R. Rivera-Fernández, R. Nap, G. Vázquez-Mata, D. Rei-Miranda.
Analysis of physiologic alterations in intensive care unit patients and their relationship with mortality.
J Crit Care, 22 (2007), pp. 120-128
R. Moreno, D.R. Miranda, V. Fidler, R. Van Schilfgaarde.
Evaluation of two outcome prediction models on an independent database.
Crit Care Med, 26 (1998), pp. 50-61
R. Moreno, D. Reis Miranda.
Nursing staff in intensive care in Europe: the mismatch between planning and practice.
Chest, 113 (1998), pp. 752-758
R. Moreno, G. Apolone, D.R. Miranda.
Evaluation of the uniformity of fit of general outcome prediction models.
Intensive Care Med, 24 (1998), pp. 40-47
G. Iapichino, D. Radrizzani, G. Bertolini, L. Ferla, G. Pasetti, A. Pezzi, et al.
Daily classification of the level of care. A method to describe clinical course of illness, use of resources and quality of intensive care assistance.
Intensive Care Med, 27 (2001), pp. 131-136
R. Moreno, G. Apolone.
Impact of different customization strategies in the performance of a general severity score.
Crit Care Med, 25 (1997), pp. 2001-2008
D. Reis Miranda, R. Moreno, G. Iapichino.
Nine Equivalents of Nursing Manpower Use Score (NEMS).
Intensive Care Med, 23 (1997), pp. 760-765
J.L. Vincent, R. Moreno, J. Takala, S. Willatts, A. De Mendonça, H. Bruining, et al.
The SOFA (sepsis-related organ failure assessment) score to describe organ dysfunction/failure.
Intensive Care Med, 22 (1996), pp. 707-710
R. Moreno, D. Reis Miranda, R. Matos, T. Fevereiro.
Mortality after discharge from intensive care: the impact of organ system failure and nursing workload use at discharge.
Intensive Care Med, 27 (2001), pp. 999-1004
J.A. Hanley, B.J. Mcneil.
The meaning and use of the area under receiver operating characteristics (ROC) curve.
R.G. Miller.
The Jacknife – a review.
Biometrika, 61 (1974), pp. 1-15
J.M. Rothschild, C.P. Landrigan, J.W. Cronin, R. Kaushal, S.W. Lockley, E. Burdick, et al.
The Critical Care Safety Study: the incidence and nature of adverse events and serious medical errors in intensive care.
Crit Care Med, 33 (2005), pp. 1694-1700
M.C. Chambrin.
Alarms in the intensive care unit: how can the number of false alarms be reduced?.
Crit Care, 5 (2001), pp. 184-188
M. Imhoff, S. Kuhls.
Alarm algoritms in critical care monitoring.
Anesth Analg, 102 (2006), pp. 1525-1537
B. Goldstein, J. McNames, A. McDonald, M. Ellenby, S. Lai, Z. Sun, et al.
Physiologic data acquisition system database for the study of disease dynamics in the intensive care unit.
Crit Care Med, 31 (2003), pp. 433-441
I. Korhonen, M. van Gils, J. Gade.
The challeges in creating critical-care databases.
IEEE Eng Med Biol Mag, 20 (2001), pp. 58-62
I. Kropyvnytskyy, F. Saunders, P. Schierek, M. Pols.
A computer system for continuous long-term recording, processing, and analysis of physiologic data of brain injured patients in ICU settings.
Brain Inj, 15 (2001), pp. 577-583
E.J. Siachalou, I. Kitsas, J. Konstantinos, E.T. Zadelis, C.D. Saragiotis, Y.A. Tolias, et al.
ICASP: an intensive care acquisition and signal processing integrated framework.
J Med Syst, 29 (2005), pp. 646-663
G.D. Clifford, W.J. Long, G.B. Moody, P. Szolovits.
Robust parameter extraction for decision support using multimodal intensive care data.
Phil Trans R Soc A, 367 (2009), pp. 411-429
W.C. Hanson, B.E. Marshall.
Artificial intelligence applications in the intensive care unit.
Crit Care Med, 29 (2001), pp. 427-435
L. Dominguez, P. Enríquez, P. Álvarez, M. De Frutos, V. Sagredo, J. López-Messa, et al.
Evaluación de la reproducibilidad de la recogida de datos para el APACHE II: APACHE III adaptado para España y SAPS II en nueve Unidades de Cuidados Intensivos en España.
Med Intensiva, 32 (2008), pp. 15-22
E.L. Grogan, P.R. Norris, T. Speroff, A. Ozdas, D.J. France, P.A. Harris, et al.
Volatility: a new vital sign using a novel bedside monitoring strategy.
J Trauma, 58 (2005), pp. 7-12
Norris Pr, J.A. Morris, A. Ozdas, E.L. Grogan, A.E. Williams.
Heart rate variability predicts trauma patient outcome as early as 12h: implications for military and civilian triage.
J Surg Res, 129 (2005), pp. 122-128
P.r. Norris, J.A. Morris, A. Ozdas, L.R. Waitman, F.E Harrell Jr., A.E. Williams, et al.
Reduced Heart variability: an indicator of cardiac uncoupling and diminished physiologic reserve in 1425 trauma patients.
B. Goldstein, D.H. Fiser, M.M. Kelly, D. Mickelsen, V. Ruttimann, M.M. Pollac.
Descomplexification in critical illness and injury: relationship between heart rate variability, severity of illness, and outcome.
Crit Care Med, 26 (1998), pp. 352-357
T.G. Buchman, P.K. Stein, B. Goldstein.
Heart rate variability in critical illness and critical care.
Curr Opin Crit Care, 8 (2002), pp. 311-315
M.L. Render, H.M. Kim, J. Deddens, S. Sivaganesin, D.E. Welsh, K. Bickel, et al.
Variation in outcomes in Veterans Affairs intensive care units with a computerized severity measure.
Crit Care Med, 33 (2005), pp. 930-939
L. Tarassenko, D.A. Clifton, M.R. Pinsky, M.T. Hravnak, J.R. Woods, P.J. Watkinson.
Centile-based early warning scores derived from statistical distributions of vital signs.
Resuscitation, 82 (2011), pp. 1013-1018
Copyright © 2011. Elsevier España, S.L. and SEMICYUC
Medicina Intensiva
Opciones de artículo
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?