RESEARCH—HUMAN—CLINICAL STUDIES

RESEARCH—HUMAN—CLINICAL STUDIES

Predicting Outcome in Traumatic Brain Injury:

Development of a Novel Computerized
Tomography Classification System (Helsinki
Computerized Tomography Score)
Rahul Raj, PhD(c), BM* BACKGROUND: Early computerized tomography (CT) abnormalities are important
Jari Siironen, MD, PhD* predictors of outcome after traumatic brain injury (TBI).
Markus B. Skrifvars, MD, PhD‡ OBJECTIVE: To develop a novel CT scoring system (Helsinki CT score) and to compare it
with the Marshall CT classification and the Rotterdam CT score in predicting long-term
Juha Hernesniemi, MD, PhD*
outcome of patients with TBI.
Riku Kivisaari, MD, PhD*
METHODS: Eight hundred sixty-nine consecutive TBI patients were included in this
*Departments of Neurosurgery and open-cohort, retrospective, single-center study. Logistic regression was used to develop
‡Intensive Care, Helsinki University the Helsinki CT score. The scores from the Marshall, Rotterdam, and Helsinki CT scoring
Hospital, Helsinki, Finland methods were added to a clinical model based on age, motor score, and pupils to
evaluate their value in predicting outcome. Internal validity was assessed by a bootstrap
Correspondence:
Rahul Raj, PhD(c), BM, technique and expressed as area under the curve (AUC). Outcome was 6-month
Department of Neurosurgery, unfavorable neurological outcome and mortality.
Helsinki University Hospital,
RESULTS: Variables included in the Helsinki CT score were bleeding type and size, intra-
Topeliuksenkatu 5, PB-266, 00029 HUS,
Finland. ventricular hemorrhage, and suprasellar cisterns. In the present data set, the performance of
E-mail: [email protected], the Helsinki CT score was superior to that of the Marshall CT and Rotterdam CT scores (AUC,
[email protected]
0.74-0.75 vs 0.63-0.70; P , .001). Addition of the Helsinki CT score modestly increased
Received, May 29, 2014.
prognostic performance of the clinical model (AUC neurological outcome 10.02 [P = .002];
Accepted, August 7, 2014. AUC mortality, 10.01 [P = .21]). In contrast, the Marshall and Rotterdam CT scores were of no
Published Online, August 29, 2014. additional predictive value to the clinical model (P . .05).
CONCLUSION: Use of the novel Helsinki CT score improved outcome prediction accu-
Copyright © 2014 by the
Congress of Neurological Surgeons. racy, and the Helsinki CT score is a feasible alternative to the Rotterdam and Marshall CT
systems. External validation of the Helsinki CT score is advocated to show generalizability.
KEY WORDS: Computerized tomography, Helsinki CT, Marshall CT, Prediction model, Prognosis, Rotterdam CT,
Sensitivity and specificity, Traumatic brain injury

Neurosurgery 75:632–647, 2014 DOI: 10.1227/NEU.0000000000000533 www.neurosurgery-online.com

T
he use of prognostic models is becoming
ABBREVIATIONS: AOR, adjusted odds ratio; AUC, increasingly important in traumatic brain
area under the receiver-operating curve; CI, confi-
injury (TBI) research for baseline risk
dence interval; EDH, epidural hematoma; GCS,
Glasgow Coma Scale; GOS, Glasgow Outcome stratification in clinical trials and standardization
Scale; ICH, intracerebral hemorrhage; IMPACT, of case-mix in comparative effectiveness research.1
International Mission for Prognosis and Analysis Clinically, TBI is often classified according to
WHAT IS THIS BOX? of Clinical Trials in TBI; IVH, intraventricular hem- the Glasgow Coma Scale (GCS) into mild,
A QR Code is a matrix orrhage; SDH, subdural hematoma; TBO, traumatic moderate, and severe.2 Although GCS is of
barcode readable by QR brain injury; tSAH, traumatic subarachnoid
scanners, mobile phones
great descriptive value and is one of the
hemorrhage
with cameras, and strongest predictors of outcome in TBI, early
smartphones. The QR Supplemental digital content is available for this article. structural pathological abnormalities detected
Code above links to Direct URL citations appear in the printed text and are by computerized tomography (CT) may be of
Supplemental Digital provided in the HTML and PDF versions of this article on
Content from this the journal’s Web site (www.neurosurgery-online.com).
similar predictive value.3-5 Furthermore, GCS
article. is subject to error resulting from, for instance,

632 | VOLUME 75 | NUMBER 6 | DECEMBER 2014 www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

 THE HELSINKI CT SCORE

alcohol intoxication, sedation and intubation, and inter-rater outpatient clinic follow-up notes with a neurosurgeon or neurologist.
variability.6,7 Thus, CT imaging could offer a more objective Neurological outcome was dichotomized to unfavorable (GOS score 1-3:
and reliable approach to stratify patients with TBI. death, vegetative state, severe disability) and favorable (GOS score 4-5:
Since its introduction in 1991, the Marshall CT classification8 moderate disability, good recovery) outcome.11 GOS score assessment
agreement was good between the authors (k, 0.90; 95% confidence
has become largely accepted for its descriptive and predictive
interval [CI], 0.86-0.95); discrepancies were resolved by verbal discussion.
value. For example, the IMPACT (International Mission for The ethics committee of the Helsinki University Hospital approved
Prognosis and Analysis of Clinical Trials in TBI) prognostic the study and waived the need for informed consent from individual
model applies the Marshall CT score for 6-month outcome patients. The standards for treating TBI in the Helsinki University
prediction in patients with moderate to severe TBI.9 The Hospital follow the Brain Trauma Foundation guidelines.12 The
Marshall CT classification was, however, not designed for methodology of the present study is in accordance with the STARD
outcome prediction, and in 2005, Maas et al10 redesigned it Guidelines (see STARD checklist, Supplemental Digital Content 1,
for 6-month mortality prediction, resulting in the Rotterdam CT available at http://links.lww.com/NEU/A673).13
score. Since its introduction in 2005, few studies have compared
the performance of the Marshall CT classification and the Test Methods
Rotterdam CT score for predicting long-term outcome (neuro- Admission patient characteristics were recorded by emergency depart-
logical and mortality) of patients with TBI. Therefore, we decided ment physicians and were retrieved from electronic records for this
to evaluate the value of the Marshall CT classification and the analysis. Two authors (R.K. and R.R.) jointly classified all admission head
Rotterdam CT score in predicting long-term outcome after TBI. CT images by the Marshall CT classification, by the Rotterdam CT score,
Furthermore, we performed an extensive analysis of early CT and by a set of predefined characteristics: mass lesion type (subdural
abnormalities in an attempt to develop an alternative CT-based hematoma [SDH], epidural hematoma [EDH], intracerebral hemorrhage
prognostic model with improved performance, compared to pre- [ICH]) and size ($25 cm3 or ,25 cm3), presence, location and thickness
of traumatic subarachnoid hemorrhage (tSAH), presence of intraven-
vious CT models. To evaluate the value of the CT scoring methods,
tricular hemorrhage (IVH), status of suprasellar cisterns, status of
we compared their individual performance in predicting 6-month ambiens cisterns, status of fourth ventricle, midline shift (in millimeters),
neurological outcome and 6-month mortality. To gain further and cortical sulcus effacement. The term mass lesion refers to all SDHs,
understanding of their performance, each CT scoring method was EDHs, and ICHs, of any size. The term large mass lesion refers to any
individually added to a clinical prognostic model, based on only age, SDH, EDH, or ICH $25 cm3, and the term small mass lesion refers to
motor score, and pupillary light reactivity.9 We hypothesized that lesions ,25 cm3. The terms ICH and contusion are used as synonyms.
the performance of the Rotterdam CT score would be higher than Mass lesion volume was measured with the ABC/2 method, which has
that of the Marshall CT classification and that the novel CT been shown to have interobserver reliability and to be accurate for both
scoring method would improve predictive accuracy. intraparenchymal and extraparenchymal bleedings.14-16 Patient outcome
was blinded at the time of CT classification.

METHODS Statistical Methods

Differences in categorical variables between patients with good and poor
Participants outcomes were tested with the x2 (2-tailed) test. All continuous data were
We performed an open-cohort retrospective study including consec- highly skewed; hence, the Mann-Whitney U test was used for continuous
utive patients with moderate to severe TBI (admission GCS score, 3-12) data testing. Categorical data are presented as number (percent) and
and complicated mild TBI (admission GCS score, 13-15, requiring continuous as median (interquartile range) unless otherwise mentioned.
intensive care unit admission) admitted to the intensive care unit of a large For the development of the new CT-based prognostic model, we chose
urban level 1 trauma center (Töölö Hospital, Helsinki University significant CT predictors (P , .05) with an incidence of at least 10%
Hospital; catchment area population, approximately 2 million) during from the univariate analysis and inserted them into a backward stepwise
a 4-year period (January 2009-December 2012). Definition of TBI was logistic regression model in order to identify the strongest predictors of
a discharge diagnosis of International Statistical Classification of Diseases outcome. Insignificant predictors rejected by the model were omitted
and Related Health Problems, 10th Edition S06.1-S06.9 caused by an from the final model. Collinearity between included variables was tested
external force. Patients with a history of head trauma but no intracranial by the variance influence factor. To make the model more clinically
pathological findings (by CT imaging) on admission and those with applicable, we created a score chart by rescaling and rounding the
subacute head injuries (.24 hours) were not considered. Further regression coefficients to whole numbers.17 The new CT-based
exclusion criteria were age ,14 years, penetrating head injury, dead on prognostic model is referred to as the Helsinki CT score.
arrival, dead before CT imaging, or intensive care unit admission. The performance of the prediction models was assessed by determining
Outcome was 6-month neurological outcome (by Glasgow Outcome their discrimination (by area under the receiver-operating curve [AUC]),
Scale [GOS]) and 6-month mortality. Non-Finnish citizens were excluded calibration (by calibration slope and intercept), and the explained variation
from the study because their follow-up data and death records may not have (by Nagelkerke pseudo-R [Nagelkerke R2]).18 Discrimination refers to the
been captured in the national electronic healthcare records. Data on ability of the model to differ between patients with good outcome and
mortality were retrieved from the Finnish population register center patients with poor outcome. Generally, the minimum required AUC for
(available for 100% of Finnish patients). Two independent authors (R.R. a prediction model is .0.70. Discrimination between the prediction
and J.S.) retrospectively adjudicated 6-month GOS on the basis of models was compared by use of the Venkatraman test.19 Calibration refers

NEUROSURGERY VOLUME 75 | NUMBER 6 | DECEMBER 2014 | 633

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

RAJ ET AL

to the agreement between observed and predicted outcomes. Well-

calibrated models have a calibration slope of 1 and an intercept of 0.18 The TABLE 2. Description of the Rotterdam Computerized Tomogra-
Nagelkerke R2 is a measure of explained variation and is considered an phy Scorea
overall performance measure: The higher the value is, the higher the
explained variation is, and the better the overall performance is.20 Rotterdam CT Score Score
To assess the usefulness of the CT-based prognostic models (Marshall Basal cisterns
CT classification, Rotterdam CT score, and Helsinki CT score) for Normal 0
outcome prediction, we first compared their AUCs with one another. Compressed 1
To further assess the usefulness of the CT-based models, we added them Absent 2
to a clinical prediction model based on age, motor score, and pupillary Midline shift
reactivity (IMPACT core) and compared the gains in AUC. The No shift or # 5 mm 0
IMPACT core is a prognostic model developed from 8509 patients from Shift . 5 mm 1
the IMPACT study, which was a collaborative study based on data from Epidural mass lesion
8 randomized controlled trials and 3 observational studies.9 The Present 0
Absent 1
Helsinki CT score, Marshall CT classification, and Rotterdam CT
IVH or tSAH 0
score are described in Tables 1 through 3.
Absent 0
The performance of a prognostic model is dependent largely on the
Present 1
underlying patient case-mix for which it is used. To improve the Sum score 11
performance of a prognostic model and to make it applicable for a new
setting, the prognostic models should ideally be customized.21-23 There
a
are 2 methods for model customization: first-level customization and CT, computerized tomography; IVH, intraventricular hemorrhage; tSAH, traumatic
subarachnoid hemorrhage.
second-level customization. First-level customization involves fitting
a new logistic regression equation with the observed outcome as the
dependent variable and the logit-transformed original prediction as the
independent variable. Thus, in first-level customization, the influence of
as the dependent variable and the original predictor as the independent
the individual variables does not change but rather calibrates their joint
variables. Internal validity of all prognostic models was assessed with an
effect on outcome. Second-level customization involves fitting all
optimized, corrected, 500-sample bootstrap technique.24 The equations
original predictors into a logistic regression model with outcome as
for the customized prediction models are found in Supplemental
the dependent variable. Second-level customization has been shown to
Digital Content 2 (http://links.lww.com/NEU/A674, which gives the
be more effective than first-level customization and should preferably be
customized equations for the Marshall CT, Rotterdam CT, and
used if the sample size is sufficient.22 For the present study, we used
IMPACT core models for predicting 6-month mortality and unfavorable
second-level customization of the Marshall CT, Rotterdam CT, and
outcome in the current data set).
IMPACT core models in which a new logistic regression model was
IBM SPSS Statistics for Windows, version 21.0 (IBM Corp, Armonk,
fitted with the outcome (6-month unfavorable outcome and mortality)
New York; released 2012) and R: A Language and Environment for
Statistical Computing (R-Foundation for Statistical Computing, Vienna,
Austria; 2013) were used for the statistical analyses. Receiver-operating
characteristics curves were compared by use of the pROC25 library, and
TABLE 1. Description of the Marshall Computerized Tomography internal validation was performed with the rms library.
Classificationa
Marshall CT RESULTS
Classification Definition
Participants
DI I No visible intracranial pathology seen on CT
DI II Cisterns present with midline shift 0-5 mm and/or Of a total of 923 eligible patients admitted during the study time,
lesion densities present; no high- or mixed- 54 patients were lost to follow-up, and the remaining 869 patients
density lesion .25 cm3; may include bone
fragments and foreign bodies
DI III Cisterns compressed or absent with midline shift
of 0-5 mm; no high- or mixed-density lesion TABLE 3. Description of the International Mission for Prognosis
.25 cm3 and Analysis of Clinical Trials in TBI Core Modela
DI IV Midline shift . 5 mm; no high- or mixed-density IMPACT Core Variables
lesion . 25 cm3
Evacuated mass Any lesion surgically evacuated Age In years
lesion Motor score Obeys, localizes, normal flexion, abnormal flexion,
Nonevacuated High- or mixed-density lesion . 25 cm3; not extension, none
mass lesion surgically evacuated Pupillary light Both react, one reacts, none reacts
reactivity
a a
CT, computerized tomography; DI, diffuse injury. IMPACT, International Mission for Prognosis and Analysis of Clinical Trials in TBI.

634 | VOLUME 75 | NUMBER 6 | DECEMBER 2014 www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

 THE HELSINKI CT SCORE

constituted the study population (Figure 1). There were no outcome (P , .001 for both). Markers of raised intracranial
significant differences in age (P = .29), motor score (P = .48), pressure, for instance, status of suprasellar and ambiens cisterns,
or pupillary reactivity (P = .12) between included patients and midline shift, and cortical sulcus effacement, showed a significant
patients lost to follow-up. Patient baseline characteristics are shown association with poor outcome (P , .001). Only 3% of patients had
in Table 4. Median patient age was 57 years (interquartile range, an abnormal fourth ventricle (blood filled or compressed).
43-68 years). Patients with favorable outcome were a median of 13
years younger than patients with an unfavorable outcome (50 vs 63 The Helsinki CT Score
years; P , .001). Of all patients, 32% were considered to have
Stepwise logistic regression found the following: mass lesion size
mild complicated TBI, 24% were considered to have moderate
$25 cm3 (adjusted odds ratio [AOR], 1.71; 95% CI, 1.21-2.41;
TBI, and 44% were considered to have severe TBI. Overall
P = .002), presence of SDH (AOR, 2.06; 95% CI, 1.41-3.022;
6-month unfavorable outcome was 48% (n = 414 of 869), and
P , .001), presence of EDH (AOR, 0.34; 95% CI, 0.19-0.63; P =
6-month mortality was 25% (n = 219 of 869). Univariate analysis
.001), presence of ICH (AOR, 1.69; 95% CI, 1.24-2.29;
found virtually all baseline characteristics, except hypotension (P =
P = .001), presence of IVH (AOR, 2.41; 95% CI, 1.51-3.89;
.51), to be significantly (P , .05) associated with outcome.
P , .001), and compressed or obliterated suprasellar cisterns
(compressed: AOR, 1.38; 95% CI, 0.96-1.98; obliterated: AOR,
Individual CT Characteristics 5.55; 95% CI, 3.39-9.12) to be the strongest predictors of
Patient admission CT characteristics are shown in Table 5. Ninety- outcome (see Figure, Supplemental Digital Content 3, http://
one percent of all patients had a detectable mass lesion (SDH, EDH, links.lww.com/NEU/A675, which is a graphical illustration of
ICH of any size) on the admission head CT; 9% of patients had the logistic regression analysis with ORs and 95% CIs; top panel
a traumatic non–mass lesion hemorrhage (tSAH or IVH) as the shows ORs for 6-month mortality, and bottom panel shows ORs
main pathological finding. SDH and ICH were seen more for 6-month unfavorable outcome [ORs .1.0 indicate a higher
frequently among patients with unfavorable outcome (P , .001 likelihood of the outcome; SS, suprasellar]). Predictors rejected
and P = .02, respectively). In contrast, EDH was seen more by the model were cortical sulcus effacement (P = .87), status of
frequently among patients with a favorable outcome (P , .001). the ambiens cisterns (P = .74), midline shift (P = .32), abnormal
Petechial ICH and a higher number of contusions also showed fourth ventricle (P = .17), and tSAH (P = .10). There was no
a significant association with unfavorable outcome. tSAH and IVH significant collinearity in the final model (maximum variance
were significantly associated with an increased risk of unfavorable influence factor = 1.39). To make the score more clinical
applicable, we converted the regression coefficients to scores.
The Helsinki CT score is summarized in Table 6 and portrayed
in Figures 2-7. The Helsinki CT score ranges from a minimum
of 23 points to a maximum of 14 points.
The Helsinki CT score may be converted to risk of 6-month
unfavorable outcome,
h i
1= 1 1 e 2 ð2 1:636 1 0:319 · HELSINKI CT SCOREÞ ;

and risk of 6-month mortality,

h i
1= 1 1 e 2 ð2 2:666 1 0:287 · HELSINKI  CT  SCOREÞ :

The Helsinki CT score prognosis for 6-month outcome ranges

from 3% and 7% to 79% and 94% for mortality and unfavorable
outcome, respectively. The concordance between the Helsinki CT
score and predicted and observed outcome is shown in Figure 8.
An Excel Spreadsheet calculator of the Helsinki CT score and the
clinical model is provided in Supplemental Digital Content 4
(http://links.lww.com/NEU/A676).

Comparison of the Individual CT Scores and the

IMPACT Core Model
FIGURE 1. Study population flowchart showing included and
excluded patients. Comparisons of the mean Helsinki CT score with the Marshall
CT classification and the Rotterdam CT score are shown in Figure 9

NEUROSURGERY VOLUME 75 | NUMBER 6 | DECEMBER 2014 | 635

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

RAJ ET AL

TABLE 4. Patient Baseline Characteristicsa

Variable All Patients (n = 869) Favorable Outcome (n = 455) Unfavorable Outcome (n = 414) P Value
Age, y 57 (43-68) 50 (33-63) 63 (53-73) ,.001
Glasgow Coma Scale score, n (%)
3-8 379 (44) 139 (31) 240 (58) ,.001
9-12 206 (24) 112 (25) 94 (23)
13-15 284 (32) 204 (44) 80 (19)
Motor score, n (%)
Obeys/localizes 556 (64) 45 (10) 138 (33) ,.001
Normal/abnormal flexion 130 (15) 59 (13) 71 (17)
Extension/none 183 (21) 351 (77) 205 (50)
Pupillary light reaction, n (%)
Both react 644 (74) 398 (87) 246 (60) ,.001
1 Reacts 101 (12) 34 (8) 67 (16)
None reacts 124 (14) 23 (5) 101 (24)
Hypoxia, n (%)b 132 (15) 50 (11) 82 (20) ,.001
Hypotension, n (%)b 66 (8) 32 (7) 34 (8) .51
tSAH, n (%) 504 (58) 248 (55) 256 (62) .03
Epidural hematoma, n (%) 90 (10) 71 (16) 19 (5) ,.001
Marshall CT classification, n (%)
DI I NA NA NA
DI II 282 (33) 202 (44) 80 (19) ,.001
DI III 37 (4) 19 (4) 18 (4)
DI IV 30 (4) 14 (3) 16 (4)
EML/NEML 520 (60) 220 (48) 300 (73)
Rotterdam CT score, n (%)
1 31 (4) 29 (6) 2 (1) ,.001
2 150 (17) 92 (20) 58 (14)
3 303 (35) 192 (42) 111 (27)
4 176 (20) 96 (21) 80 (19)
5 147 (17) 39 (9) 108 (26)
6 62 (7) 7 (2) 55 (13)
Glucose, mmol/L 7.3 (6.1-8.9) 7.1 (6.0-8.4) 7.6 (6.2-9.4) ,.001
Hemoglobin, g/dL 12.5 (11.1-14.9) 13.0 (11.7-14.2) 11.9 (10.7-13.2) ,.001

a
CT, computerized tomography; DI, diffuse injury; EML, evacuated mass lesion; IMPACT, International Mission for Prognosis and Analysis of Clinical Trials in TBI; NEML,
nonevacuated mass lesion; tSAH, traumatic subarachnoid hemorrhage. Categorical data presented as n (%); continuous data are presented as median (interquartile range).
b
Hypoxia defined as saturation , 90% at any time during the prehospital period; hypotension, as systolic blood pressure , 90 mm Hg at any time during the prehospital period.

and in Supplemental Digital Contents 5 and 6, respectively contrast, all CT scores had significantly lower AUCs than the
(http://links.lww.com/NEU/A677 and http://links.lww.com/ IMPACT core model (AUCunfavorable, 0.81, AUCmortality, 0.83;
NEU/A678). P , .001).
Internal validations of the customized prediction models are The IMPACT core model had the highest explanatory variation
shown in Table 7. The IMPACT core displayed highest (Nagelkerke R2 = 0.37 and 0.37 for neurological outcome and
discrimination (AUCunfavorable, 0.81; AUCmortality, 0.84), fol- mortality, respectively), followed by the Helsinki CT score
lowed by the Helsinki CT score (AUCunfavorable, 0.75; (Nagelkerke R2 = 0.25 and 0.20), the Rotterdam CT score
AUCmortality, 0.75), the Rotterdam CT score (AUCunfavorable, (Nagelkerke R2 = 0.16 and 0.15), and the Marshall CT
0.68; AUCmortality, 0.70), and the Marshall CT classification classification (Nagelkerke R2 = 0.09 and 0.09). All models
(AUCunfavorable, 0.63; AUCmortality, 0.64). The Rotterdam CT showed satisfactory calibration slopes (0.979-1.017) and inter-
score displayed significantly better discrimination than the cepts (0.001-0.017).
Marshall CT classification (AUCunfavorable, 0.68 vs 0.63, P ,
.001; AUCmortality, 0.70 vs 0.64, P , .001). The Helsinki CT
score, on the other hand, displayed significantly better Prognostic Value Gained by CT Scores
discrimination than the Rotterdam CT score (AUCunfavorable, All IMPACT core and CT score combined models showed good
0.75 vs 0.68, P , .001; AUCmortality, 0.75 vs 0.70, P = .006). In discrimination, with AUCs .0.80 for both unfavorable outcome

636 | VOLUME 75 | NUMBER 6 | DECEMBER 2014 www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

 THE HELSINKI CT SCORE

TABLE 5. Patient Computerized Tomography Characteristicsa

Variable All Patients (n = 869) Favorable Outcome (n = 455) Unfavorable Outcome (n = 414) P Value
Mass lesions, n (%)
Subdural hematoma 614 (71) 269 (59) 345 (83) ,.001
Epidural hematoma 90 (10) 71 (16) 19 (5) ,.001
Intracerebral hemorrhage (contusion) 475 (55) 231 (51) 244 (59) .02
Petechial hemorrhage 117 (14) 71 (16) 46 (11) .05
No. of contusions 1 (0-2) 1 (0-2) 1 (0-2) .002
Dominant lesion size, cm3
$25 514 (59) 215 (47) 299 (72) ,.001
,25 279 (32) 194 (43) 85 (21)
SAH or IVH (not measurable)b 76 (9) 46 (10) 93 (7)
Non–mass lesion hemorrhageb, n (%)
Traumatic SAH or IVH 533 (61) 261 (57) 272 (66) .01
Traumatic SAH
No 365 (42) 207 (45) 158 (38) .03
Yes 504 (58) 248 (55) 256 (62)
Traumatic SAH location
No 365 (42) 207 (45) 158 (38) ,.001
Cortical sulcus 323 (37) 179 (39) 144 (35)
Basal cisterns 11 (1) 5 (1) 6 (1)
Both 170 (20) 64 (15) 106 (26)
Traumatic SAH in basal cisterns
No 687 (79) 385 (85) 302 (72) ,.001
Thin (#1 mm) 98 (11) 42 (9) 56 (14)
Thick (.1 mm) 84 (10) 28 (6) 56 (14)
IVH 111 (13) 34 (8) 77 (19) ,.001
Markers of raised intracranial pressure
Suprasellar cisterns, n (%)
Normal 513 (59) 327 (72) 186 (45) ,.001
Compressed 217 (25) 100 (22) 117 (28)
Absent 139 (16) 28 (6) 111 (27)
Ambiens cisterns, n (%)
Normal 540 (62) 341 (75) 199 (48) ,.001
Compressed 175 (20) 78 (17) 97 (23)
Absent 154 (18) 36 (8) 118 (29)
Suprasellar and ambiens cisterns, n (%)
Both open 490 (56) 314 (69) 176 (43) ,.001
At least 1 compressed 247 (29) 117 (26) 130 (31)
Both absent 132 (15) 24 (5) 108 (26)
Midline shift, mm 4 (0-10) 2 (0-7) 6 (0-14) ,.001
Midline shift, n (%)
No shift 337 (38) 226 (49) 111 (26) ,.001
1-5 mm 173 (20) 84 (19) 89 (22)
6-10 mm 143 (17) 76 (17) 67 (16)
.10 mm 216 (25) 69 (15) 147 (36)
Cortical sulcus effacement, n (%)
No effacement 296 (34) 196 (43) 100 (24) ,.001
Unilateral effacement 327 (38) 169 (37) 158 (38)
Bilateral effacement 246 (28) 90 (20) 156 (38)
Abnormal fourth ventriclec 24 (3) 2 (0) 22 (5) ,.001
a
IVH, intraventricular hemorrhage; SAH, subarachnoid hemorrhage. Categorical data presented as n (%); continuous data, as median (interquartile range).
b
Traumatic subarachnoid hemorrhage or intraventricular hemorrhage.
c
Compressed, obliterated, or blood filled.

and mortality prediction. Addition of the Marshall CT classifi- (AUC, 10.00 for unfavorable outcome and mortality; P . .05;
cation or the Rotterdam CT score did, however, not significantly Table 8). In contrast, addition of the Helsinki CT score to the
improve the discriminative power of the IMPACT core model IMPACT core model resulted in significantly higher discrimination

NEUROSURGERY VOLUME 75 | NUMBER 6 | DECEMBER 2014 | 637

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

RAJ ET AL

(AUC, 10.01; Nagelkerke R2, 10.02). In addition, all combined

TABLE 6. The Helsinki Computerized Tomography Score Chart IMPACT–CT score models showed good calibration, with
Variable Score
calibration slopes between 0.974 and 0.982 and intercepts between
20.021 and 0.004.
Mass lesion type(s)
Subdural hematoma 2
Intracerebral hematoma 2 DISCUSSION
Epidural hematoma 23
Mass lesion size . 25 cm3 2 Key Findings
Intraventricular hemorrhage 3 We conducted a large, open-cohort, retrospective, single-center
Suprasellar cisterns study investigating the usefulness of the Marshall CT classification
Normal 0
and Rotterdam CT score in predicting long-term outcome in patients
Compressed 1
Obliterated 5 with mild complicated or moderate to severe TBI. Furthermore, we
Sum score 23 to 14 conducted an extensive analysis of early CT-detectable abnormalities,
which led to the development of a novel CT scoring system, the
The probability of 6-month outcome is defined as 1/(1 1 e 2LP ), where Helsinki CT score. Individually, the Marshall CT and Rotterdam CT
LPMortality = 22.666 1 0.287 · Helsinki computerized tomography score and
LPUnfavorable outcome = 21.636 1 0.319 · Helsinki computerized tomography
scores were of limited value in predicting outcome (AUC, # 0.65 and
score. The Helsinki Computerized Tomography score ranges from a minimum 0.70, respectively), whereas the novel, more detailed Helsinki CT
of 23 to a maximum of 14 points. score showed significantly higher performance (AUC, 0.75).
Furthermore, addition of the Marshall CT and Rotterdam CT
scores to the IMPACT model did not result in any gained
(P = .002) and explanatory variation for unfavorable outcome performance. In contrast, addition of the Helsinki CT score
prediction (AUC, 10.02; Nagelkerke R2, 10.05). On the other improved unfavorable neurological outcome (AUC, 10.02; P =
hand, only a slight improvement in performance, not reaching .002) and mortality prediction (AUC, 10.01; P = .21), although the
statistical significance (P = .21), was noted for mortality prediction latter did not reach statistical significance. The present study also
after the Helsinki CT score was added to the IMPACT core model shows that early CT features alone may be of limited predictive value

FIGURE 2. Subdural hematoma classification. Left, a large ($25 cm3) right-sided subdural hematoma (white arrow).
Marshall computerized tomography (CT) class, evacuated mass lesion (EML) class; Rotterdam CT score, 4 of 6 (11 midline shift
. 5 mm, 11 absence of epidural hematoma, 11 traumatic subarachnoid hemorrhage or intraventricular hemorrhage, and 11
extra point); Helsinki CT score, 7 of 14 (12 mass lesion size $25 cm3, 12 subdural hematoma, 13 intraventricular
hemorrhage). Right, a small (,25 cm3) left-sided subdural hematoma (white arrow). Asterisk marks the deep brain stimulation
electrodes in a patient with Parkinson disease. Marshall CT class, diffuse injury (DI) II; Rotterdam CT score, 2 of 6 (11 absence
of epidural hematoma, 11 extra point); Helsinki CT score 2 of 14 (12 subdural hematoma).

638 | VOLUME 75 | NUMBER 6 | DECEMBER 2014 www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

 THE HELSINKI CT SCORE

FIGURE 3. Epidural hematoma classification. Left, a small (,25 cm3) left-sided epidural hematoma. Marshall computerized
tomography (CT), class, diffuse injury (DI) II; Rotterdam CT score, 1 of 6 (11 extra point); Helsinki CT score, 23 of 14 (23 epidural
hematoma). Right, a large ($25 cm3) left-sided epidural hematoma. Marshall CT class, evacuated mass lesion (EML); Rotterdam CT,
score 2 of 6 (11 midline shift . 5 mm, 11 extra point); Helsinki CT score, 21 of 14 (23 epidural hematoma, 12 large mass lesion).

FIGURE 4. Intracerebral hemorrhage classification. Left, a small (,25 cm3) right-sided frontal intracerebral hemorrhage (used
synonymously with contusion). Marshall computerized tomography (CT) class, diffuse injury (DI) IV; Rotterdam CT score, 3 of 6
(11 midline shift . 5 mm, 11 epidural hematoma absent, 11 extra point); Helsinki CT score, 2 of 14 (12 intracerebral
hemorrhage). Right, a large right-sided intracerebral hemorrhage with intraventricular hemorrhage and a thin right-sided
subdural hematoma. Marshall CT class, nonevacuated mass lesion (NEML); Rotterdam CT score, 4 of 6 (11 midline shift . 5
mm, 11 epidural hematoma absent, 11 intraventricular hemorrhage, 11 extra point); Helsinki CT score, 9 of 14 (12
subdural hematoma, 12 intracerebral hemorrhage, 12 large mass lesion, 13 intraventricular hemorrhage).

NEUROSURGERY VOLUME 75 | NUMBER 6 | DECEMBER 2014 | 639

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

RAJ ET AL

FIGURE 5. Classification of suprasellar cisterns. White arrow shows normal suprasellar cisterns (left), compressed suprasellar cisterns (middle), and obliterated suprasellar
cisterns (right). Note the frontal pneumocephalus in the left image, indicating a skull base fracture.

in patients with TBI and that prognostic accuracy is considerably value.4,10,26-29 In contrast, we found both the Marshall CT
improved with the addition of core clinical data. classification and the Rotterdam CT score to show modest
discrimination for both 6-month mortality and unfavorable
outcome prediction (Rotterdam CT: AUC, 0.70 mortality ,
Interpretation and Comparison With Previous Studies AUCunfavorable, 0.68; Marshall CT: AUC mortality, 0.64,
Several studies have found both the Marshall CT classifica- AUCunfavorable, 0.63, respectively). The Marshall CT classi-
tion and the Rotterdam CT score to be of good predictive fication, introduced in 1991, was developed on the basis of

FIGURE 6. Comparison of the Marshall computerized tomography (CT) classification, Rotterdam CT score, and Helsinki CT
score in a patient example with large subdural hematoma, obliterated suprasellar cisterns, and intraventricular hemorrhage. The
2 images are from the same patient with a large subdural hematoma. Marshall CT class, evacuated mass lesion (EML); Rotterdam
CT score, 6 of 6 (12 absent basal cisterns, 11 midline shift $ 5 mm, 11 absence of epidural hematoma, 11 presence
of traumatic subarachnoid or intraventricular hemorrhage, 11 extra point); Helsinki CT score, 12 of 14 (12 mass lesion size $
25 cm3, 12 subdural hematoma, 13 intraventricular hemorrhage, 15 obliterated suprasellar cisterns).

640 | VOLUME 75 | NUMBER 6 | DECEMBER 2014 www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

 THE HELSINKI CT SCORE

FIGURE 7. Traumatic subarachnoid hemorrhage and intraventricular hemorrhage classification. Left, a traumatic subarachnoid hemorrhage in the basal cisterns and
cortical sulci (*). Marshall computerized tomography (CT) class, diffuse injury (DI) II; Rotterdam CT score, 3 of 6 (11 absence of epidural hematoma, 11 traumatic
subarachnoid hemorrhage, 11 extra point); Helsinki CT score, 5 of 14 (12 intracerebral hemorrhage, 13 intraventricular hemorrhage). Middle, blood-filled fourth
ventricle. Marshall CT class, diffuse injury II; Rotterdam CT score, 3 of 6 (11 absence of epidural hematoma, 11 intraventricular hemorrhage, 11 extra point); Helsinki CT
score, 5 of 14 (12 intracerebral hemorrhage, 13 intraventricular hemorrhage). Right, extensive intraventricular hemorrhage in the lateral ventricles. Marshall CT class,
diffuse injury II; Rotterdam CT score, 3 of 6 (11 absence of epidural hematoma, 11 intraventricular hemorrhage, 11 extra point); Helsinki CT score, 3 of 14 (13
intraventricular hemorrhage).

746 patients with severe TBI admitted in 1984 to 1987 whose Rotterdam CT score does not differentiate between type and size
data were collected to the Traumatic Coma Data Bank. 8 The of mass lesion (with the exception of EDH). Recent findings from
Marshall CT classification is based on 3 key variables: the IMPACT study found both SDH and ICH to be strong
presence of mass lesion, midline shift, and status of the predictors of poor outcome and presence of EDH to be
perimesencephalic cisterns. In 2005, Maas et al 10 refined the significantly associated with favorable outcome.4 Correspond-
Marshall CT classification by introducing the Rotterdam CT ingly, we found both the presence of SDH and ICH to be of
score, which is based on 2269 patients with moderate and equal predictive performance of poor outcome and the presence
severe TBI from the tirilazad trials. In contrast to the of EDH to decrease likelihood of poor outcome. In the Helsinki
Rotterdam CT score, the Marshall CT classification was CT score, presence of EDH gives minus points. In comparison,
not designed for predictive purposes, which may explain in the Rotterdam CT score gives plus points for the absence (not
part the poorer performance of the Marshall CT classification presence) of EDH. We chose to give minus points for EDH to
in the present study. Furthermore, the Rotterdam CT score avoid confusion between the presence of SDH and ICH and the
was developed for 6-month mortality prediction, not unfavor- absence EDH. Thus, the Helsinki CT score ranges from 23 to
able neurological outcome prediction, which could confound 14 points (alternatively, from 0 to 17 points if one were to give
our results. However, to allow proper comparison with the plus points for the absence of EDH).
novel Helsinki CT score, we conducted a second-level The definition of large mass lesion as $25 cm3 and the
customization of the Marshall CT classification and the dichotomization of midline shift to #5 vs .5 mm originates
Rotterdam CT score, adapting them for the present study back to the Marshall CT classification. Although it was recently
setting.21-23 The customized CT scores are found in Supple- shown that these cutoff values may be of limited predictive value,
mental Digital Content 2 (http://links.lww.com/NEU/A674). we chose to use the same definitions to allow comparisons
After performing an extensive analysis of early CT abnormalities between the scoring systems.30 We found mass lesion size, but
detected by CT, we found 4 main variables to predict long-term not midline shift, to be a significant predictor of outcome. In an
outcome: mass lesion type and size, presence of IVH, and status of attempt to improve the predictive weight of midline shift, we
suprasellar cisterns. Our results are similar to those of the included it as a continuous variable and as a more detailed
Rotterdam CT study with some exceptions. For example, the categorization (0, 1-5, 6-10, .10 mm) for the stepwise regression

NEUROSURGERY VOLUME 75 | NUMBER 6 | DECEMBER 2014 | 641

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

RAJ ET AL

FIGURE 8. Concordance between observed and predicted outcome for the Helsinki computerized tomography (CT) score. Top, concordance between
observed and predicted 6-month mortality. Bottom, concordance between observed and predicted 6-month unfavorable outcome.

model. However, midline shift remained insignificant and was a recent study, the ambiens cistern was found to be the cistern with
excluded from the final CT score. Midline shift remained the strongest predictive value.31 However, after adjustment for
insignificant probably because it was improved by surgical other CT characteristics, we found the status of the suprasellar, but
decompression and hematoma evacuation. Thus, it is likely that not the ambiens, cistern to be significantly associated with
postoperative rather than preoperative midline shift is a stronger outcome. Thus, ambiens cisterns were omitted from the final
predictor of outcome, although this could not be verified in the model, whereas suprasellar cisterns remained included. The status
present study. of the pentagonal suprasellar cisterns is divided into normal,
Most previous studies do not differ between the individual compressed, and obliterated (absent). To facilitate the interpreta-
cerebral cisterns but use the term basal cisterns. Although most tion of forthcoming studies using the Helsinki CT score, the
studies refer to the perimesencephalic cisterns when referring to the suprasellar cisterns classification is illustrated in Figure 5.
basal cisterns, no clear definition exists. We looked at the status of 2 tSAH and IVH have previously been shown to be strong
specific cisterns: the ambiens cistern and the suprasellar cisterns. In predictors of outcome in TBI.4,10,32,33 Similarly, in univariate

642 | VOLUME 75 | NUMBER 6 | DECEMBER 2014 www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

 THE HELSINKI CT SCORE

FIGURE 9. Helsinki computerized tomography (CT) score vs Marshall CT classification and Helsinki CT score vs Rotterdam CT score. Shown are the corresponding mean
Helsinki CT scores for each Marshall CT class group (left) and each Rotterdam CT score group (right). For example, patients with a Marshall CT class IV had a mean Helsinki
CT score of 5.1 compared with patients with a Marshall class nonevacuated mass lesion (NEML) who had a mean Helsinki CT score of 6.5. Similarly, patients with
a Rotterdam CT score of 3 had a mean Helsinki CT score of 3.3, whereas patients with a Rotterdam CT score of 6 had a mean Helsinki CT score of 10.8. The Marshall CT
class and Rotterdam CT score were assessed using the original definitions. DI, diffuse injury; EML, evacuated mass lesion.

analysis, we found a significant association between tSAH and Since the introduction of the Marshall CT classification and
poor outcome. However, after adjustment for other CT the Rotterdam CT score in 1991 and 2005, respectively, 2 new
characteristics in multivariate analysis, tSAH was not an CT prediction models have been introduced: the Stockholm CT
independent predictor and was left out of the final model. score in 2011 and the Radboud CT models in 2013.29,36 These
To further investigate the role of tSAH on outcome, we models have shown good performance in the development data
categorized it by presence in cortical sulci only, in basal set, but, like the Helsinki CT score, they lack external
cisterns only, or in both cortical sulci and basal cisterns. Even validation. Similar to the Helsinki CT score, both the Stock-
after this categorization, tSAH remained insignificant after holm CT score and the Radboud CT models are based on
adjustment for the other CT predictors. Our findings are logistic regression. However, logistic regression may not be the
controversial in that, for example, the IMPACT study found best method of presenting a prognostic model, which might
tSAH to be the strongest CT predictor of poor outcome.4 We limit the practicality of these CT scores. Thus, to make the
believe that differentiating between tSAH and IVH may Helsinki CT score more clinically applicable, we transformed the
explain this finding and suggest that the presence of IVH may regression coefficients to a score chart, similar to the Rotterdam
be a stronger predictor of outcome than tSAH, probably as CT score.10,17 This, we hope, will facilitate the use of our CT
a result of the higher risk of posttraumatic hydrocephalus after scoring system. To further expedite the use of the Helsinki CT
IVH.34,35 score, illustrative examples are shown in Figures 2 through 7, and

TABLE 7. Predictive Value of the Customized Prediction Modelsa

Area Under the Curve Nagelkerke R2 Calibration
Models Apparent Optimism Corrected Apparent Optimism Corrected Slopeb Interceptb
6-mo unfavorable outcome
Marshall CT 0.632 0.632 0.093 0.092 1.017 0.001
Rotterdam CT 0.682 0.683 0.156 0.155 1.011 0.005
Helsinki CT 0.750 0.749 0.249 0.246 1.001 0.003
IMPACT core 0.810 0.807 0.378 0.370 0.982 20.009
6-mo mortality
Marshall CT 0.635 0.635 0.087 0.086 1.016 0.017
Rotterdam CT 0.699 0.698 0.153 0.151 1.007 0.013
Helsinki CT 0.744 0.746 0.203 0.199 1.010 0.006
IMPACT core 0.835 0.832 0.382 0.374 0.979 20.019

a
AUC, area under the curve; CT, computerized tomography; IMPACT, International Mission for Prognosis and Analysis of Clinical Trials in TBI.
b
Only optimism-corrected calibration slope is reported because the apparent slope of all models is one following second-level customization.

NEUROSURGERY VOLUME 75 | NUMBER 6 | DECEMBER 2014 | 643

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

RAJ ET AL

TABLE 8. Added Predictive Ability of the International Mission for Prognosis and Analysis of Clinical Trials in TBI Core Model After Adding the
Different Computerized Tomography Scoring Systemsa
Area Under the Curve Nagelkerke R2 Calibration Slope
Models Apparent Optimism Corrected Apparent Optimism Corrected Slopeb Interceptb
6-mo unfavorable outcome
IMPACT core 1 Marshall CT 0.813 0.809 0.385 0.376 0.980 20.009
IMPACT core 1 Rotterdam CT 0.816 0.813 0.392 0.394 0.980 0.004
IMPACT core 1 Helsinki CT 0.831 0.827 0.425 0.415 0.982 20.005
6-mo mortality
IMPACT core 1 Marshall CT 0.838 0.833 0.391 0.379 0.974 20.021
IMPACT core 1 Rotterdam CT 0.837 0.833 0.392 0.382 0.980 20.018
IMPACT core 1 Helsinki CT 0.843 0.838 0.408 0.395 0.981 20.014

a
AUC, area under the curve; CT, computerized tomography; IMPACT, International Mission for Prognosis and Analysis of Clinical Trials in TBI.
b
Only optimism-corrected calibration slope is reported because the apparent slope of all models is one following second-level customization.

an Excel Spreadsheet Calculator presented is in Supplemental validity of the Helsinki CT score should be assessed in
Digital Content 4 (http://links.lww.com/NEU/A676). independent settings. Second, the Helsinki CT score includes
a total of 6 variables (although only 4 main variables: lesion type
Future Implications and size, IVH, and suprasellar cisterns) and may be considered
more complex than the Rotterdam or Marshall scores. The
The use of prognostic models is an important tool in TBI
increased level of complexity may decrease interobserver
research for baseline risk stratification in clinical trials and
reliability. In the present study, 2 authors jointly classified the
standardization of case-mix in comparative effectiveness
CT images, which did not cause any interobserver discrep-
research.1 As of today, the IMPACT prognostic model is the
ancies. However, interobserver reliability should be considered
most robustly validated and clinically applicable model.37-40
in the validation of the Helsinki CT score, especially for
The IMPACT model uses the Marshall CT classification and
assessing the status of suprasellar cisterns, because it is both the
presence of EDH and tSAH as the main CT predictors. We
strongest predictor and probably the most user-dependent
found the Marshall CT classification to exhibit the lowest
variable. To aid interpretation of the suprasellar cisterns,
predictive value of the 3 tested CT scoring systems. However,
illustrative examples are presented in Figure 5. Third, because
substituting the Marshall CT classification with the Rotter-
of the retrospective nature of the study, we had to assess
dam CT score did not result in any gained predictive
neurological outcome on the basis of outpatient clinical
performance of the IMPACT core model. We acknowledge
follow-up, which may be a source of interpretation bias.
the descriptive purpose of the Marshall CT classification and
Furthermore, 54 patients were lost to follow-up, which
recommend that it should be used as such. However, for
reduced our study population by 6%. However, there were
prognostic purposes, the Marshall CT classification may be of
no statistically significant differences in clinical baseline
limited value, and more sophisticated CT scoring systems
characteristics between included patients and those excluded
such as the Helsinki, Rotterdam, Stockholm, and Radboud
because of missing outcome data. Hence, we do not believe
models should be considered when recalibrating the IMPACT
this to be a source of bias. Fourth, we had a study population of
model. Further validation studies of these CT models in
869, which is significantly smaller than the development
independent data sets are advocated to help guide which
sample of the Rotterdam CT score (2269 patients), which may
model to use.
limit the robustness of our findings.10 However, we do not
The Helsinki CT score is a novel method for predicting long-
believe this to be a major limitation of the present study,
term outcome in patients with TBI. The model should be validated
considering that the Marshall CT classification (746 patients)
in other settings with possible coefficient updating and re-evaluation
and 2 other recently introduced CT prognostic models, the
for other possible strong predictors that we found to be insignificant
Radboud CT models (605 patients) and the Stockholm CT
in the present study such as tSAH, midline shift, status of ambiens
score (861 patients), were developed using patient sample sizes
cisterns, and an abnormal fourth ventricle.4,36
comparable to our sample size.8,29,36 Fifth, we included
patients with mild complicated, moderate, and severe TBI.
Limitations In contrast, the Marshall CT classification and Rotterdam
There are some limitations of the present study. First, our CT score were designed primarily for patients with moderate
data originate from one single center, and thus, the external and severe TBI, which may confound our results. Sixth,

644 | VOLUME 75 | NUMBER 6 | DECEMBER 2014 www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

 THE HELSINKI CT SCORE

comparison of an internally developed model with externally 12. Brain Trauma Foundation, American Association of Neurological Surgeons,
Congress of Neurological Surgeons. Guidelines for the management of severe
developed models will always favor the internal model. Thus,
traumatic brain injury, XV: steroids. J Neurotrauma. 2007;24(suppl 1):
part of the reason why the performance of the Helsinki CT score S91-S95.
was higher than for the Rotterdam and Marshall CT scores may 13. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate
be that all models were tested on the very population used to reporting of studies of diagnostic accuracy: the STARD Initiative. Ann Intern Med.
2003;138(1):40-44.
develop the Helsinki CT score. Although customization of the 14. Broderick JP, Brott TG, Duldner JE, Tomsick T, Huster G. Volume of
Rotterdam and Marshall CT scores reduced this advantage, it intracerebral hemorrhage: a powerful and easy-to-use predictor of 30-day
probably did not fully compensate for it. mortality. Stroke. 1993;24(7):987-993.
15. Kothari RU, Brott T, Broderick JP, et al. The ABCs of measuring intracerebral
hemorrhage volumes. Stroke. 1996;27(8):1304-1305.
CONCLUSION 16. Gebel JM, Sila CA, Sloan MA, et al. Comparison of the ABC/2 estimation
technique to computer-assisted volumetric analysis of intraparenchymal and
The use of the Helsinki CT score significantly improved subdural hematomas complicating the GUSTO-1 trial. Stroke. 1998;29(9):
1799-1801.
outcome prediction accuracy, and the Helsinki CT score is 17. Moons KG, Harrell FE, Steyerberg EW. Should scoring rules be based on
a feasible alternative to previous CT scoring systems. The odds ratios or regression coefficients? J Clin Epidemiol. 2002;55(10):
Rotterdam and Marshall CT systems were of modest value in 1054-1055.
predicting long-term outcome in this large sample of patients with 18. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of
prediction models: a framework for traditional and novel measures. Epidemiology.
mild complicated, moderate, and severe TBI. External validation 2010;21(1):128-138.
of the Helsinki CT score in independent data sets is advocated to 19. Seshan VE, Gönen M, Begg CB. Comparing ROC curves derived from regression
show generalizability. models. Stat Med. 2013;32(9):1483-1493.
20. Nagelkerke N. A note on a general definition of the coefficient of determination.
Biometrika. 1991;78(3):691-692.
Disclosures 21. Zhu BP, Lemeshow S, Hosmer DW, Klar J, Avrunin J, Teres D. Factors
This study was funded by grants from Finska Läkaresällskapet, the Maire affecting the performance of the models in the Mortality Probability Model II
system and strategies of customization: a simulation study. Crit Care Med. 1996;
Taponen Foundation, and a Helsinki University Hospital EVO grant
24(1):57-63.
(TYH2012142). The authors have no personal, financial, or institutional interest
22. Moreno R, Apolone G. Impact of different customization strategies in the
in any of the drugs, materials, or devices described in this article. performance of a general severity score. Crit Care Med. 1997;25(12):
2001-2008.
REFERENCES 23. Bakhshi-Raiez F, Peek N, Bosman RJ, de Jonge E, de Keizer NF. The impact of
different prognostic models and their customization on institutional comparison of
1. Maas AI, Murray GD, Roozenbeek B, et al. Advancing care for traumatic brain intensive care units. Crit Care Med. 2007;35(11):2553-2560.
injury: findings from the IMPACT studies and perspectives on future research. 24. Steyerberg EW. Clinical Prediction Models. New York: Springer Verlag; 2010.
Lancet Neurol. 2013;12(12):1200-1210. 25. Robin X, Turck N, Hainard A, et al. pROC: an open-source package for R
2. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical and S1 to analyze and compare ROC curves. BMC Bioinformatics. 2011;
scale. Lancet. 1974;2(7872):81-84. 12:77.
3. Marmarou A, Lu J, Butcher I, et al. Prognostic value of the Glasgow Coma Scale 26. Hukkelhoven CW, Steyerberg EW, Habbema JD, et al. Predicting outcome
and pupil reactivity in traumatic brain injury assessed pre-hospital and on after traumatic brain injury: development and validation of a prognostic
enrollment: an IMPACT analysis. J Neurotrauma. 2007;24(2):270-280. score based on admission characteristics. J Neurotrauma. 2005;22(10):
4. Maas AI, Steyerberg EW, Butcher I, et al. Prognostic value of computerized 1025-1039.
tomography scan characteristics in traumatic brain injury: results from the 27. Huang YH, Deng YH, Lee TC, Chen WF. Rotterdam computed tomography
IMPACT study. J Neurotrauma. 2007;24(2):303-314. score as a prognosticator in head-injured patients undergoing decompressive
5. Raj R, Skrifvars M, Bendel S, et al. Predicting six-month mortality of patients with craniectomy. Neurosurgery. 2012;71(1):80-85.
traumatic brain injury: usefulness of common intensive care severity scores. Crit 28. Mata-Mbemba D, Mugikura S, Nakagawa A, et al. Early CT findings to predict
Care. 2014;18(2):R60. early death in patients with traumatic brain injury: Marshall and Rotterdam CT
6. Stocchetti N, Pagan F, Calappi E, et al. Inaccurate early assessment of neurological scoring systems compared in the major academic tertiary care hospital in
severity in head injury. J Neurotrauma. 2004;21(9):1131-1140. northeastern Japan. Acad Radiol. 2014;21(5):605-611.
7. Shahin H, Gopinath SP, Robertson CS. Influence of alcohol on early Glasgow 29. Nelson DW, Nyström H, MacCallum RM, et al. Extended analysis of early
Coma Scale in head-injured patients. J Trauma. 2010;69(5):1176-1181; computed tomography scans of traumatic brain injured patients and relations to
discussion 1181. outcome. J Neurotrauma. 2010;27(1):51-64.
8. Marshall LF, Marshall SB, Klauber MR, Van Berkum Clark M. A new 30. Jacobs B, Beems T, van der Vliet TM, Diaz-Arrastia RR, Borm GF, Vos PE.
classification of head injury based on computerized tomography. J Neurosurg. Computed tomography and outcome in moderate and severe traumatic brain
1991;75(11):S14-S22. injury: hematoma volume and midline shift revisited. J Neurotrauma. 2011;28(2):
9. Steyerberg EW, Mushkudiani N, Perel P, et al. Predicting outcome after 203-215.
traumatic brain injury: development and international validation of 31. Jacobs B, Beems T, van der Vliet TM, Borm GF, Vos PE. The status of the fourth
prognostic scores based on admission characteristics. PLoS Med. 2008;5 ventricle and ambient cisterns predict outcome in moderate and severe traumatic
(8):e165. brain injury. J Neurotrauma. 2010;27(2):331-340.
10. Maas AI, Hukkelhoven CW, Marshall LF, Steyerberg EW. Prediction of 32. Wardlaw JM, Easton VJ, Statham P. Which CT features help predict outcome
outcome in traumatic brain injury with computed tomographic character- after head injury? J Neurol Neurosurg Psychiatry. 2002;72(2):188-192;
istics: a comparison between the computed tomographic classification and discussion 151.
combinations of computed tomographic predictors. Neurosurgery. 2005;57 33. Servadei F, Murray GD, Teasdale GM, et al. Traumatic subarachnoid
(6):1173-1182; discussion 1173-1182. hemorrhage: demographic and clinical study of 750 patients from the European
11. Jennett B, Bond M. Assessment of outcome after severe brain damage. Lancet. Brain Injury Consortium survey of head injuries. Neurosurgery. 2002;50(2):261-
1975;1(7905):480-484. 267; discussion 267-269.

NEUROSURGERY VOLUME 75 | NUMBER 6 | DECEMBER 2014 | 645

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

RAJ ET AL

34. LeRoux PD, Haglund MM, Newell DW, Grady MS, Winn HR. Intraventricular that classification of TBI, both in the acute phase and in determining
hemorrhage in blunt head trauma: an analysis of 43 cases. Neurosurgery. 1992;31 outcome at later stages, should be multidimensional. Prognostic mod-
(4):678-684; discussion 684-685.
eling can aid in improving our understanding of possibly causal relations
35. Lee JP, Lui TN, Chang CN. Acute post-traumatic intraventricular hemorrhage analysis
of 25 patients with emphasis on final outcome. Acta Neurol Scand. 1991;84(2):85-90. between baseline characteristics and outcome. The applications of
36. Jacobs B, Beems T, van der Vliet TM, et al. Outcome prediction in moderate and prognostic modeling in a heterogeneous disease like TBI are multiple,
severe traumatic brain injury: a focus on computed tomography variables. being related to insight (eg, biology) and use (eg, informing, risk
Neurocrit Care. 2013;19(1):79-89. adjustment, benchmarking).
37. Raj R, Siironen J, Kivisaari R, et al. External validation of the international mission This study by Raj et al2 is a welcome and important contribution to the
for prognosis and analysis of clinical trials model and the role of markers of
literature on prognostic analysis in TBI. It advances our knowledge and
coagulation. Neurosurgery. 2013;73(2):305-311.
38. Panczykowski DM, Puccio AM, Scruggs BJ, et al. Prospective independent contributes to the development of new classification systems. The study
validation of IMPACT modeling as a prognostic tool in severe traumatic brain focuses specifically on computerized tomography (CT) characteristics.
injury. J Neurotrauma. 2012;29(1):47-52. The proposed Helsinki CT score would appear to perform better than
39. Roozenbeek B, Chiu YL, Lingsma HF, et al. Predicting 14-day mortality after existing CT models such as the Marshall and Rotterdam classification
severe traumatic brain injury: application of the IMPACT models in the Brain systems. The authors claim a statistically significant improved perfor-
Trauma Foundation TBI-trac New York State database. J Neurotrauma. 2012;
mance in terms of discrimination in comparison to these scores. There
29(7):1306-1312.
40. Lingsma H, Andriessen TM, Haitsema I, et al. Prognosis in moderate and severe are important perspectives to this study, both clinically and statistically.
traumatic brain injury: external validation of the IMPACT models and the role of From a clinical perspective, more detailed knowledge on interactions
extracranial injuries. J Trauma Acute Care Surg. 2013;74(2):639-646. between CT predictors and combining these predictors into a prognostic
model offer substantial advances. The study therefore constitutes an
Supplemental digital content is available for this article. Direct URL citations important stepping stone on the road to the development of a multidi-
appear in the printed text and are provided in the HTML and PDF versions of this mensional classification system that would include CT characteristics.
article on the journal’s Web site (www.neurosurgery-online.com). From a statistical perspective, there are perhaps even more intriguing
and possibly controversial aspects to this study. Compared with the
Marshall and International Mission for Prognosis and Analysis of Clinical
Acknowledgments Trials in TBI (IMPACT) classification systems, which each have 6 cate-
We wish to thank Jaakko Lappalainen, MD, PhD, for his valuable comments gories, the Helsinki CT score is more granular, consisting of a 17-point
on the manuscript and Giovanni Nattino (RCCS-Istituto di Richerche “Mario scale ranging from 23 to 14. By definition, the greater granularity of the
Negri” Laboratory of Clinical Epidemiology) for his statistical guidance. Helsinki will lead to better discrimination, as quantified by the area
under the receiver-operating characteristics curve (AUC). Furthermore,
comparing the AUCs between the Marshall, Rotterdam, and Helsinki
COMMENTS scores as performed by the authors is not fully a fair analysis. The external

T his article describes a new computerized tomography scale developed validation of the Marshall and Rotterdam scored is directly compared
at Helsinki for improving prognostication after traumatic brain with an apparent validation of the Helsinki score within the development
injury. It is 1 of 3 such systems developed in the past 5 years for improving population. External validation is important to demonstrate generaliz-
the predicative outcome of prior computerized tomography–based scales ability of findings in new patients. The AUC at apparent validation is
by using modern advanced statistical methods. The authors systematically prone to statistical optimism (a “home advantage”). These AUCs can
compare the accuracy of this newly developed scale with previously vali- hence not be fairly compared without complex internal validation
dated scales, including the Marshall and Rotterdam scales, and demon- procedures such as bootstrapping with replication of the full model
strate its superiority when used both for independent analysis and in development process.3,4 We further note that the discriminatory per-
conjunction with clinical data obtained during the International Mission formance of a model is influenced by the case-mix of the population
for Prognosis and Analysis of Clinical Trials in TBI (IMPACT) score under study.5 The population on which this model was developed is
assessment. However, like the other 2 recently developed scales, this apparently different from the populations on which the Marshall
system has yet to be validated on independent populations (ie, populations and Rotterdam models have been previously validated. In particular,
outside the same ones used to generate the scoring systems). Nevertheless, the Helsinki population includes a wider range of clinical severity
it represents a valiant effort in developing better tools for predicting and a substantially larger number of patients with intracranial lesions
prognosis in patients with traumatic brain injury, which is crucial in (Table 4; 60% had a mass lesion). Thus, the high number of patients in
guiding treatment strategies. evacuated/nonevacuated mass lesion categories limits the discriminatory
ability of the Marshall CT classification.
Duemani Reddy Prognostic modeling has a rich tradition in TBI, with the first pre-
Shankar Gopinath diction model being published in 1976.6 Major improvements in pre-
Houston, Texas diction models have come from the IMPACT and Corticosteroid
Randomisation After Significant Head Injury (CRASH) analyses.7,8

T he importance of prognostic modeling in clinical care and research is

being increasingly recognized and is particularly relevant to diseases
characterized by a large degree of heterogeneity such as traumatic brain
Modern computational, statistical, and bioinformatics approaches may
potentially further advance the development of prognostic models in
TBI, eg, by better exploiting the information of CT scans.
injury (TBI).1 Comparisons between patient groups are not possible The present article constitutes substantial advances both from clinical and
without knowledge of and adjustment for case-mix. Prognostic modeling statistical perspectives. The methodological aspects warrant further dis-
allows calculation of the baseline prognostic risk, essential for bench- cussions among epidemiologists, statisticians, and clinicians. From a clinical
marking quality of healthcare delivery. It is increasingly being recognized perspective, the Helsinki CT model appears promising but is not yet ready

646 | VOLUME 75 | NUMBER 6 | DECEMBER 2014 www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited

 THE HELSINKI CT SCORE

for prime time: External validation is required to determine generalizability 8. Steyerberg EW, Mushkudiani N, Perel P, et al. Predicting outcome after traumatic
and performance outside the development population. We should not be brain injury: development and international validation of prognostic scores based on
surprised if the improved performance as claimed by the authors then be- admission characteristics. PLoS Med. 2008;5:e165; discussion e165.
comes less evident. It is somewhat sobering to note that the additional value
of the CT characteristics contained within the Helsinki CT score is limited
compared with the prognostic effects contained in the 3 main predictors:
age, Glasgow Coma Scale motor score, and pupillary abnormalities.
I mproved accuracy of prediction of the outcome of severe traumatic
brain injury (TBI) is desirable, particularly in the first week after
injury. The authors have developed the Helsinki computerized
tomography (CT) score to improve the prediction of the outcome of
Andrew I.R. Maas TBI. They have compared their new model with the Marshall CT score,
Edegem, Belgium which was developed as a severity score, not as part of a predictive model,
and with the Rotterdam CT score. The Helsinki score contains more
Ewout W. Steyerberg
data entry points than the Marshall or Rotterdam CT score. On the basis
Rotterdam, Netherlands of this retrospective, single-center study of 869 consecutive TBI patients
using logistic regression techniques, the Helsinki CT score significantly
improved outcome prediction accuracy both on its own and when
1. Lingsma HF, Roozenbeek B, Steyerberg EW, Murray GD, Maas AI. Early combined with the and International Mission for Prognosis and Analysis
prognosis in traumatic brain injury: from prophecies to predictions. Lancet Neurol. of Clinical Trials in TBI (IMPACT) model of prediction. There was
2010;9(5):543-554. statistical significance for unfavorable outcome (P = .002) but not for
2. Raj et al, Predicting outcome in traumatic brain injury: development of a novel mortality (P = .21) when the Rotterdam CT score was combined with
computerized tomography classification system (the Helsinki CT score). Neuro- IMPACT. The Glasgow Outcome Scale score at 6 months was used as
surgery 2014;x:xx. the study end point.
3. Steyerberg EW, Bleeker SE, Moll HA, Grobbee DE, Moons KG. Internal and
The findings of this study need to be confirmed in a larger prospective
external validation of predictive models: a simulation study of bias and precision in
study involving other centers and should include the assessment of the
small samples. J Clin Epidemiol. 2003;56(5):441-447.
4. Steyerberg EW, Schemper M, Harrell FE. Logistic regression modeling and the reproducibility of the score by neurosurgery residents, who are the ones
number of events per variable: selection bias dominates. J Clin Epidemiol. 2011;64 likely to perform the Helsinki score in everyday practice.
(12):1464-1465; author reply 1463-1464. Any predictive tool such as this should always be regarded as a guide that
5. Vergouwe Y, Moons KG, Steyerberg EW. External validity of risk models: use of helps to inform sound clinical judgment, particularly when it concerns
benchmark values to disentangle a case-mix effect from incorrect coefficients. Am J withdrawal of active treatment and institution of palliative care. The
Epidemiol. 2010;172(8):971-980. authors list 2 other CT prediction models (Radboud and Stockholm).
6. Jennett B, Teasdale G, Braakman R, Minderhoud J, Knill-Jones R. Predicting Once more data are available on all of these CT prediction models,
outcome in individual patients after severe head injury. Lancet. 1976;1(7968):
neurosurgeons will need to agree on the best model.
1031-1034.
7. MRC CRASH Trial Collaborators; Perel P, Arango M, Clayton T, et al. Predicting
outcome after traumatic brain injury: practical prognostic models based on large Jeffrey V. Rosenfeld
cohort of international patients. BMJ. 2008;336:425-429. Melbourne, Australia

NEUROSURGERY VOLUME 75 | NUMBER 6 | DECEMBER 2014 | 647

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited