Appendix I: Statistical Analysis in Response to Comments

Response to Comments of the Alliance of Automotive Manufacturers based on a Study by Exponent Failure Analysis Associates, Inc. titled: The Relative Importance of Factors Related to the Risk of Rollover Among Passenger Vehicles

Background

The agency has proposed expanding the New Car Assessment Program (NCAP), which tests vehicle performance in front and side crashes, to include information on rollover resistance. We proposed a rollover metric for consumer information based on the Static Stability Factor (SSF) and described the approach in a Request for Comments, Notice for Rollover NCAP ("the Notice," docket NHTSA 2000-6859, item 1, June 1, 2000). The Appendix to the Notice described a statistical analysis of four years of data (1994 to 1997) from six states (Florida, Maryland, Missouri, North Carolina, Pennsylvania, and Utah), and we provided more details of the analysis (definitions, programming statements, and computer output) in another submission to the Rollover NCAP docket (item 4). The Alliance of Automobile Manufacturers ("the Alliance") reviewed the Notice and the supplemental material and submitted their comments to that docket (item 25).

Appendix 4 of their comments is a paper prepared for the Alliance by Exponent Failure Analysis Associates, Inc. ("the Exponent report") on The Relative Importance of Factors Related to the Risk of Rollover Among Passenger Vehicles (Alan C. Donelson, Farshid Forouhar, and Rose M. Ray, in a paper dated August 30, 2000). The Exponent report critiqued our linear regression analysis of the summarized crash data and suggested an alternative approach based on logistic regression analysis of individual crash events. This paper is a comparison of the two approaches (the linear model from summarized data and the logistic model of individual crash events) in response to those comments.

Overview

The Exponent report listed four goals for their study (page 4 of that report), and we will address their conclusions in our response. The four goals were as follows:

(1) "to evaluate the statistical study offered by NHTSA as a basis for comparative 'ratings' [emphasis in original] of rollover risk,"

(2) "to gauge the strength of SSF as a predictor of rollover relative to the influence of non-vehicular factors,"

(3) "to quantify the relationship between SSF and risk of rollover after adjusting for the influence of non-vehicular factors," and

(4) "to estimate the magnitude and reliability of apparent changes in rollover risk with changes in SSF."

The Exponent report offered three corrections to our vehicle group definitions, questioned the use of linear models of summarized data, and recommended logistic models of individual crash events as an improvement (their goals 1 and 2). In response, we have made the suggested corrections, used updated VIN-decoded data, added a year of data (the 1998 calendar year data are now available for all six states used in our original analysis), and refit the model. Details on the data definitions are included below in "Available Data," and the results of are described in "Refitting the Linear Model." We have also used our data to fit logistic regression models, and these results are described in "Fitting Logistic Models." A comparison of the two approaches is provided in "Comparing the Models."

Our logistic models produced results that were similar to those produced by our linear model of summarized data and to the logistic models described in the Exponent report (which were based on a slightly different group of states, calendar years, and explanatory variables). That is, the choice of model form and data source do not affect our essential conclusion: the SSF is strongly related (both in terms of statistical significance and magnitude of effect) to rollover risk. However, there are some differences among the models in the estimated sensitivity of rollover risk to changes in the SSF.

Where we disagree most with the Exponent report is in the interpretation of the results. The authors of the Exponent report argue that the SSF plays a smaller role in rollover causation than do driver and other road-use factors (their goals 2 and 4). Goal 2 (gauging the relative strength of the SSF and non-vehicle factors) is so important to the authors that they used it as the title of their report. We believe that our analysis indicates that the SSF is very important in describing rollover risk, as measured by the fit of each model, the significance of the coefficient of the SSF term, and the magnitude of the coefficient of the SSF term. We do recognize that driver and other road-use variables are also important. Federal, state, and local education and enforcement programs are all aimed at the vulnerability of road users to human error, and we recognize that the driver plays a large role in causing or avoiding crashes. However, what we set out to address in the Notice is whether the SSF provides information that is useful to consumers - information they can use in selecting a vehicle, deciding whether to use seat belts and child seats, and adapting their driving style to a new vehicle. We describe this point in more detail below, in "Interpreting the Analytical Results," using an example based on the relationship between crash severity, belt use, and injury severity.

In summary, we believe that our statistical models (both the linear model of summarized data and the logistic models of individual crash events) and the statistical models offered in the Exponent report support our conclusion that the SSF is a useful measure of rollover risk that will help the consumer choose a new vehicle and use it wisely.

Available Data

The analysis described in the Notice was based on single-vehicle crashes, which we defined to exclude crashes with another motor vehicle in transport or with a nonmotorist (such as a pedestrian or pedalcyclist), animal, or train. We eliminated vehicles without a driver and all vehicles that were parked, pulling a trailer, designed for certain special or emergency uses (ambulance, fire, police, or military), or on an emergency run at the time of the crash. Our only criterion for including a vehicle model in the analysis was a reliable measure of the SSF. The 100 vehicle groups we identified were described in the Notice, and the definitions for these groups were included in another submission to the same docket (item 4).

Exponent reviewed this information and pointed out three errors in the specifications of the vehicle groups (page 37). First, vehicle group 65 should have been defined as model years 1990-1995 (not 1988-1996). Second, vehicle group 66 should have been defined as model years 1996-1998 (not 1997-1998). And third, vehicle group 91 should have included model code "SKI" (not "SCI"), as defined by the output from The Polk Company's PC VINA^® software (PC VINA^® for Windows User's Manual, October 20, 1998). We also found a typographical error in the specification of vehicle group 79: the number of drive wheels should have been specified as "not equal to 4" (rather than "equal to 4"). We corrected these mistakes in the list and computer programs, and the corrected list of vehicles is included here as Tables 1 through 4.

Our understanding of some important differences in state crash reporting are included in Table 5. The Notice described our criteria for including a state in the analysis, which were as follows:

(1) data availability (the state must participate in the agency's State Data System (SDS) and have provided the 1997 data),

(2) VIN reporting (the vehicle identification number (VIN) must be coded on the electronic file), and

(3) rollover identification (we must be able to determine whether a rollover occurred, regardless of whether it was a first or subsequent event in the crash).

Six states (Florida, Maryland, Missouri, North Carolina, Pennsylvania, and Utah) met all three criteria. Two states (New Mexico and Ohio) met two of the three criteria; these states participate in the SDS and the VIN is available on the electronic file, but rollovers are identified only if they are reported as the first harmful event in the crash. We have made some use of all eight states in this updated analysis, but most of the analysis is based on the six states with the best rollover reporting. These are the six states that were the basis for the analysis described in the Notice.

For this analysis, we used the SDS data and the VIN-decoded data available on NHTSA's Research and Development Local Area Network ( LAN). The National Center for Statistics and Analysis (NCSA, an office in R&D) recently rebuilt the 1997 VIN files for Maryland and Missouri, and the numbers of relevant cases differ slightly from those reported in the Notice. The major changes were a slightly more-conservative approach to dealing with mistakes in VIN transcription and some additional vehicle-make codes. We also expanded somewhat our definition of "rollover" in North Carolina (adding information from the four impact-type variables), which increased the number of rollovers in that state over what was reported in the Notice. The number of relevant vehicles identified for each state and calendar year are shown as Table 6. Note that Ohio reported a relatively small percentage of VINs in 1998 (about 29 percent of vehicles had a VIN on the electronic file), so case counts for the vehicles relevant to this study are low. Our analysis is not too sensitive to missing VIN information because it is based on internal comparisons of the crash data (specifically, on rollovers per single-vehicle crash); this would not be the case if we were basing our analysis on comparisons with an external source, such as rollovers per registered vehicle.

We added a calendar year of data (1998) for the six states used in the analysis described in the Notice. However, Pennsylvania no longer includes on the electronic file some environmental variables that we need for this analysis (specifically, CURVE and GRADE), so we could not use the 1998 Pennsylvania data in the analysis. The variables available for this analysis are shown as Table 7. We calculated the SSF to two decimal places (with observed values between 1.00 and 1.53), we defined NUMOCC as the count of occupants in each vehicle, and we defined all the other road-use factors as dichotomous variables (with "0" coded for "no," and "1" coded for "yes").

All eight states reported the following data: ROLL, SSF, DARK, STORM, FAST, HILL, CURVE, BADSURF, MALE, YOUNG, OLD, and DRINK. Speed limit is not reported in New Mexico, so we defined FAST based on the roadway function class after reviewing the relationship between these two variables among New Mexico cases in the 1994-1998 Fatality Analysis Reporting System (FARS) data. We assumed, based on our review of the FARS data, that (1) interstate and rural arterial roads had a speed limit of at least 55 mph, (2) local roads and urban arterial roads, collectors, and ramps had a speed limit of no more than 50 mph, and (3) the speed limit was unknown for all other roads. RURAL was unavailable for two states (Maryland and Missouri), BADROAD was unavailable for two states (Missouri and Pennsylvania), NOINSURE was unavailable for three states (Maryland, North Carolina, and Utah), and NUMOCC was unavailable for Missouri (where uninjured passengers need not be reported).

Refitting the Linear Model

We refit the linear model using the approach described in the Notice. There were 241,036 single-vehicle crashes available for this analysis (that is, involving a vehicle in one of the 100 vehicle groups, occurring between 1994 and 1998, and occurring in the six states we studied in preparing the Notice (Florida, Maryland, Missouri, North Carolina, Pennsylvania, and Utah), and 48,996 of these (20.33 percent) involved rollover. We eliminated the 1998 Pennsylvania data because CURVE and GRADE are not available on the electronic file, and this left 227,194 single-vehicle crashes, of which 45,880 (20.19 percent) involved rollover.

We summarized the data for each vehicle group in each state, which produced 599 summary records (there were no reported single-vehicle crashes involving vehicle group 54 in Utah). As with the earlier analysis, we eliminated any summary record that was based on fewer than 25 cases because we thought estimates based on smaller samples were too unreliable. This left us with 518 summary records, representing the experiences of 226,117 single-vehicle crashes, including 45,574 (20.16 percent) rollovers. Figure 1 shows the rollover rate (rollovers per single-vehicle crash) as a function of the SSF plotted for each of the 100 vehicle groups. These data have not been adjusted for differences in vehicle use or state reporting practices, but they do show a strong tendency for lower rollover rates with higher values of the SSF.

We used the 1994-1998 General Estimates System (GES) for a comparison with the six-state rollover rate for the study vehicles as a group. The five years of GES data include 9,910 sampled vehicles that we identified as being in one of the 100 vehicle groups (based on decoding the VIN with the PC VINA^® software for those states that include the VIN on their police reports) involved in a single-vehicle crash, and 2,377 of these rolled over. Weighting the GES data to reflect the sample scheme (but not adjusting for missing VIN data) produces estimates of 1,185,474 single-vehicle crashes per year, of which 236,335 (19.94 percent) involved rollover. That is, the six states in our study have a rollover rate for police-reported crashes that is essentially the same as the national estimate produced from the GES data (with the qualification that the GES estimate is based on data from just those states that include the VIN on the police report).

We defined the dependent variable ROLL as the fraction of single-vehicle crashes that involved rollover. The independent (explanatory) variables in the six-state combined model were those available in all six states. They were expressed as the fraction of single-vehicle crashes that involved each of the following ten situations: DARK, STORM, FAST, HILL, CURVE, BADSURF, MALE, YOUNG, OLD, and DRINK. We also defined dummy variables for five states (DUMMY_FL, DUMMY_MD, DUMMY_NC, DUMMY_PA, and DUMMY_UT, with Missouri used as the baseline case) to capture state-to-state differences in reporting thresholds and definitions. These variables have the value "1" if the crash occurred in that state (for example DUMMY_MD = 1 for all Maryland crashes), and they have the value "0" otherwise (for example, DUMMY_MD = 0 for all crashes in Florida, Missouri, North Carolina, Pennsylvania, and Utah). These are the fourteen variables we used in the earlier analysis (described in the Notice), plus the variable OLD.

We ran the stepwise linear regression analysis against these 518 summary records to describe the natural logarithm of rollovers per single-vehicle crash, which we call LOGROLL, as a function of a linear combination of the explanatory variables. (To avoid losing information on vehicle models with a low risk of rollover, we set ROLL to 0.0001 if there were no rollovers represented by the summary record.) We used the option that gives more weight to data points that are based on more observations, so vehicle groups with more crashes count for more in the analysis. Each data point was weighted by the number of single-vehicle crashes it represented, but the weighting was capped at 250. That is, data points based on more than 250 observations were weighted by 250. Our rationale was that we wanted the model to fit well across the full range of SSF values, so we did not want to over-weight the data for the most-common models on the road.

We ran a preliminary model using the SSF and the five state dummies to estimate LOGROLL. The model had an R² of 0.73, and the coefficient of the SSF term (-2.8634) was highly significant (the t-statistic indicates that the probability that the coefficient is really zero is less than 0.0001); the details are included as Table 8a. Thus, it appears that the SSF is very useful in understanding rollover risk. We then performed a stepwise linear regression (using forward variable selection and a significance level of 0.15 for entry and removal from the model) on the six-state data; this is the same approach we used for the analysis described in the Notice. The stepwise regression procedure with the SSF chose three variables that describe the driving situation (DARK, FAST, and CURVE), three variables that describe the driver (MALE, YOUNG, and DRINK), and all five state dummy variables. The F-statistic for the model as a whole was 311, and the probability of a value this high by chance alone is less than 0.0001. The model had an R² of 0.88 and the coefficient of the SSF term (-3.3760) was highly significant; more details on the fit of the model are included as Table 8b. Note that adding the road-use variables increased both the model R² (from 0.73 to 0.88) and the absolute value of the coefficient of the SSF term (from -2.8634 to -3.3760). That is, the effect of the SSF on rollover risk is estimated to be even greater after adjusting for differences in road use.

We used the results of the model to adjust the observed number of rollovers per single-vehicle crash to account for differences among vehicle groups in their road-use characteristics in single-vehicle crashes. For each of the 518 summary records, we used the regression results and the typical road use to estimate what LOGROLL would have been if road use for that vehicle group had been the typical road use observed for all the vehicles in the study. The approach is the one used in the Notice. We used an intermediate step to account for differences in road use and adjust the data towards the average experience for the study vehicles:

ADJ_LOGROLL _i


= LOGROLL_i

-BETA_DARK × (DARK_i -MEAN_DARK)

-BETA_FAST × (FAST_i -MEAN_FAST)

-BETA_CURVE × (CURVE_i -MEAN_CURVE)

-BETA_MALE × (MALE_i -MEAN_MALE)

-BETA_YOUNG × (YOUNG_i -MEAN_YOUNG)

-BETA_DRINK × (DRINK_i -MEAN_DRINK)

-BETA_DUMMY_FL × DUMMY_FL_i

-BETA_DUMMY_MD × DUMMY_MD_i

-BETA_DUMMY_NC × DUMMY_NC_i

-BETA_DUMMY_PA × DUMMY_PA_i

-BETA_DUMMY_UT × DUMMY_UT_i

+ MEAN_DUMMIES,


where:


ADJ_LOGROLL_i is the estimate of what LOGROLL would have been for each summary record if all vehicles were used the same way,

LOGROLL_i is the value of LOGROLL observed for each summary record,

BETA_DARK through BETA_DRINK are the coefficients (Beta-values) of the road-use variables, DARK through DRINK, that were produced by the model (as shown in Table 8b),

BETA_DUMMY_FL through BETA_DUMMY_UT are the coefficients of the state dummy variables, DUMMY_FL through DUMMY_UT, that were produced by the model,

DARK_i through DRINK_i are the values of the road-use variables observed for each summary record,

DUMMY_FL_i through DUMMY_UT_i are the values of the state dummy variables for each summary record (with no more than one of these equal to "1," and all the rest equal to "0"),

MEAN_DARK through MEAN_DRINK are the average values of the road-use variables observed in the study data (with MEAN_DARK = 0.4314, MEAN_FAST = 0.4807, MEAN_CURVE = 0.3315, MEAN_MALE = 0.6276, MEAN_YOUNG = 0.3987, and MEAN_DRINK = 0.1509), and

MEAN_DUMMIES is the average state adjustment in the study data.


MEAN_DUMMIES was calculated for these 226,117 single-vehicle crashes from the coefficient of the state dummy variables and the number of cases in each state as follows:


( 1.2253 × number of Florida cases

+ 0.6933 × number of Maryland cases

+ 0.0000 × number of Missouri

+ 0.6969 × number of North Carolina cases

+ 1.2449 × number of Pennsylvania cases

+ 0.8622 × number of Utah cases)

/ Total number of cases

= 0.8019,

The adjusted rollover rate for each vehicle group is then estimated by:

ADJ_ROLL = e ^{(ADJ_LOGROLL)}.

This is our estimate of what the rollover rate would have been if all vehicle groups were used in the same way, and it reflects the average use patterns of all vehicles in the study. The adjusted rollover rates are shown in Figure 2.

The average adjusted number of rollovers per single-vehicle crash for all the study vehicles in the six states is 0.1982, which is essentially the same as the rollover rate in the original study data (0.2016) and the rollover rate estimated from the GES data (0.1994) for these 100 vehicle groups. A linear model fit through the adjusted data is described by the equation:

LOGROLL = 2.5861 - 3.3760 × SSF.

The model has an R² of 0.85, and the coefficient of the SSF term was highly significant. Details on the fit of the model through the adjusted rollover rates are included as Table 8c. Exponentiating both sides of the equation produces an estimate that the number of rollovers per single-vehicle crash is approximated by the curve:

ROLL = 13.28 × e^{(-3.3760 × SSF)}.

The estimated rollover rates for the SSF values between 0.95 and 1.55 are shown in Table 19 in the column labeled "Model 1," and the estimates for the observed range (SSF values from 1.00 to 1.53) are shown as Figure 2. This model form has very useful properties. The increase in the SSF that is associated with halving the number of rollovers per single-vehicle crash is estimated as 0.21. For example, the number of rollovers per single-vehicle crash under average conditions is estimated as:

0.44 for a SSF of 1.01

0.23 for a SSF of 1.22, and

0.11 for a SSF of 1.43.

Thus, rollover risk drops by a half when the SSF increases from 1.01 to 1.22, and it drops in half again when the SSF increases from 1.22 to 1.43.

The SSF is both highly significant in the model and very important in describing rollover risk (the estimated rollover risk increases by a factor of 6.0 over the observed range of the data, from a SSF of 1.00 to 1.53). This means that changes in the SSF (or changes in how vehicles with low SSF values are used) has the potential for large reductions in rollover risk.

Fitting Logistic Models

The Exponent report questioned the validity of using a linear regression analysis of summarized data, though they noted the advantages of this approach for describing the data. They suggested using a logistic regression analysis with the SSF and road-use variables, and they also suggested (as a way of dealing with potential cross-correlations) an approach that uses crash-risk scenarios in place of the road-use variables. They provided results from the states they used in their analysis, and we did a similar analysis of the eight states available to us. The data for two states, New Mexico and Ohio, were not combined with the data from the other six states because a rollover is reported in New Mexico or Ohio only if it is considered to have been the first harmful event in the crash. However, we did look briefly at these data because we were curious about how the rollover definition affects the analysis. We wanted to see how the risk of a rollover occurring as the first harmful event in a single-vehicle crash varies as a function of the SSF as reported in these two states.

We ran a logistic regression analysis for each state to model rollover as a function of the SSF and the road-use variables. For each state, we used the explanatory variables available for the linear regression analysis plus other variables that were available in each state, as described in Table 7. The fits of the models are summarized in Tables 9a through 16a. Each model seems to fit the data well. The coefficient of the SSF term varies from (-3.0800) in North Carolina to (-4.3908) in Florida. The values for New Mexico (-3.0809) and Ohio (-4.3642) fall in this range, which suggests that the choice between "all rollovers" and "first harmful event rollovers" may not be critical for a basic understanding of the sensitivity of rollover risk to the SSF (though the choice is important in determining the absolute level of rollover risk). In all cases, the coefficient of the SSF term was highly significant; the probability of a chi-square this large by chance alone (the smallest chi-square values were 209 for New Mexico and 416 for Utah) was estimated as less than 0.0001.

We then combined the data from the six states that have the best rollover reporting (that is, data that were not limited to first-harmful-event rollovers) and used them together in a logistic model, using the explanatory variables they have in common. We used the approach Charles Kahane described in his study of the safety effects of vehicle size. He used dummy variables to capture reporting differences in a logistic model of state data, and the results are included in Relationships between Vehicle Size and Fatality Risk in Model Year 1985-93 Passenger Cars and Light Trucks (Charles J. Kahane, Evaluation Division, Office of Plans and Policy, National Highway Traffic Safety Administration, DOT HS 808 570, January 1997). The results of the six-state combined model are shown as Table 17a. The model fits the data well, and the SSF is highly significant in the model (with a chi-square value of 7,230).

The coefficient of the SSF term in the logistic model for each state and for the six-state combined model describes the relationship between the rollover rates for any two values of the SSF, and we can use this relationship to estimate the rollover rate under average road-use conditions for each value of the SSF. We used the method that Ellen Hertz described in her study of the safety effects of vehicle weight. She estimated injury risk based on a logistic model of state data, and the results are included in A Collection of Recent Analyses of Vehicle Weight and Safety (T.M. Klein, E. Hertz, and S. Borener, Mathematical Analysis Division, National Center for Statistics and Analysis, Research and Development, National Highway Traffic Safety Administration, DOT HS 807 677, May 1991). We defined:

BETASSF = the coefficient of the SSF term in the logistic model for a state,

ROLL_SSF = the rollover rate at a specific value of the SSF, and

ODDS_SSF = the odds of rollover at a specific value of the SSF.

We choose a SSF of 1.00 as the basis for the calculations. The relationship between ROLL_1.00 and any other ROLL_SSF can be calculated for each state as follows:

ROLL_SSF = ODDS_SSF / (1 + ODDS_SSF)

where

ODDS_SSF = e ^{((SSF - 1.00) × BETASSF)} × ROLL_1.00 / (1 - ROLL_1.00).

The results of the logistic analysis of the Florida data are shown in Table 9a, including an estimate that:

BETASSF = (-4.3908),

so all we need for rollover rate estimates across the range of the SSF is an estimate of ROLL_1.00 in Florida. We estimated ROLL_1.00 using the following approach. For each state, we defined:

ODDS_ALL = odds of rollover for the study vehicles as a group,

LOGODDS_ALL = the natural logarithm of ODDS_ALL, and

MEANSSF = the average SSF for the study vehicles.

The model says that:

LOGODDS = T + (BETASSF × SSF),

where

T = a linear function of the explanatory variables,

and we solved for the "average" value of T such that:

LOGODDS_ALL = T + (BETASSF × MEANSSF).

That is, we assumed that the results of the logistic model apply to the average rollover rate and SSF value for the vehicles as a group, and this means that:

T = LOGODDS_ALL -(BETASSF × MEANSSF).

The rollover rate for all the vehicles included in the Florida study was 0.2044 and their average SSF was 1.2894, which means that:

T = log _e (0.2044 / 0.7956) - (-4.3908 × 1.2894) and

T = 4.3025 at the average rollover odds and SSF values.

We call this specific value of the function T, "T0." Then, after controlling for other factors, LOGODDS_SSF is estimated as:

LOGODDS_SSF = T0 + (BETASSF × SSF),

and at SSF=1.00 in Florida, this is calculated as:

LOGODDS_1.00 = 4.3025 - (4.3908 × 1.00), so LOGODDS_1.00 = (-0.0883).

ROLL_1.00 is estimated from the LOGODDS_1.00 as:

e ^x / (1 + e ^x),

where x is the LOGODDS_1.00, so the rollover rate at a SSF value of 1.00 is estimated as 0.4778 rollovers per single-vehicle crash. The rollover rate for all other values of the SSF can be estimated using:

ODDS_SSF = e ^{((SSF - 1.00) × BETASSF)} × ROLL_1.00 / (1 - ROLL_1.00)

and

ROLL_SSF = ODDS_SSF / (1 + ODDS_SSF).

We used this approach for the each state and for the six-state combined model. The average rollover rate and SSF for each state and for the six-state combined data are shown in Table 18, along with the estimated rollover rates for a SSF of 1.00. For example, the rollover risk for the six-states combined is estimated as 0.4031 at an SSF of 1.00, and it is shown in the column for the results of the models based on "individual variables." (The results of the models based on "crash scenarios" are described below.) The results for each value of the SSF are shown in the column labeled "Model 2" in Table 19.

As a check of the six-state combined model, we calculated the average rollover risk for each value of the SSF based on the individual state models. For example, we calculated the average rollover rate for a vehicle with a SSF of 1.00 by taking the average of the estimates for these six states (that is, Florida, Maryland, Missouri, North Carolina, Pennsylvania, and Utah), weighted by the size of each state (as measured by the number of single-vehicle crashes involving any study vehicle in each state). The result is an estimated risk of 0.4101 rollovers per single-vehicle crash for an SSF of 1.00, and the same procedure was applied to each value of the SSF from 0.95 to 1.55. The results are shown as the column labeled "Model 3" in Table 19.

The Exponent report also suggested using an approach they called a "crash scenario analysis" to address possible interactions among the explanatory variables. This idea is interesting and conceptually simple. The single-vehicle crashes from each state are categorized into cells defined by the possible combinations of the road-use variables. For example, the Florida logistic analysis used 14 road-use variables: DARK, STORM, RURAL, FAST, HILL, CURVE, BADROAD, BADSURF, MALE, YOUNG, OLD, NOINSURE, DRINK, and NUMOCC. NUMOCC is the count of occupants in each vehicle, and the other 13 variables take on the value "0" or "1" (indicating "no" or "yes"). This produces a large number of possible combinations of the variable values:

2¹³ × the number of levels of NUMOCC.

Converting NUMOCC into a dichotomous variable (for example, one that identifies vehicles with at least three occupants) yields 14 dichotomous variables, which means 2¹⁴ combinations of these variables, or 16,384 cells for the various crash scenarios. In practice, not all combinations will occur (there were 2,034 non-zero cells in the Florida data), and some non-zero cells have very low counts (there were 267 cells in the Florida data with at least 25 observations). The rollover rate for each cell can be calculated from these data, and this is a measure of the risk associated with that scenario. This rate can be used in place of all the road-use explanatory variables (for example, in place of the 14 original road-use variables in the Florida analysis). The Exponent report recommends a refinement to this calculation so that the scenario-risk variable for each specific vehicle reflects the rollover rate for all other vehicles in its cell. For example, in a cell with 100 vehicles and 20 rollovers, the scenario-risk variable (SCENRISK) will be calculated as:

20 / (100 - 1) for each nonrollover vehicle

and as

(20 - 1) / (100 - 1) for each rollover vehicle.

Using a crash-scenario variable is an interesting idea, even though the analytical results in the Exponent report seem to show that the individual-variable and crash-scenario logistic models produced very similar results. The standardized estimates for the coefficients of the SSF term produced by the two approaches (and our own results) are shown in Table 20. We attempted to duplicate the crash-scenario analysis based on the description provided in the Exponent report. The concept seems clear and logical, and we made the following decisions in implementing it for this analysis. First, we reviewed the output from the logistic regression on individual variables for each state and selected those for which the probability of a greater chi-square value was less than 0.20. We reasoned that using a large number of variables to define the crash scenarios would tend to produce many cells with small sample sizes, and that the variables with smaller chi-square values would be missed less. A review of Tables 9a through 16a shows that this eliminated only one variable in Florida (DARK), but it eliminated five variables in Utah (STORM, HILL, MALE, YOUNG, and OLD). Second, we converted NUMOCC into MANYOCC (with value "1" meaning three or more occupants, and "0" meaning one or two occupants). Again, the purpose of this was to reduce the number of cells with small sample counts, while retaining the essential information.

Third, we tabulated the number of single-vehicle crashes (SVACCS) and the number of rollovers (ROLLACCS) for each combination of DARK, STORM, RURAL, FAST, HILL, CURVE, BADROAD, BADSURF, MALE, YOUNG, OLD, NOINSURE, DRINK, and MANYOCC that had been selected for inclusion in each state. We eliminated any combination (that is, any crash scenario) with fewer than 25 observations. The results are summary data describing the experience of all vehicles in each crash scenario. Fourth, we merged the crash-scenario summary data for each state back onto the original data (that is, the data for each individual single-vehicle crash), so that each crash was linked to a count of the total number of single-vehicle crashes and the total number of rollovers that occurred in its crash scenario (its cell). We defined the scenario-risk variable, SCENRISK, as the rollover rate for all other vehicles in that crash scenario in that state. The calculation was a follows:

SCENRISK = (ROLLACCS - ROLL) / (SVAACCS - 1).

Recall that ROLL is coded as "1" if the vehicle rolled over and "0" if it did not, so this equation produces an estimate of the rollover rate for all vehicles in the crash scenario except for the one case under study; this was the method recommended by the Exponent report. This scenario-specific rollover rate is calculated for each vehicle on the file and is then available as an explanatory variable for a logistic model.

We ran a logistic regression analysis against the data for each state and for the six-state combined data to model rollover risk as a function of two variables: the SSF and SCENRISK. The fits of the models are summarized in Tables 9b through 17b. Each table shows the number of crash scenarios with at least 25 observations and the total number of crashes in these more-frequent scenarios. Each model seems to fit the data well. The coefficient of the SSF term in the crash-scenario logistic model for each state describes the relationship between any two values of the SSF. We applied the approach we used for the individual-variable logistic model to estimate the rollover risk for each value of the SSF and to combine the values across states. The rollover rates at a SSF of 1.00 are shown in Table 18, and the estimated rollover rates as a function of the SSF are shown in Table 19. The column labeled "Model 4" shows the results for the six-state model, and the column labeled "Model 5" shows the average of the individual models for the six states. Note that the individual-variable and the crash-scenario approaches produce very similar numbers. This is consistent with the results reported in the Exponent report (and summarized in Table 20, using the standardized estimates of the coefficients).

Comparing the Models

The rollover rates estimated across the range of SSF values for the six states combined are shown in Table 19 for all five statistical models (the linear model of summarized data and the four versions of the logistic model), and the estimates for the observed values of the SSF are plotted in Figure 3. The five models are as follows:

Model 1: Linear model of the summarized data,

Model 2: Logistic model of the six-state combined data, based on individual variables,

Model 3: Average of the logistic models for the six states, based on individual variables,

Model 4: Logistic model of the six-state combined data, based on crash scenarios, and

Model 5: Average of the logistic models for the six states, based on crash scenarios.

There are important similarities between the estimates produced by the two approaches: both the linear model of summarized data and the logistic models suggest a strong relationship (in terms of statistical significance and in terms of the magnitude of the effect) between the SSF and rollover risk. The average slope across the range of the observed SSF values (from 1.00 to 1.53) shown in Figure 3 is -0.713 for the linear model; the logistic models produce estimates of a slightly smaller effect, with average slopes between -0.598 and -0.555. Both types of models agree in estimating a large increased risk for vehicles with a low SSF. The four logistic models produce very similar results, and each suggests that rollover risk is very sensitive to the SSF (only slightly less so than estimated from the results of the linear model of the summarized data).

Figure 3 shows that the greatest absolute differences in the rollover rate estimates are at the lowest values of the SSF. The values of the rollover rate estimated for a SSF of 1.00 were as follows:

Model 1 = 0.4551 (linear model of the summarized data),

Model 2 = 0.4101 (logistic model of the six-state combined data with individual variables),

Model 3 = 0.4031 (average of the logistic models for the six states with individual variables),

Model 4 = 0.3999 (logistic model of the six-state combined data with crash scenarios), and

Model 5 = 0.3929 (average of the logistic models for the six states with crash scenarios),

The results of the four logistic models are almost indistinguishable in Figure 3: the crash-scenario approach produces results that are only slightly different from the individual-variable approach (the former are a little lower at a SSF of 1.00 and little higher at an SSF of 1.53), and the average of the logistic models for the six states produces results that are only slightly different from the logistic model of the six-state combined data (the former are a little lower at a SSF of 1.00 and little higher at an SSF of 1.53).

The results of our logistic analyses seem to differ only slightly from those described in the Exponent report, and much of the difference may be the result of our decision to omit wheelbase from the models. We did not include wheelbase as an explanatory variable because we could not identify any physical reason for an effect on rollover risk. However, we reran each analysis with the addition of wheelbase to test the sensitivity of the results to this decision. In every case, adding wheelbase to the model produced a higher estimate of the effect of the SSF on rollover risk and a higher estimate of rollover risk for the lowest values of the SSF. This occurred for all 18 models (those estimated using both the individual-variable and crash-scenario approaches for each of the eight states and for the six-state combined data), despite a negative value for the coefficient of the wheelbase term in each model. That is, the coefficient of the SSF term was negative in each of the original models, it became more negative in the presence of wheelbase, and wheelbase itself had a negative coefficient in each model in which it was included.

Adding wheelbase seemed to produce results closer to those in the Exponent report. That report does not include the estimates of the variable coefficients, but it does include the standardized coefficients. These are shown in our Table 20, along with the corresponding values from our analysis. For example, when we ran the logistic regression analysis on the Florida data and used wheelbase as one of the explanatory variables, we obtained values of (-0.392) and (-0.374) for the standardized coefficients from the individual-variable and crash-scenario models, respectively. These are higher than the values we obtained without wheelbase, (-0.349) and (-0.327), and they are very close to the values in the Exponent report, (-0.383) and (-0.381). Adding wheelbase to our models produced higher estimates of the coefficient for the SSF term and higher estimated rollover rates for vehicles with lower SSF values. For example, the six-state models that included wheelbase produced estimates that the coefficients of the SSF term are (-3.9525) and (-3.7918) and the estimated rollover rates for a SSF of 1.00 are 0.4338 and 0.4228 for the individual-variable and crash-scenario approaches, respectively.

There is also one important difference between the linear analysis of summary data and the logistic analysis of individual crashes. We limited the summary data to those based on at least 25 observations and we capped the weighting at 250 to avoid over-emphasizing the more-popular vehicles. However, the logistic regression analysis on individual crashes uses all observations equally. When we removed the two thresholds from the linear analysis, we obtained slightly lower estimates of the effect of the SSF on rollover risk, and the relationship between the adjusted rollover rates and the SSF is described by:

ROLL = 10.99 × e^{(-3.2356 × SSF)}.

This model produces an estimate of 0.4323 rollovers per single-vehicle crash at an SSF of 1.00, which is closer to the estimates from our logistic models (and essentially the same as the estimates from the logistic models that include wheelbase as an explanatory variable).

Interpreting the Analytical Results

Many of the comments in the Exponent report reflect an interest in evaluating the relative strength of the driver and vehicle contributions to rollover risk. We agree that this is an interesting question, but it is not the one we set out to address. Our perspective is that of a person choosing a new vehicle who wants to know how his choice of vehicle will affect his risk of being involved in a rollover. We are interested in eliminating the confounding effects of road use so we can isolate the effect of the vehicle on rollover risk. The importance of road-use factors does not preclude a role for vehicle-specific information.

Also, a factor can be important without suggesting an easy remedy. Consider two factors that increase the risk of rollover given a single-vehicle crash: driver age (specifically, the effect of young, inexperienced drivers) and curved roads. We do have some influence over their effect on rollover risk: better driver training and better road design can help reduce rollovers even among young drivers on curved roads. However, some additional risk is a given for people who are still gaining on-road experience, and curved roads are a necessity in many places. So, while driver and other road-use factors are important to understanding rollover risk, this is not the same as saying that all rollovers can be prevented by driver and other road-use remedies. Vehicle design plays an important role in understanding and mitigating rollover risk even among young drivers on curved roads by making vehicles more-forgiving of driver and road limitations, and our analysis describes the magnitude of that effect.

Another comparison may help clarify why we believe that the SSF can be useful even though driver and other road-use factors are such valuable predictors of rollover risk. Using the same approach Exponent used for SSF and other factors involved in rollover, one can statistically demonstrate that seat belt use is insignificant in preventing injuries from a crash. The 1998-1999 National Automotive Sampling System (NASS) data include 7,631 investigated unbelted drivers of light passenger vehicles that were towed from a frontal nonrollover crash (Table 22), and weighting these data to reflect the sampling plan produces an annual average estimate of 171,284 drivers involved each year. An estimated 11,569 of these were seriously injured (that is, they died or received an injury rated as three or higher on the Abbreviated Injury Scale). The overall risk of serious injury was 6.75 percent, but the risk varied greatly as a function of the change in vehicle velocity during the impact (that is, the delta V). For delta V less than 10 mph, the risk of serious injury was 0.76 percent.

If all 171,284 drivers in these towaway crashes had been injured at the same rate as those in the lowest delta V range, we would have seen:

0.0076 × 171,284 = 1,302 serious injuries

among unbelted drivers in frontal crashes. Half of these (601 serious injuries) could have been prevented if the drivers had used a lap-and-shoulder belt. Thus, we have the following:

171,284 serious injuries among unbelted drivers, of which

1,202 would have occurred if delta V was low, of which

601 would have occurred if belts were used.

According to the logic proposed by Exponent, we would interpret the results as follows:

99.30 percent of serious injuries are attributable to high crash speeds, and

0.35 percent are attributable to neglecting to use belts.

Clearly this is nonsense. Belt use will prevent serious injury even among those in higher-speed crashes (half of the 11,569 serious injuries that did occur among unbelted drivers at any crash speed could have been prevented by belt use, for a reduction of 5,784 serious injuries from belt use). More importantly, belts offer a practical solution, while there is no practical way to reduce all crash speeds to less than 10 mph.

Note that this is comparable to the approach that the Exponent report used in arguing that the value of the SSF in understanding rollover risk was in the range of 3-8 percent. They estimated the relative risk of the lowest-risk scenario, estimated how many rollovers could be prevented if all single-vehicle crashes occurred with the risk of the lowest-risk scenario, and relegated the importance of the SSF to a fraction of the small amount of risk that remained. The lowest-risk scenario that they use as their standard appears to be (based on the table on page 31 of their report) crashes that did not involve a vehicle defect and that did involve a mature driver who had not been drinking or engaged in risky driving, on a straight, urban road with a speed limit of 50 mph or less, and for which the first harmful event was a collision with a traffic unit in a single-vehicle crash; the bulk of these crashes may be collisions with pedestrians and pedalcyclists, which would tend to be reported because of the injuries to the non-motorists.

These are crashes with almost no chance of rollover, and so they are essentially irrelevant to a rollover-prevention program. Also note that some of these factors can be addressed by the driver (driving more carefully and when fully sober), but others are beyond the control of the driver (roads are curved, through rural areas, and with speed limits of 55 mph so traffic can move efficiently through all parts of the country). Young drivers gain experience through driving, and they eventually become mature drivers; in the meantime, they also benefit from more-stable vehicles. It is difficult to see how Exponent's the low-risk scenario could be used as an alternative to the SSF as the basis for a rollover safety program.

The approach described in the Exponent report (comparing the risk associated with the SSF to all the risks associated with road-use factors) would suggest, in our example based on NASS data, that reducing delta V should be a higher safety priority than increasing belt use. (To use an extreme example to make a point, using the approach described in the Exponent report for a study of air crashes would suggest that preventing gravity is more important than regular maintenance of the airplane.) However, belt use programs have been successful because the remedy is simple and cost-effective and because the importance of delta V does not reduce the importance of belt use in preventing injury. We believe a similar argument can be made for focusing on the SSF, while agreeing that driver and other road-use variables may be the basis for other safety improvements.

Conclusion

The Exponent report acknowledged the potential advantages of multiple linear analysis, and their recommendation is relevant here:

Multiple regression analysis can have some value as an explanatory tool for describing factors related to vehicle rollover. Linear regression analysis, however, must only be used in this heuristic way - and only when prior research has demonstrated that linear regression produced essentially the same results as did a rigorous and valid statistical analysis. [page 28]

Table 19, Figure 3, and the sensitivity analyses described above suggest that the linear and logistic regression approaches produce essentially the same results. The Exponent report recommended a logistic approach and concluded that the linear approach based on summarized data overstated the value of the SSF in understanding rollover risk. This does not seem to be the case. The linear approach produces estimates of rollover risk that are a little more conservative (in the sense that they are lower) than those from the logistic models for most observed values of the SSF and for most vehicles on the road today. The Exponent report included much lower estimates for rollover risk across the range of SSF values, but this was not a result of the logistic approach. Rather, it was the result of tying the estimates to the low-risk scenario (where rollover is unlikely).

* The Average Slope was calculated for the observed range of SSF values for our vehicles in the state data (1.00 to 1.53), as the difference in the estimated rollover rates divided by the difference in the SSF values.

Table 20: Standardized Estimate for the Coefficients Produced by the Logistic Models of Rollover as a Function of the SSF
State	Exponent: Individual Variables	Exponent: Crash Scenarios	NHTSA: Individual Variables, without Wheelbase	NHTSA: Crash Scenarios, without Wheelbase	NHTSA: Individual Variables, with Wheelbase	NHTSA: Crash Scenarios, with Wheelbase
Alabama	-0.282	-0.282
Florida	-0.383	-0.381	-0.349	-0.327	-0.392	-0.374
Idaho	-0.308	-0.318
Maryland	-0.303	-0.310	-0.295	-0.283	-0.320	-0.310
Missouri			-0.288	-0.274	-0.312	-0.304
New Mexico			-0.233	-0.228	-0.239	-0.236
North Carolina	-0.287	-0.292	-0.239	-0.231	-0.269	-0.266
Ohio			-0.326	-0.322	-0.343	-0.342
Pennsylvania	-0.271	-0.284	-0.245	-0.236	-0.266	-0.260
Utah			-0.330	-0.331	-0.368	-0.369

Table 21: Coefficients of the SSF Variable from the Logistic Models
State	Model without Wheelbase		Model with Wheelbase
	Individual Variables	Crash Scenarios	Individual Variables	Crash Scenarios
Florida	-4.3908	-4.1227	-4.9284	-4.7108
Maryland	-3.7203	-3.5760	-4.0387	-3.9206
Missouri	-3.8283	-3.6441	-4.1553	-4.0384
New Mexico	-3.0809	-3.0129	-3.1559	-3.1185
North Carolina	-3.0800	-2.9751	-3.4710	-3.4260
Ohio	-4.3642	-4.3136	-4.6001	-4.5838
Pennsylvania	-3.0793	-2.9574	-3.3352	-3.2562
Utah	-4.0534	-4.0590	-4.5195	-4.5338
Six-state model (FL, MD, MO, NC, PA, UT)	-3.6054	-3.4555	-3.9525	-3.7918

Table 22: Risk of Serious Injury Among Unbelted Drivers of Towed Light Vehicles in Frontal Nonrollover Crashes (1988-1999 NASS Investigated Cases and Annualized National Estimates)
Delta V (in mph)	Investigated Cases		Annualized Estimates
	All Involved Drivers	Drivers with Serious Injury	All Involved Drivers	Drivers with Serious Injury	Percent with Serious Injury
00-09	517	8	24,196	183	0.76
10-19	3,758	305	107,038	3,342	3.12
20-29	2,343	623	33,564	4,573	13.63
30-39	718	381	4,822	2,370	49.15
40-49	207	146	1,198	726	60.58
50 +	88	73	465	374	80.46
Total	7,631	1,536	171,284	11,569	6.75