Building the P/ECM
Placing members of the five general P/ECM categories into smaller sets (groups) was an inductive process that often demanded that the researcher use professional judgment to augment the statistical results. In building these types of models, a researcher tries various combinations of indicators to form groups and reviews the results. Some splits made by grouping software may be reasonable; others may not. Some splits may make logical or clinical sense; others may not. Thus, some measure of researcher judgment was exercised in building the case-mix model presented here.
Within each of the five basic categories used in the P/ECM, four sets of variables were considered for the possibility of developing useful groups: ADL status, continence, the presence of behavior problems, and the provision of habilitation services. The term “habilitation” is used, rather than rehabilitation, because it is more appropriate for a younger population. IADL status was initially included in the testing, but the results indicated that it had little utility in the modeling process.
Developing case-mix indices
Each P/ECM group’s average expenditure was translated into a case-mix index (CMI).7 The CMIs were calculated by dividing the average group expenditure by the average expenditure for the sample. For example, the sample average in this instance was roughly $12,000. Thus, any group with a mean of $36,000 would have a CMI of three (3.0).
The use of a CMI creates the possibility that the model can be used across settings, where the average resource allocation varies from that in the development site. Average resource allocation may vary across settings or populations. However, relative payment or resource use differences among groups (CMIs) should largely remain stable. This assumption will be tested when additional data from other settings are available.
Assessing the P/ECM’s explanatory power
The most basic test of the quality of a classification model is the percentage of variance (adjusted R2) the case-mix groups explain in the criterion variable.7 To evaluate the P/ECM, ordinary least squares (OLS) models were estimated using the 24 P/ECM groups as independent variables.
These models were estimated for the sample as a whole and for important sub-populations in the sample. In addition, a variety of variables (age, gender, specific conditions, and the DSHS regional office involved in the assessment) were added to the multivariate models to determine whether these variables had significant effects on expenditures over and above the P/ECM groups.
The quality of the P/ECM was also tested using logistic regression. These analyses provided information on how well the P/ECM predicted membership in the highest or lowest deciles or quartiles of expenditures. The logistic regressions speak to two issues. First, they provide information on the potential usefulness of the P/ECM categories as a screener. Second, they help pinpoint where predictions based on the P/ECM may be strongest or weakest.
Assessing the potential external validity of the P/ECM
A common strategy when building classification models is to use one-half of the data to build the model and test the model on the remaining half to evaluate the model’s external validity.7 The relatively small number of persons in some important groups in these analyses forced the abandonment of that strategy. Thus, in this effort, the P/ECM was developed using the entire sample. In order to develop relatively stable expenditures estimates a minimum group size (20) was specified. To approximate the split-sample approach to examining the model’s external validity, however, the model was tested using 10 randomly generated 50% sub-samples.
Details of the analysis
The groups within the five classification categories were developed with the classification and regression tree procedure in XLSTAT 2015, using the Chi-Square Automatic Interaction Detection method developed by Kass.29 Basic data manipulation, bivariate analyses, and multivariate analyses were performed using STATA 14.