Cost Analysis Report (CA Drug Cost Analysis Study), SACP, 2006
Download original document:
Document text
Document text
This text is machine-read, and may contain errors. Check the original document to verify accuracy.
SACPA COST ANALYSIS REPORT (First and Second Years) Prepared by: UCLA Integrated Substance Abuse Programs 1640 S. Sepulveda Blvd., Suite 200; Los Angeles, CA 900257535. Authored by: Douglas Longshore*, Ph.D. Angela Hawken, Ph.D. Darren Urada**, Ph.D. M. Douglas Anglin, Ph.D. Submitted to: California Department of Alcohol and Drug Programs Submission date: March 13, 2006 Acknowledgements: Thanks are due to Kenneth Nyberg, Ph.D., California State University, Bakersfield, and to Elizabeth Evans, M.A., Bradley T. Conner, Ph.D., Michael Prendergast, Ph.D., and YihIng Hser, Ph.D., UCLA Intergrated Substance Abuse Programs, for editorial review and commentary. * Deceased December 30, 2005. **Comments or questions about this report can be submitted via email to: durada@ucla.edu. 1 TABLE OF CONTENTS EXECUTIVE SUMMARY .........................................................................................................3 SACPA First Year Cost Analysis (Study 1) ......................................................................3 SACPA Drug Treatment Participation Cost Analysis (Study 2) ........................................4 SACPA Second Year Replication (Study 3) .....................................................................4 Specific Substudies .........................................................................................................5 Conclusions and Recommendations..................................................................................5 Research Methods ............................................................................................................5 SACPA COST ANALYSIS REPORT (First and Second Years) .................................6 Introduction......................................................................................................................6 Background......................................................................................................................7 FINDINGS..................................................................................................................................8 SACPA First Year Cost Analysis (Study 1) ......................................................................8 SACPA Drug Treatment Participation Cost Ratios (Study 2)..........................................16 SACPA Second Year Replication (Study 3) ...................................................................18 HighCost Offender Substudy .......................................................................................20 Adult Welfare Substudy................................................................................................21 CONCLUTIONS AND RECOMMENDATIONS .....................................................................21 Conclusion 1: SACPA substantially reduced incarceration costs....................................21 Conclusion 2: SACPA results in greater cost savings for some eligible offenders than for others. ......................................................................................22 Conclusion 3: SACPA can be improved. . ....................................................................22 RESEARCH METHODS..........................................................................................................23 Design............................................................................................................................23 Comparison Groups........................................................................................................24 Rationale for Analytic Time Frames...............................................................................26 Causal Inference.............................................................................................................27 Costs Covered ................................................................................................................28 Analytic issues ...............................................................................................................30 Appendix A: Research Questions .............................................................................................31 Appendix B: Expanded SACPA Evaluation Conclusions and Recommendations .....................33 Appendix C: DifferenceinDifferences (DID) Hypothetical Example ......................................39 Appendix D: Historical Context of SACPA Implementation ....................................................41 2 The California Substance Abuse and Crime Prevention Act of 2000 (SACPA) requires that an offender convicted of a nonviolent, drugrelated crime be offered probation and community based drug treatment in lieu of jail or prison. UCLA conducted three studies assessing the cost implications and benefitcost ratios of SACPA. Each showed that SACPA yielded cost savings to state and local governments. Study 1, using a before SACPA comparison group and all firstyear SACPA eligible offenders, found a net savings of $2,861 per offender (N=61,609), yielding a benefitcost ratio of nearly 2.5 to 1 (i.e., $2.50 was saved for every $1 invested). Study 2 determined that SACPA participants who completed the program achieved a benefitcost ratio of approximately 4 to 1 (i.e., “completers” saved $4 for every $1 allocated). Study 3 found that cost savings for the second year of SACPA were similar to Study 1, with a benefitcost ratio of 2.3 to 1. Three conclusions result from the cost analyses: SACPA substantially reduced incarceration costs; SACPA resulted in greater cost savings for some eligible offenders than for others; and SACPA can be improved. Recommendations encompass actions within and across multiple areas: Statewide collaboration and coordination, offender eligibility criteria and alternative practices for highcost offenders, systems integration, criminal justice, drug treatment, and strategic planning. EXECUTIVE SUMMARY The California Substance Abuse and Crime Prevention Act of 2000 (SACPA) requires that offenders convicted of a nonviolent, drugrelated crime be offered communitybased drug treatment and probation in lieu of jail or prison. Evaluation reports on the first three years of SACPA have been produced. 1 To examine the cost offsets and the benefitcost ratios of SACPA, three studies were conducted. Study 1 assessed SACPA as a policy, regardless of the degree of offender participation. Study 2 assessed variability of cost ratios in relation to offenders’ degree of participation in drug treatment after their acceptance of SACPA at conviction. Study 3 compared cost estimates from the first to the second year of SACPA. All studies used a comparison group taken from a period before SACPA was implemented. An additional analysis examined the characteristics of highcost offenders identified in SACPA. The analysis covered costs in eight areas. Five of these were in criminal justice: jail, prison, probation, parole, and arrests and convictions. Two were social service areas: drug treatment and healthcare. One additional area allowed accrued costs to be reduced by taxes paid by offenders on earnings and purchases. All studies used the “taxpayer perspective,” in which the focus is on costs to state and local governments. Costs were adjusted to 2004 dollars using the consumer price index or, where appropriate, the medical price index. SACPA First Year Cost Analysis (Study 1) Study 1 found a benefitcost ratio of nearly 2.5 to 1, indicating that $2.50 was saved for every $1 allocated to fund SACPA. Across the 8 areas assessed, SACPA led to a total cost savings of $2,861 per offender over the 30month followup period, representing a net savings to government of $173.3 million. Study 1 defined a SACPAera group of 61,609 offenders that included all adults (age 18 or older) who were, during SACPA’s first year (July 1, 2001, to June 30, 2002), convicted of a SACPAeligible offense with no concurrent offense or other circumstance that should have made them ineligible. The 30month followup period ended on or before December 31, 2004. The comparison group (N=68,543) consisted of all adults convicted between January 1, 1997, and 1 Longshore, D. et al., (2005). Evaluation of the Substance Abuse and Crime Prevention Act: 2004 Report. Los Angeles: UCLA Integrated Substance Abuse Programs; Longshore, D. et al., (2004) Evaluation of the Substance Abuse and Crime Prevention Act: 2003 Report; Longshore, D. et al., (2003) Evaluation of the Substance Abuse and Crime Prevention Act: 2002 Report. 3 June 30, 1998, of an offense for which they would have been SACPAeligible with no concurrent offense or other circumstance that should have made them ineligible. The followup period ended on or before December 31, 2000. Study 1 used 30month pre (followback or baseline) and post (followup) periods 2 from the date of each offender’s conviction. The benefitcost profile of SACPA showed cost savings for prison ($3,547 per offender), jail ($1,531), parole ($221), and taxes ($59). Higher SACPA costs accrued for probation ($198 per offender), arrests and convictions ($1,326), drug treatment ($743), and healthcare ($230). SACPA Drug Treatment Participation Cost Ratios (Study 2) Study 2 found that savings for drug treatment completers reflected a benefittocost ratio of about 4 to 1, despite higher treatment costs for this group. This indicates that approximately $4 was saved for every $1 allocated to a treatment completer under SACPA. Overall costsavings per offender were more than twice as high for those who completed treatment ($5,601) compared with those who never entered ($2,468) or did not complete treatment ($2,386). 3 Study 2 examined variation in cost ratios in relation to SACPA participation. The study was based on the same SACPA population as in Study 1 but excluded those offenders who refused diversion to SACPA. Those who accepted the SACPA sentencing alternative were assessed in three groups: (1) offenders who were referred to SACPA but did not enter drug treatment, (2) offenders who entered but did not complete treatment, and (3) offenders who completed treatment. Like Study 1, Study 2 covered 30month pre and postperiods from the date of each offender’s conviction. For those who accepted SACPA but did not enter drug treatment, prison savings per offender were $2,459, compared to $4,058 for those with some treatment and $6,175 for those who completed treatment. Jail savings followed a similar pattern, $1,411, $1,822, and $2,372, respectively. Parole savings were relatively equivalent at $211, $245, and $225. However, probation costs were higher at $111, $329, $336. Drug treatment costs were higher for treatment noncompleters and treatment completers at $1,335 and $2,027. There was a small drug treatment cost reduction for those who did not enter treatment ($48). Healthcare costs were higher with level of treatment participation at $154, $260, and $434. Arrest and conviction cost increases were lower for treatment completers ($552) than for those who did not enter treatment ($1,440) and for those who did not complete treatment ($1,859). SACPA Second Year Replication (Study 3) Using 12month pre and post periods, the benefitcost ratio in the secondyear was very close to the firstyear at 2.1 to 1 for the first year and 2.3 to 1 for the second year. The overall cost savings to government were $2,280 per offender for firstyear SACPA offenders ($140.5 million in total) and $2,306 for secondyear offenders ($158.8 million in total). Study 3 examined costs in SACPA’s first two years using each year’s total SACPA eligible population (as in Study 1), but with a pre and postperiod of 12 months. Cost outcomes and ratios were relatively stable across the two years. Only a few differences occurred. Prison cost savings were $1,879 for the firstyear SACPA offenders and $1,826 for the secondyear; jail cost savings were $1,555 and $1,211, respectively. Arrest and conviction cost increases were lower for the secondyear cohort ($313) compared to the firstyear cohort ($555). 2 Although the respective meanings are the same, the terms pre and post are commonly used in discussion of experimental design; the terms followback and followup are typical in economic research; and the term baseline is used comparatively in reporting results. UCLA used the terms as appropriate to the context within the report. 3 Although SACPA offenders who received some drug treatment showed reductions in prison and jail time over those who did not enter treatment, these savings were offset by treatment costs and somewhat higher rates of arrest and conviction in the followup period. When only criminal justice costs are considered, the cost savings are as expected: no treatment, least; some treatment, intermediate; completed treatment, most. 4 Specific Substudies: Findings Highcost offenders. UCLA found that a few SACPA offenders (1.6%) contributed a disproportionate share of criminal justice costs. These offenders (N=1,010) were mostly composed of individuals with 5 or more convictions in the 30months before their SACPA eligible conviction. Their postperiod crime costs were 10 times higher than those for the typical offender: $21,175 versus $2,254, respectively. Adult welfare. At the time of SACPA implementation, welfare reform was well underway in California. Changes in beneficiary eligibility and duration of benefits were such that the adult welfare data could not be utilized in the econometric cost model. A brief descriptive analysis of these data is provided in the main report. Conclusions and Recommendations 4 Three conclusions result from the cost analyses: SACPA substantially reduced incarceration costs: From the perspective of state and local government, continued funding for SACPA is justified; SACPA resulted in greater cost savings for some eligible offenders than for others; and SACPA can be improved. The savings from SACPA are largely due to reductions in jail and prison time. Cost increases were primarily due to subsequent arrests and convictions and drug treatment. Probation and parole cost changes were modest, as were increases in healthcare costs and contributions from taxable earnings. Briefly stated, recommendations cover six areas, requiring the involvement of multiple systems and agencies: Statewide collaboration and coordination, offender eligibility and alternative procedures and practices for highcost offenders, systems integration, criminal justice, drug treatment, and strategic planning. Activities to achieve the following goals should yield a more efficacious program, with attendant cost benefits: (1) increased collaboration and coordination within and across state and county government, (2) improved system integration by all involved agencies within the counties, including greater utilization of probation and program urine test results; (3) more attention to suitability screening for and higher acceptance and participation rates by offenders referred to drug treatment under SACPA, as well as increased use of strategies to improve offender accountability; (4) improved matching of severity of dependence to intensity of services; (5) more accessible and culturally relevant services for special populations (e.g., those with psychiatric problems, minorities); and (6) more attention to continuity of care and treatment aftercare services. The Department of Alcohol and Drug programs should develop and implement a longrange strategic plan, guided by past and ongoing program experience and evaluation research, to improve SACPA in each successive year. Research Methods The cost analysis uses a regressionadjusted 5 d5ifferencesindifferences (DID) approach to estimate cost changes under SACPA. Data were obtained from administrative databases maintained by the relevant state agency. The UCLA DID analysis compared the cost differences from pre to postconviction for each offender in SACPA to the differences observed in the comparison group. The DID design improves on prior studies in the following ways: enables the identification of changes directly attributable to the law; overcomes concerns of potential biases with selfreport data by the use of administrative data; and uses longer baseline and followup periods. Overall, these improvements in design, analytic procedures, and data sources lend additional weight to the findings over other cost evaluation methodologies. 4 See also main report and Appendix B, where more extended conclusions, recommendations, and suggested strategies are discussed. 5 Regression adjustment uses multivariate statistical techniques that minimize any spurious effects due to differences in sample composition or other possible covariates. 5 SACPA COST ANALYSIS REPORT (First and Second Years) Introduction In November 2000, California voters passed Proposition 36, which was enacted into law as the Substance Abuse and Crime Prevention Act of 2000 (SACPA). SACPA represented a major shift in criminal justice policy in California. Under the law, adults convicted of nonviolent drug offenses and who meet SACPA eligibility criteria can be sentenced to probation with drug treatment instead of incarceration or probation without treatment. Offenders on probation or parole who committed nonviolent drug offenses or who violate drugrelated conditions of their supervision can also receive treatment. SACPA legislation required that a public university in California evaluate the effectiveness and financial impact of the programs funded pursuant to the requirements of the act. The California Department of Alcohol and Drug Programs (ADP), through a competitive bid process, chose the University of California, Los Angeles, (UCLA) Integrated Substance Abuse Programs to conduct the evaluation over a 5½year period beginning January 1, 2001, and ending June 30, 2006. Evaluation reports on the first three years of SACPA have been produced by UCLA. 6 This cost analysis is a critical portion of the evaluation. The purpose of the SACPA cost analysis was to examine costs as related to: 1) overall costs to state and local government for drug offenders eligible for SACPA; 2) cost patterns based on the degree of SACPA participation by offenders; and 3) possible changes in cost outcomes over SACPA’s first and second years. Study 1 calculated the cost savings attributable to SACPA as a policy, that is, as a change in law that applied to all offenders throughout the state, regardless of the degree of offender participation. Study 2 examined variation in cost ratios in relation to offenders’ degree of participation in SACPA. This study assessed cost outcomes for offenders who accepted the drug treatment option at conviction (i.e., accepted referral to the SACPA program), whether they complied and entered drug treatment, and whether they completed the planned treatment duration. A particular focus of Study 2 was the cost ratio for drug treatment completers. Study 1 and Study 2 were based on SACPA’s firstyear population of eligible offenders and covered a 30month pre, or baseline, period and a 30month post, or followup, period from the eligibility conviction. Study 3 examined the potential change in cost estimates from the first to the second year of SACPA to assess if maturity of the policy and its implementation may have changed cost outcomes. Study 3 used a pre and postperiod of 12 months. All three studies used the “taxpayer perspective,” focusing on costs to state and local governments. Results are expressed in average cost or savings per offender. Furthermore, all costs were adjusted to 2004 dollars to allow standardization across multiple years. This report provides the essential findings and the resulting conclusions and recommendations of the three studies. It also summarizes the complex analytic process undertaken to provide valid and consistent data, appropriate analysis, and suitable adjustments for the cost components under consideration. The analysis was originally designed to cover costs in 10 areas. Five of these were in criminal justice: jail, prison, probation, parole, and arrests and convictions. Four were 6 Longshore, D., Urada, D., Evans, E., Hser, YI., Prendergast, M., & Hawken, A. (2005). Evaluation of the Substance Abuse and Crime Prevention Act: 2004 Report. Los Angeles: UCLA Integrated Substance Abuse Programs. Longshore, D., Urada, D., Evans, E., Hser, YI., Prendergast, M., Hawken, A., Bunch, T., & Ettner, S. (2004). Evaluation of the Substance Abuse and Crime Prevention Act: 2003 Report. Los Angeles: UCLA Integrated Substance Abuse Programs. Longshore, D., Evans, E., Urada, D., Teruya, C., Hardy, M., Hser, YI., Prendergast, M., & Ettner, S. (2003). Evaluation of the Substance Abuse and Crime Prevention Act: 2002 Report. Los Angeles: UCLA Integrated Substance Abuse Programs. 6 in social service areas: drug treatment, healthcare, mental health, and welfare. One additional domain assessed offender contributions to state and local government through taxes on earnings (income tax) and purchases (sales tax). However, two social service areas, mental health and welfare, could not be assessed under the analytic design of the cost analysis due to insufficient data coverage over the time frame of the analysis (mental health) or because of competing policy changes that could not be disentangled from those of SACPA (i.e., welfare reform). Hence, cost results are based on the remaining eight areas. The savings and costs profile across all areas represents the net savings (or costs) that can be attributed to SACPA. Background SACPA was enacted into law by California voters as a statewide policy that changed the course of criminal justice processing for all eligible offenders, whether or not they chose to participate in the program. The policy also affected all service entities that interact with the pool of eligible offenders. The most rigorous and conservative scientific approach required the construction of a comparison group. Since the most preferred study design was not possible, one in which similar offenders are randomly assigned to either SACPA or nonSACPA interventions, a comparison group was constructed by selecting similar offenders convicted of SACPAeligible crimes from a period before SACPA was implemented. UCLA compared the total statewide costs for drug offenders eligible for SACPA (N=61,609) during its first year to total statewide costs for a selected comparison group of drug offenders before SACPA was initiated (N=68,543). 7 The analytic approach used is a significant improvement to that of cost studies limited to single group, pre/postdesigns, such as the California Drug and Alcohol Treatment Assessment (CALDATA) and the California Treatment Outcome Project (CalTOP). 8 The SACPA cost analysis also improved on such studies by using official records for data sources, thus removing the need to rely primarily on subject selfreport. Finally, the study used lengthy pre and post periods, thus limiting the effects of “regression to the mean,” which can spuriously inflate post intervention benefits. 9 The cost analysis is comprised of three studies. Study 1 calculated the net savings (or costs) and benefit to cost ratio attributable to SACPA as a policy applied statewide to all eligible offenders. Study 2 examined variation in cost ratios in relation to offenders’ degree of participation in SACPA. A particular focus of Study 2 was the benefit to cost ratio for offenders who completed their SACPA drug treatment program. Study 1 and Study 2 were based on SACPA’s firstyear population and covered a 30month preperiod and a 30month postperiod from the eligibility conviction. Study 3 examined the change in net savings (or costs) from the first to the second 7 While the comparison and SACPA groups had different sample sizes, these were utilized only to obtain per offender costs in the eight areas. Once these were determined, the calculation of total costs was rebased to the SACPA sample size. 8 Gerstein et al. (1994). Evaluating recovery services: the California Drug and Alcohol Treatment Assessment (CALDATA). California Alcohol and Drug Programs. https://www.adp.cahwnet.gov/pdf/caldata.pdf. Hser et al. (2005). The California Treatment Outcome Project (CalTOP) Final Report. UCLA Integrated Substance Abuse Programs. http://www.uclaisap.org/caltop/FinalReport/Cover%20Page.pdf. 9 “Regression to the mean” refers to the tendency for individuals with below average problems and costs in one period to have more problems and higher costs in the next period, and vice versa for those with above average problems and costs. Many individuals enter treatment when they have the most problems (Harwood et al., 2002). This is especially true for individuals entering treatment under a court mandate following a conviction. It is quite possible that in the absence of the treatment intervention, the client would have improved on a number of outcome measures, i.e., part of the beneficial pre/post change would have been observed anyway. 7 year of SACPA to assess if maturity of the policy may have changed cost outcomes. This analysis made it possible to more precisely compare each year’s costs to the $120 million annual allocation provided for drug treatment and other services by SACPA. 10 Study 3 provides a replication of the firstyear analysis, and confers greater confidence in the results of Study 1. However, since the secondyear cohort was drawn from a more recent period than the firstyear cohort, there was a shorter time period available for followup. Therefore, Study 3 used 12 month pre and postperiods before and after the SACPA eligible conviction in order to capture equal periods for comparison of the first and secondyear SACPA offenders. As noted earlier, all three studies used the “taxpayer perspective,” in which the focus is on costs to state and local governments. All costs were adjusted to 2004 dollars using the consumer price index or, where appropriate, the medical price index. 11 Cost amounts have been rounded to the nearest dollar. The findings, conclusions and recommendations, and analytic methods are summarized in this report and its appendices. Study findings are presented in the first section, followed by conclusions and recommendations. A final section describes the analytic design employed, the data utilized, and methodological techniques applied. Four appendices supply pertinent detail. FINDINGS SACPA First Year Cost Analysis (Study 1) Study 1 compared offenders eligible for SACPA to a before SACPAimplementation group of offenders who would have been eligible for SACPA under the law’s provisions. 12 The purpose of this analysis was to calculate the cost attributable to SACPA as a policy. The SACPA group was the population of adults (age 18 or older) who were, during SACPA’s first year (July 1, 2001, to June 30, 2002), convicted of a SACPAeligible offense with no concurrent nondrug offense or other circumstance that made them ineligible. The 30month followup period for each SACPA offender ended on or before December 31, 2004. The comparison group 13 was drawn from a population of adults convicted of an offense for which they would have been SACPAeligible had they been convicted after SACPA was implemented with no concurrent nondrug offense or other circumstance that would have made them ineligible. This population of offenders was convicted between January 1, 1997, and June 30, 1998. The 30month follow up period for all comparison offenders ended on or before December 31, 2000, at least 6 months before SACPA may have begun to affect the involved systems. Findings covered a 30month pre and postperiod beginning with the date of each offender’s conviction. 10 The study conservatively assumes programmatic costs to be $117 million under SACPA, excluding $3 million of the $120 million annual allocation that was used to cover statelevel administrative expenses. 11 U.S. Department of Labor, Bureau of Labor Statistics. Consumer Price Index: 19132005, [data file]. Washington, DC: www.bls.gov/data. 12 Offenders were drawn from official California Department of Justice records on arrests and convictions with subsequent computerized eligibility screening. These numbers are larger than those estimated in prior reports, which were obtained from stakeholder surveys or the centralized SACPA Reporting Information System (SRIS) derived from county inputs. 13 Because the comparison group was, of necessity, drawn from a different period of time, it is technically known as a timelagged comparison group. 8 This section first reports the differenceindifferences, or DID, calculated as the difference between (1) the SACPA group’s preconviction and postconviction difference in costs and (2) the comparison group’s preconviction and postconviction difference in costs (see Research Methods). This yields a DID average cost per offender in each cost area. Later, the cost profile of SACPArelated costs or savings across all eight areas is reported. Cost per offender The estimates below reflect regression adjusted average 14 (mean) savings or costs per offender for the total comparison and SACPA groups in each cost category. 15 Costs were calculated based on actual events, as captured in state administrative data bases, multiplied by the costs associated with the event, as determined from actual data or from published sources. The figures report costs in the pre, or baseline, period, costs in the post, or followup, period, differences from pre to post for each group, and the DID between groups (costs are positive numbers and savings are negative numbers). Because the number of jail and prison days served by SACPA offenders was similar to the full census of a midsize facility, the average cost of a jail or prison day was used rather than the marginal cost. The full assumptions and statistical techniques underlying these estimates are provided in the Research Methods section. Prison Prison incarceration costs are shown in Figure 1. Cost per offender increased by $2,390 over a 30month baseline (pre) period for the SACPA group and by $5,937 for the for the comparison group, which led to a DID prison cost savings of $3,547 during SACPA. This indicates that prison costs in California was $3,547 lower per offender for the followup 30month period than what would have been expected had SACPA not been implemented. For the 61,609 offenders eligible for SACPA in its first year, the total savings to the state in prison costs were $218.5 million. 14 Cost distributions were not normally distributed, so the average cost in each category was above the median cost due to a relatively small number of offenders with very highcost events (e.g., armed robbery). However, government pays such costs and using the average cost better captures their cost consequences. 15 These are the covarianceadjusted values obtained by using multivariate regression techniques on offender characteristics and contextual trends in the larger society (see Research Methods). Totals and differences reported here may differ slightly from what a reader may compute from other reported numbers due to rounding. 9 Figure 1. Prison Costs $10,000 $9,187 Dollars per offender $9,000 $8,000 $6,467 $7,000 $5,937 $6,000 $5,000 $4,000 $4,077 $3,250 $3,000 $2,390 $2,000 $1,000 $0 Comparison Pre SACPA DID = $3,547 Difference Post Notes: Data for number of days served in prison are from the OffenderBased Information System. Cost of a prison day ($84.74) was obtained from the California Department of Corrections (2005). Jail Jail incarceration costs are shown in Figure 2. Cost per offender increased by $1,500 over baseline during SACPA and by $3,031 for the comparison group, a DID jail cost savings of $1,531. This indicates that jail costs under SACPA was $1,531 lower per offender during the 30month followup period than would have been expected in the absence of SACPA. Total savings in jail costs to counties for firstyear SACPA offenders were $94.3 million. Figure 2. Jail Costs $5,000 $4,796 $4,500 Dollars per offender $4,000 $3,452 $3,500 $3,031 $3,000 $2,500 $2,000 $1,952 $1,765 $1,500 $1,500 $1,000 $500 $0 Comparison Pre SACPA DID = $1,531 Post Difference Notes: Data for number of days sentenced to jail are from Department of Justice’s Automated Criminal History System. The 2005 ADP County Survey was used to adjust to actual days served. Cost of a jail day was obtained by county from the County Survey and the 2003 California Board of Corrections Survey. 10 Probation The cost of probation supervision is shown in Figure 3. Cost per offender increased by $1,399 over baseline during SACPA and by $1,201 for the comparison group, which led to a DID probation supervision cost increase of $198. This result indicates that probation costs under SACPA were $198 higher per offender for the 30month period than would have been expected in the absence of SACPA. Total additional cost to the counties for probation was $12.2 million. Figure 3. Probation Costs $2,500 $2,255 Dollars per offender $2,018 $2,000 $1,399 $1,500 $1,201 $1,000 $856 $817 $500 $0 Comparison Pre Post SACPA DID = $198 Difference Notes: Data for number of days on probation is from sentencing records in Department of Justice Automated Criminal History System. Cost of a probation day was obtained by county from the 2005 ADP County Survey. Parole The cost of parole supervision is shown in Figure 4. Parole cost per offender increased by $39 over baseline during SACPA and by $260 for the comparison group, a DID parole supervision cost decrease of $221. This indicates that parole costs under SACPA were $221 lower per offender for the 30month period than what would have been expected in the absence of SACPA. This difference was expected, given the lower number of prison days (see Figure 1) served by SACPA offenders. Parole cost savings to the state under SACPA were $13.6 million. 11 Figure 4. Parole Costs $700 $667 Dollars per offender $600 $553 $514 $500 $407 $400 $300 $260 $200 $100 $39 $0 Comparison Pre SACPA Post DID = $221 Difference Notes: Data for number of days on parole is from sentencing records in Department of Justice Automated Criminal History System. Cost of a paroleday ($9.21) is from the California Department of Corrections (2004). Arrests and Convictions Arrest and conviction costs are shown in Figure 5. Although arrest and conviction costs dropped for both groups in the followup period, they did not drop as much for the SACPA group. In large part this was due to the greater amount of time that offenders in the comparison group were "off the street" during the followup period due to incarceration. 16 Since offenders who are incarcerated are unavailable to be rearrested in the community, these differences in street time would be expected to reduce rearrests and convictions to a greater degree in the comparison group than in the SACPA group. 17 Further analysis determined that a disproportionately large share of criminal justice costs was created by 1.6% (N=1010) of SACPAeligible offenders (see Figure 13 and HighCost Offender Substudy for further details). Costs for offenders in this subgroup were 10 times ($21,175) higher than those for the typical (median) offender ($2,254). Costs per offender decreased by $286 relative to baseline levels during SACPA and by $1,612 for the comparison group. DID arrestandconviction costs were $1,326 higher for the 30month period than what would have been anticipated had SACPA not been implemented, resulting in a total increase of $81.7 million in criminal justice processing costs. 16 In the 30month followup period, 71.2% of the comparison group had jail time compared to 58.9% of the SACPA group. Likewise, 28.2% served prison time compared to 23.3% for the SACPA group. The SACPA group also experienced shorter stays when incarcerated. 17 Every judicial decision to place an offender on probation contains a degree of risk that reoffending in the community will occur. This is also true when inmates are paroled. In general, any population of offenders under legal supervision has rates of reoffending that increase in proportion to time on the street. For example, nationwide 55% of parolees and 40% of probationers reoffend at a level that places them into custody (November 2005. Bureau of Justice Statistics Bulletin. U.S. Department of Justice, Office of Justice Programs). Many policy studies regarding the benefit cost of incarceration have been conducted to assess the “balance point” between the high cost of incarceration versus the greater risk of reoffending under lowercost community supervision 12 Figure 5. Arrest and Conviction Costs $6,000 $5,458 $5,431 $5,172 Dollars per offender $5,000 $3,818 $4,000 $3,000 $2,000 $1,000 $0 $1,000 Comparison $2,000 SACPA DID = $1,326 $1,612 Pre $286 Post Difference Notes: Numbers of arrests and convictions are from sentencing records in the Department of Justice Automated Criminal History System. Costs for crime were adjusted from Miller & Cohen (1996) and French et al. (2006). Drug Treatment Drug treatment costs are shown in Figure 6. Cost per offender increased by $1,131 over baseline for the SACPA group and by $368 for the comparison group, a DID increase of $743 per offender, resulting in $45.8 million more in treatment costs than would have occurred without SACPA. Figure 6. Drug Treatment Costs $2,000 $1,800 Dollars per offender $1,750 $1,500 $1,250 $1,131 $891 $1,000 $669 $750 $524 $500 $368 $250 $0 Comparison Pre SACPA DID = $743 Post Difference Notes: Data for number of days in drug treatment, by modality, are from the California Alcohol and Drug Data System (CADDS). Perdiem treatment costs are from Ettner et al. (2005) adjusted to 2004 dollars. Healthcare Healthcare costs are shown in Figure 7. Greater access to drug treatment was associated with a greater utilization of healthcare (see also Study 2). Costs per offender increased by $648 for the SACPA group and by $418 for the comparison group. Such costs were $230 higher per offender 13 for the 30month followup period than would have been anticipated had SACPA not been implemented. Healthcare costs to the state increased by $14.2 million under SACPA. Figure 7. Healthcare Costs $1,800 $1,694 Dollars per offender $1,600 $1,400 $1,293 $1,200 $1,000 $1,046 $874 $800 $648 $600 $418 $400 $200 $0 Comparison Pre Post SACPA DID = $230 Difference Notes: Data for health costs are from DHS MediCal/Medicaid files. Income and Sales Tax Taxes received by government are shown in Figure 8. Tax receipts declined by $149 for the SACPA group and by $208 for the for the comparison group. This resulted in a DID increase in taxes collected under SACPA of $59. A total of $3.6 million was received by the state and county governments for taxes paid on income and purchases. Figure 8. Taxes Paid $2,000 Dollars per offender $1,750 $1,916 $1,866 $1,767 $1,658 $1,500 $1,250 $1,000 $750 $500 $250 $0 $250 Comparison SACPA $208 Pre Post $149 DID = $59 Difference Notes: Data on earnings was obtained from EDD. Taxes were computed using California tax tables and were adjusted to 2004 dollars. Values reported reflect estimates of income taxes and sales taxes paid. 14 In both groups, earnings fell immediately in the first quarter following the eligible conviction but increased in the quarters that followed. For the comparison group, the reduced earnings can be explained by increased incarceration and therefore fewer days available for work. For the SACPA group, the initial reduction in earnings is attributed to offenders being in treatment. SACPA Overall CostOffsets Figure 9 shows a summary of SACPA DID costs over all areas examined. The zeroline can be interpreted as costneutral. Any bar above the line represents a cost increase and any bar below the line represents a cost savings. There was a total DID cost savings of $2,861 per offender under SACPA over the 30month followup period 18 , resulting in a total cost savings to government of $173.3 million. Figure 9. DID Cost Summary for Study 1 $2,000 $1,326 $743 Dollars per offender Dollars per offender $1,000 Prison Jail $198 Parole Probation $221 $230 Tax Health $59 Total $0 $1,000 $1,531 $2,000 $3,000 $4,000 Arrest and Treatment Conviction $2,861 $3,547 Study 1 allowed the calculation of a total DID cost for the population of 61,609 offenders in SACPA’s first year. Before turning to the calculation of the benefitcost ratio, it must be noted that the initial year required a massive rampup effort on the part of the involved county systems. The expansion of existing provider contracts and the development and award of new contracts was, in many cases, a lengthy process. In addition, during this year, state and county governments were coping with the overall budget constraints of a faltering economy. In some counties, nonrecurring funds were used in ways that allowed savings to accrue in the allocated SACPA funds. These savings could then be carried forward into future years. In the first year of the study, 55 of the 58 counties reported 19 a total expenditure of SACPA funds of about $85 million, an amount less than actually spent. Using this figure would have produced a spuriously high benefitcost ratio for the first year. Accordingly, UCLA used an estimate of SACPA operation costs ($120 million less $3 million used for state administrative costs) as a conservative estimate of expenditures (a figure that stabilized in the subsequent years of SACPA). To determine the benefitcost ratio per offender for the first year, total costs over the 30month period (expressed as a negative number, which represents savings) are multiplied by the total number of offenders convicted of a SACPAeligible offense during the first year of SACPA (N = 18 Most of these savings accrued in the first 12 months of this period, although savings continued to accrue over the remaining 18 months of the 30month period. See results of Study 3. 19 Figures cited are from the SACPA Reporting Information System (SRIS). 15 61,609). 20 From this total, the $117 million actually allocated for programmatic costs is subtracted to avoid “double” counting costs that had already been paid for via SACPA expenditures ($120 million less the $3 million used in SACPA administration). In other words, operation costs are a credit to the numerator since counties were paid this amount to offset SACPA costs. The resulting sum is divided by the $120 million allocated for firstyear SACPA costs. In brief, the benefit cost ratio reported is the total savings net of programmatic costs derived from SACPA, divided by the $120 million allocation. 21 For Study 1, UCLA estimated a benefitcost ratio of 2.44:1, indicating that nearly $2.50 was saved under SACPA for every $1 allocated to fund the program. SACPA Drug Treatment Participation Cost Ratios (Study 2) Prior UCLA reports (see Footnote 1) indicated that in its first and second years, 85% of offenders who accepted SACPA reported for assessment. Of these, almost equal percentages entered drug treatment, 69% and 71% respectively. Of those entering treatment, a little over 34% in each year completed their required treatment as reported to the statewide database on drug treatment admissions and discharges. As noted in the earlier reports, these show and completionrates conform to the general literature concerning the treatment of substance abusing offenders. Study 2 examined variation in cost ratios in relation to level of SACPA participation. The study was based on the population of adults (age 18 or older) who, during SACPA’s initial year (July 1, 2001, to June 30, 2002), participated in SACPA—that is, those who accepted a SACPA referral. The population was broken into three groups: (1) offenders who were referred to SACPA but did not enter drug treatment, (2) offenders who entered but did not complete treatment, and (3) offenders who completed treatment. Like the first study, this second study covered 30month pre and post periods beginning with the date of each offender’s conviction. Figure 10 provides a summary of costoffsets by treatment status. The zeroline is interpreted as costneutral. Any bar above the line represents a cost increase and any bar below the line represents a cost savings. Prison Drug treatment participation was strongly associated with reductions in incarceration costs relative to the comparison group costs. Prison costs were $2,459 lower for offenders who never entered treatment, $4,058 lower for individuals who entered but did not complete treatment, and $6,175 lower for offenders who completed treatment than what would have been expected had SACPA not been implemented. 20 Earlier UCLA reports estimated the number of eligible offenders from the Stakeholder Survey for the first year and from the SACPA Reporting Information System (SRIS) for the second (reported by county lead agencies). The cost analysis improved on these estimates by using Department of Justice official records. 21 SACPA programmatic costs are first subtracted from the numerator to avoid double counting of costs. The savings ratio is: Ratio = ((S * N) – P)/A; where S = average savings per offender expressed as a negative amount; N = number of SACPA eligibles; P = programmatic costs (note that it too becomes a negative number when subtracted); A = SACPA allocation. 16 Jail Jail costs were $1,411 lower for offenders who never entered drug treatment, $1,822 lower for individuals who entered but did not complete treatment, and $2,372 lower for offenders who completed treatment. Probation Probation costs were $111 higher for offenders who never entered drug treatment, $329 higher for individuals who entered but did not complete treatment, and $336 higher for offenders who completed treatment. Figure 10. DID Cost Summary by Drug treatment Status Dollars per offender Dollars per offender $4,000 $4,000 $2,000 $2,000 $0 $0 $2,000 $2,000 $4,000 $4,000 $6,000 $6,000 $8,000 $8,000 Arrest & Arrest & Treat Treat Conviction Conviction ment ment Prison Prison Jail Jail Probation Probation Parole Parole No treatment No treatment $2,459 $2,459 $1,411 $1,411 $111 $111 $211 $211 $1,440 $1,440 Some treatment Some treatment $4,058 $4,058 $1,822 $1,822 $329 $329 $245 $245 $1,859 $1,859 Completed treatment Completed treatment $6,175 $6,175 $2,372 $2,372 $336 $336 $225 $225 $552 $552 $2,027 $2,027 No treatment No treatment Some treatment Some treatment Health Health Tax Total Total $48 $48 $154 $154 $45 $2,468 $1,335 $1,335 $260 $260 $44 $2,386 $434 $434 $177 $5,601 Completed treatment Completed treatment Parole Parole costs were $211 lower for offenders who never entered drug treatment, $245 lower for individuals who entered but did not complete treatment, and $225 lower for offenders who completed treatment. Note that there is little variation from the full SACPA population reduction in parole costs, at $221 per offender, reported in Study 1 (see Figure 4). Arrests and Convictions Arrest and conviction costs were $1,440 higher for offenders who never entered drug treatment, $1,859 higher for individuals who entered but did not complete drug treatment, and $552 higher for offenders who completed treatment. UCLA found that SACPA offenders who did not report for treatment were mainly of two types: offenders with low numbers of prior arrests and convictions and offenders with high numbers of prior arrests and convictions. The former group may have felt themselves to be only recreational users and believed they did not require treatment. The latter group may have chosen not to participate in treatment in the belief that sanctions from the criminal justice system were too unlikely or too distant to hold them accountable. 17 Drug treatment As expected, drug treatment costs were higher depending on level of participation. Treatment costs were $1,335 higher for those who entered but did not complete treatment and $2,027 higher for those who completed treatment. Those who did not enter treatment had a $48 lower treatment cost. Healthcare Individuals in treatment are more likely to seek out care for other health needs. Statefunded healthcare costs were $154 higher for offenders who never entered treatment, $260 higher for individuals who entered but did not complete treatment, and $434 higher for offenders who completed treatment. Taxes Paid There was a slight increase in tax revenues collected related to drug treatment participation under SACPA. Tax revenues were $45 higher for offenders who never entered treatment, $44 higher for those who did not complete treatment, and $177 higher for those who completed treatment. Total CostOffset by Drug treatment Status Total costs saved were $2,468 for offenders who were referred to SACPA but never entered drug treatment, $2,386 for individuals who did not complete treatment, and $5,601 for offenders who completed treatment. Treatment and new arrests and convictions costs constituted a major part of cost increases, whereas total costs savings were driven largely by savings in incarceration (jail and prison) costs. While prison and jail costs were reduced under SACPA for those never treated, these savings were offset by higher arrest and conviction costs in the postperiod for this group. Incarceration savings were even higher for the some treatment group, but were offset by an increase in arrest and conviction costs and in higher treatment costs. Cost Comparison Average costsavings per offender were more than twice as high for those who completed drug treatment compared with those who never entered or did not complete treatment. For treatment completers, the cost savings reflect a benefittocost ratio of about 4:1, indicating that approximately $4 was saved under SACPA for every $1 allocated to a treatment completer. Two methods were applied for sensitivity analysis of this result; one resulted in a ratio of 3.9:1, the other 3.8:1. Notably, although SACPA offenders who received some treatment showed reductions in prison and jail time over those who did not enter treatment, these savings were offset by treatment costs and somewhat higher rates for arrests and convictions in the followup period. When only criminal justice costs were considered, the cost savings were as expected: no drug treatment, least; some treatment, intermediate; completed treatment, most. SACPA Second Year Replication (Study 3) Study 3 examined costs in SACPA’s first and second years and compared costs in each of those years to the $120 million annual allocation. Here, costs in SACPA’s first year were based on the firstyear SACPAeligible population (N=61,609), but the baseline and followup periods were restricted to 12 months. Costs in SACPA’s second year were based on the secondyear SACPA eligible population (N=68,883) and a baseline and followup period of 12 months. 18 Figure 11 provides a summary of costs per offender during the 12month followup of the first two SACPA years. UCLA found a slight decline (1.5%) in the total cost per offender in the post period in the second year of SACPA. This decrease was largely attributable to a 6% decrease in arrest and conviction costs for offenders during SACPA’s second year. Figure 11: Summary of PostSACPA Costs for Year 1 and Year 2 Offenders 22 12000 Dollars per offender 10000 8000 6000 4000 2000 0 Prison Jail Probation Parole Arrest & Conviction Health Treatment Total Total Year 1 2396 2035 924 179 2319 656 1126 9635 Year 2 2337 2164 931 167 2179 678 1044 9500 Year 1 Year 2 The estimates for first year and second year SACPA offenders are not directly comparable. The comparison group and the first year SACPA offenders both experienced preperiods with no SACPA policy in effect. However, the second year SACPA offenders have a preperiod during the SACPA era. Nonetheless, the findings in Figure 11 show that average offender costs were very similar across the two years when restricted to equal periods of time for both years. The DID estimates are provided in Figure 12. The zeroline is interpreted as costneutral: Any bar above the line represents a cost increase and any bar below the line represents a cost savings. Figure 12 is not directly comparable to Figure 9, which captured a 30month period. Overall cost savings in both years were nearly identical at $2,300 per offender. The higher number of offenders who were identified as SACPAeligible in the second year (N=68,883), as opposed to the first year (N=61,609), meant somewhat higher total savings to government ($158.8 million) in SACPA’s second year than in its first year ($140.5 million). The benefit to cost ratio for the first year was 2.2 to 1, indicating that $2.20 was saved by government for each dollar spent on SACPA, and 2.3 to 1 for the second year, indicating an additional $0.10 was saved for each dollar spent. (However, this slight increase in the benefit cost ratio cannot be considered a trend; more years of cost data would be necessary to define any such changes as a trend.) 22 For Study 3, tax information was not available for second year SACPA offenders due to delays in data procurement. Since contributions from taxes received by state and local governments were minimal in Study 1 and Study 2, their inclusion here would not have altered results. 19 Figure 12. DID Cost Summary by SACPA Year 1,000 Dollars per offender 0 1,000 2,000 3,000 Prison Jail Probation Year 1 1,879 1,555 48 Year 2 1,827 1,211 45 Arrest & Conviction Health Treatment Total Total 2 555 98 451 2,280 1 313 48 328 2,306 Year 1 Year 2 Parole HighCost Offender Substudy In examining factors contributing to cost, UCLA found that a small percentage of SACPA eligible offenders contributed a disproportionate share of criminal justice costs. As the number of prior convictions increased (0 through 5 or more), criminal justice costs in the followup period also increased for prison, jail, and arrests and convictions. This was true for both the comparison and SACPA groups. The greatest increase in costs occurred between the group with 4 prior convictions and those with 5 or more. UCLA chose this “break point” for group classification. Those with five or more convictions in the 30month period before their SACPA eligible convictions, constituting 1.6% (N=1,010) of the SACPA group, had postconviction crime costs in the 30month followup period 10 times higher than the typical, or median, SACPA offender: $21,175 versus $2,254, respectively. Figure 13 demonstrates this difference. Figure 13. Relative Costs for Highrate Offenders (1.6%, N=1,010) 25,000 Dollars per offender 21,175 20,000 15,000 10,000 5,000 2,254 0 Typical Off ender 5+ Prior Convictions 20 Adult Welfare Substudy Statewide data was made available to UCLA for welfare from the California Department of Social Services. Importantly in regard to the SACPA benefitcost analysis, the welfare system changed dramatically over the period of the evaluation, with major changes in eligibility, benefits, and duration. There was rapid decline in welfare caseloads after 1996. The average annual caseload fell by more than 10% per year between 1997 and 2000. The caseload decline began slowing around 2000 (6.7% decrease in 2001 and a 2.2% decrease in 2002). 24 Such changes limited UCLA’s use of this data, especially since the effects of welfare reform made the data unamenable to identifying effects attributable to SACPA. Analysis of the available data revealed that only about 10% to 13% of each sample (comparison and SACPA) could be identified as having received welfare in either the 30month baseline or followup periods. The change from baseline to followup periods in the comparison group showed a 25.6% drop in recipients and a 38% duration reduction in benefit time. For SACPA, there was a 38.2% drop in recipients and a 26% duration reduction. There is no practical analytic method to separate the known effects of welfare reform from the potential effects of SACPA. CONCLUTIONS AND RECOMMENDATIONS 25 Introduction The conclusions and recommendations of the UCLA evaluation of SACPA are presented in this section. 26 Three major conclusions can be drawn from the UCLA evaluation of SACPA: (1) SACPA substantially reduced incarceration costs; (2) SACPA results in greater cost savings for some offenders than for others, and (3) SACPA can be improved. From these conclusions, specific recommendations are drawn. Each recommendation encompasses goals that require attention at multiple levels, including: Statewide collaboration and coordination, offender eligibility and alternative strategies for highcost offenders, systems integration, criminal justice, drug treatment, and strategic planning. Conclusion 1: SACPA substantially reduced incarceration costs. Based on costs incurred by offenders who were eligible for SACPA participation during its first year of implementation, SACPA’s overall benefittocost ratio was nearly 2.5 to 1 over a 30month followup period, resulting in $173.3 million in net savings to the state and local governments. Over a 12month followup period, SACPA’s overall benefittocost ratio was 2.2 to 1 in its first year and 2.3 to 1 in its second year. Recommendation 1.1: From the state and local government perspective, continued funding of SACPA is justified. 24 California Department of Social Services, 2004. Additional, more detailed recommendations that provide suggestions for resources and mechanisms of action are contained in Appendix B. 26 The evaluation of SACPA required a diverse and detailed methodological approach. UCLA made extensive use of administrative databases on criminal history, substance abuse treatment, healthcare, and employment outcomes. In addition, throughout the evaluation, UCLA sought the expert counsel of the Evaluation Advisory Group (EAG), the views of the Statewide Advisory Group (SAG) and the County Alcohol and Drug Program Administrators Association of California (CADPAAC), and the input of the Department of Alcohol and Drug Programs (ADP). 25 21 Recommendation 1.2: ADP should have statutory authority and responsibility to develop a strategic plan for the ongoing operation and continual improvement of SACPA. Attendant evaluation should encompass continuous and timely feedback processes to stakeholders, the Governor’s Office, and the Legislature. Conclusion 2: SACPA results in greater cost savings for some eligible offenders than for others. In particular, drug treatment completers had a benefittocost ratio of 4 to 1, a savings of $5,601 per offender. In addition, UCLA found that the typical SACPA offender (at the median level in the cost distribution) had no convictions in the 30 months following their SACPA eligible conviction. In contrast, offenders with 5 or more convictions in the 30month period prior to their SACPAeligible conviction produced costs 10 times higher than those of the typical offender. Recommendation 2.1: SACPA criteria should be modified so that offenders with high rates of prior nondrug convictions (e.g., 5 or more prior convictions over the prior 3 years) would be placed into more controlled settings, including, but not limited to, residential treatment or prison or jailbased treatment programs. Recommendation 2.2: Eligible offenders with high drug severity (e.g., histories of serious or lengthy use) should receive greater criminal justice supervision (e.g., drug court management) and more intense drug treatment services (e.g., residential or day treatment). Conclusion 3: SACPA can be improved. UCLA found that assessment rates were better in counties where assessment units or centers were located in or near the court, where offenders were allowed more days to report for assessment, and where assessment and treatment placement occurred in a single visit. Further, assessment and treatment “show” rates were higher in counties using one or more “Drug Court” procedures. At the drug treatment level, residential and methadone modalities were underutilized for high drug severity and opiate dependent offenders. 27 Finally, outcomes were best for SACPA treatment completers. Recommendation 3.1: Based on client assessments and research findings on successful strategies, greater resources should be dedicated to increasing treatment engagement, retention, and completion (see also Recommendation 2.2). Recommendation 3.2: Resources should be allocated to ensure suitable and most favorable drug treatment options locally. This may require capacity expansion, more efficient location and higher utilization of residential services, and greater utilization of narcotic substitution therapy. Recommendation 3.3: Collaboration and coordination among court, probation, parole, and drug treatment systems should continue to be improved with the goal of admitting offenders into appropriate treatment in the shortest possible time, as well as maintaining appropriate levels of oversight and supervision. 27 Limitations in the use of these two treatment modalities may have been constrained by countylevel slot or funding restrictions. 22 Recommendation 3.4: Incentives should be considered for providers who demonstrate more success in drug treatment engagement, retention, and completion for SACPA clients. Recommendation 3.5: A greater utilization of both probation and community program drug testing information should be used to determine additional services and/or intermediate sanctions of increasing severity for problematic or recalcitrant offenders. Such sanctions could include initially short jail stays that lengthen with each successive violation. 28 Several additional issues merit close attention. First, insufficient consideration was given to implementing drug treatment “aftercare” or “continuity of care” in SACPA. Proven models for continuing care should be communicated to drug treatment providers for incorporation into their clinical process. Second, with Proposition 63 29 funding now established, consideration should be given to applying some of these funds to services for SACPA offenders with cooccurring mental health disorders. Third, the use of administrative databases has proven essential and productive in evaluating SACPA and other statewide policies. A concerted, collaborative effort should be made to streamline access to and use of centralized state data for authorized policy and evaluation studies. Finally, further policyrelevant substudies on accumulated and new SACPA data should be conducted to complement findings from the cost analysis. RESEARCH METHODS A number of both common economic analytic issues and distinct research issues were addressed in the methodology used for the SACPA cost analysis. These included design, data, analytic, and other aspects that were resolved on the basis of common econometric practice, logic, or empirical assessment. A detailed discussion is presented below. Design From the perspective of general econometric research, the SACPA evaluation is among the few studies of major policy change involving drug treatment in which a timelagged 30 comparison group was constructed, allowing for greater precision and credibility in the findings than the more typical single group, pre to postintervention assessment. One disadvantage of a time lagged comparison group is the possibility that timerelated events in addition to the intervention might differ across the two periods under study. To compensate for this possibility, measured differences between the comparison and SACPA groups and time trends in important variables (e.g., strength of the economy and national crime trends) were statistically controlled in the analyses using covariate adjustments. 31 Moreover, as noted earlier, the SACPA evaluation relied on administrative records as data sources (which are more objective than client selfreports), and 28 The effectiveness of short jail stays has not been firmly established in the research literature. Contextual conditions of sanction application, timing, and perceived fairness require careful consideration of optimal options. 29 California voters approved Proposition 63, the California Mental Health Services Act, to provide additional funding for mental health services. Funds became available to the counties in January, 2006. 30 Although similar in SACPAeligible conviction and in offender characteristics, the fact that the comparison group was derived from a period before SACPA was implemented requires the term “timelagged” as a qualifier. 31 “Covariate adjustments” here refers to a standard statistical procedure that allows us to adjust for imbalances in baseline variables that may be related to the outcomes under consideration. The adjustments help correct for potential differences in the groups' predispositions that may cause them to behave differently from the outset, for example, the effect of differences in the economic climate on the groups’ access to employment opportunities. 23 on a more extended baseline and followup period of observation. Overall, these improvements in design, analytic procedures, and data sources lend additional credibility to the findings over other cost determination approaches. The three studies were designed to answer the research questions enumerated in the request for evaluation proposals, as amended by suggestions from UCLA Integrated Substance Abuse Programs (ISAP) staff and consultants. In addition, later adjustments were made based on the cumulative suggestions by the SACPA Statewide Advisory Group and the SACPA Evaluation Advisory Group. The final set of research questions related to costs is specified in Appendix A. This section describes the comparison group adopted in the cost analysis and explains the rationale for analytic time frames. Steps taken to identify SACPA’s cost as precisely as possible are explained, as are the cost sources and issues addressed over the course of the analysis. Comparison Groups The evaluation’s three cost studies served different purposes. UCLA employed comparison groups and time periods as appropriate to each purpose. Study 1 – SACPA as a Policy Study 1 compared (1) offenders eligible for SACPA and (2) a before SACPAimplementation group of offenders who would have been eligible for SACPA. The purpose of this study was to calculate the cost attributable to SACPA as a policy. Findings cover a 30month period before and a 30month period after the date of each offender’s conviction. The SACPAera group was the population of adults (age 18 or older) who were, during SACPA’s first year (July 1, 2001, to June 30, 2002), convicted 32 of a SACPAeligible offense with no concurrent offense or other circumstance that should have made them ineligible. SACPA eligibility is determined at sentencing and depends on the nature of the offense(s) for which a conviction is entered. Thus, convictions provided the best possible indicator of eligibility for SACPA. The term “SACPAera” is used because, since the law had statewide applicability, all offenders convicted of a SACPAeligible offense were included in the population, whether they actually participated in SACPA or not. A timelagged comparison group was drawn from the period before SACPA was enacted. The comparison group consisted of adults convicted of an offense for which they would have been SACPAeligible with no concurrent offense or other circumstance that should have made them 32 An alternative indicator of SACPA eligibility is arrest rather than conviction. This alternative was considered because the conviction is “farther from the crime” than the arrest. That is, convictions reflect not only the arrest itself but also the postarrest behavior of actors in the criminal justice system (e.g., the decision to file charges and the parameters of plea bargaining) and of offenders (e.g., acceptance of a plea bargain that renders the offender eligible or ineligible for SACPA). In addition, data on convictions (dispositions) were missing for 30%40% of SACPAera and comparison offenders arrested for SACPAeligible offenses, indicating a possibility that adjudication did not proceed after the arrest. On the other hand, a majority of offenders (66%) arrested for an eligible offense in SACPA’s first year did not participate in SACPA, and an unknown portion of them may have been ineligible for SACPA at sentencing. Compared to convictions data, arrest data may therefore provide a much weaker indication of SACPA’s cost. There was no empirical basis at present for resolving these considerations. The decision to use convictions as the indicator of SACPA eligibility was therefore made on logical grounds— namely, SACPA eligibility is determined upon conviction, not arrest. 24 ineligible. This population was offenders convicted between January 1, 1997, and June 30, 1998. The 30month followup period for each comparison offender therefore ended on or before December 31, 2000. This enddate minimized the chance that the larger society background trends in drug use, law enforcement patterns, or other factors might have influenced findings (the before SACPA period is as close to the SACPA years as possible), and the follow up period for that population ended in advance of any change in criminal justice practice that might have occurred in anticipation of SACPA implementation on July 1, 2001. The cost study of SACPA as a policy is most important because SACPA is a voluntary program; persons convicted of a SACPAeligible offense may accept or decline the opportunity to be sentenced to SACPA. The voluntary nature of SACPA participation opened the possibility that offenders who accepted SACPA might be different in ways that could affect their costs. In program evaluation, this problem is known as “selfselection bias.” To avoid such bias, the program group must be constructed from all those eligible for the program, whether or not they selfselected into it. In the case of SACPA, the prior event closest to an offender’s decision to accept or decline SACPA is conviction on a SACPAeligible offense (or, for the comparison group, conviction on an offense for which they would have been SACPAeligible). Additional steps to ensure comparability of the SACPAera and comparison groups are described later under Causal Inferences. Study 2 – Degree of SACPA Participation The purpose of Study 2 was to determine how cost ratios varied in relation to offenders’ degree of participation in SACPA. The study of variation in cost ratios in relation to SACPA participation was based on the population of adults (age 18 or older) who, during SACPA’s initial year (July 1, 2001, to June 30, 2002), participated in SACPA—that is, those who accepted a SACPA referral. This population was identified on the basis of three sources: a disposition in the criminal justice record indicating that the offender was referred to SACPA upon conviction; a SACPAreferred admission noted in the California Alcohol and Drug Data System (CADDS), and county records indicating SACPA participation. The population was divided into three groups: (1) offenders who were referred to SACPA but did not enter drug treatment, (2) offenders who entered but did not complete treatment, and (3) offenders who entered and completed treatment. Compared to the more traditional drug treatment evaluation study based on a single group design, as was the case in CalDATA and CalTOP (see Footnote 8), this comparison was more precise since the source of data was official records and the baseline and followup periods were lengthy. Like Study 1, Study 2 covered a 30month baseline and followup period beginning with the date of each offender’s conviction. Study 3 – Replication of SACPA Outcomes Study 3 reported costs in SACPA’s first and second years and compared costs in each of those years to the $120 million annual allocation. Costs in SACPA’s first year were again based on the firstyear SACPA population, defined on the basis of referral to SACPA (as in Study 1), but the followup period was restricted to 12 months. Costs in SACPA’s second year were based on secondyear SACPAeligible offenders and used an equivalent followup period of 12 months. A 12month cutoff sacrificed the advantages of longer followup (see Rationale for Analytic Time Frames) but created an opportunity for replication across the first and secondyear 25 populations. Given data reporting lags and the timetable for evaluation reporting, it was not possible to construct a followup period longer than 12 months for the secondyear population. The Study 3 replication established the extent to which SACPA costs were similar across the two years and aided in interpretation of findings from Study 1 and Study 2. Specifically, Study 3 answers the question: Do the second year findings suggest that costs in the first year were in any way atypical? Substudies: Highcost Offenders and Welfare Two issues emerged in the overall cost analysis that merited further attention. One explored that group of SACPAeligible offenders who contributed disproportionately to costs in the 30month follow up period (see Findings). The second issue arose from welfare payments to SACPA eligible offenders. Here the impact of welfare reform during the SACPA evaluation period created such shifts in the data that a valid DID cost assessment could not be conducted. However, descriptive information about the welfare status of the samples is provided in Findings. Rationale for Analytic Time Frames A followup period of 30 months had two important advantages. First, 23% of offenders in the comparison group and 9% of SACPAera offenders were sentenced to jail or prison due to their SACPAeligible conviction. This difference reflects one key aspect of the policy change being evaluated. That is, offenders who would have been eligible for community supervision and drug treatment after SACPA was implemented were, in the years before SACPA, more likely to be incarcerated. Median time served in jail was 2.5 months and median prison time was 12.9 months for offenders in the comparison group. During incustody months, there were incarceration costs (which include the cost of custody, healthcare, and other services delivered in jail or prison) but no other costs. In particular, while incarcerated, offenders had no opportunity to commit new offenses in the community. This fact was important in Study 1 because re offending was a contributor to total costs. Over a followup period of 30 months, however, the typical comparison offender who began the period in jail or prison will have been back “on the street” for 17 months or more and will therefore have had time in which to reoffend. Assessment of costs under SACPA was therefore more complete with a 30month followup period than would have been possible in a shorter period. Second, SACPA may show elevated costs of healthcare and social services over the first few months of an individual’s participation in a drug treatment program as he/she finally begins to access services long needed and potentially important to recovery. In a followup period as long as 30 months, the effects of any shortterm “blip” in service access will have receded. Coverage of costs under SACPA and before SACPA implementation was therefore more reliable in a 30 month followup period than would have been possible in a shorter period. With a 12month followup, Study 3 was more limited in its coverage in that potential costs (or savings) beyond this period could not be considered during the evaluation time frame. For this reason, major conclusions about the total cost and cost ratios were based on Study 1 and Study 2. 26 Study 3 still has value in showing whether costs were similar across years. Importantly, if benefitcost ratios are similar across the two years, the findings show the stability of cost outcomes. Causal Inferences Although it was the best design possible, even with close time proximity for the SACPA and comparison groups a timelagged comparison group retains some analytic problems. For example, problems were identified due to changes in data system efficiencies and/or coverage. In the instance of two of the intended cost areas, mental health and welfare, lack of administrative data for the former in the baseline period for the comparison group and competing changes caused by welfare reform for the latter eliminated these data from DID cost estimation. To remediate other analytic issues due to sample composition and time lag, a twofold approach was employed to strengthen causal inference from the analysis, i.e., to isolate the SACPA cost as precisely as possible. First, a widely accepted differenceindifferences econometric modeling approach was used to compare offender groups. Second, covariance adjustments were used at two levels in an effort to, first, statistically minimize the effects of offender background characteristics and, second, to correct for general contextual conditions in which certain time trends in events (such as overall state economic conditions and changes in crime statistics that were unrelated to SACPA) could have varied significantly over the full period of comparison. Such variation might have had extraneous and/or spurious effects on findings. However, despite including trends of several of the important variables as covariates, such statistical corrections may not have fully eliminated noncomparability of data. The DID and covariance adjustment approaches are described in greater detail below. Difference inDifferences (DID) Each of the three studies used a DID approach. This approach is common in econometric analyses of data from randomized experiments or in natural experiments such as SACPA, where a timelagged comparison group was constructed. It is important to understand the DID approach fully before interpreting findings and conclusions. Offenders in each group had a preperiod and a postperiod. The preperiod covered 30 months before the date of the offender’s conviction for a SACPAeligible offense. The postperiod covered 30 months beginning on the date of that conviction. Costs were calculated for each offender in each period and were based on two cost elements: quantities and prices. Quantities are a count of events, such as days in drug treatment, days in prison, and quarters of earned income. Events were counted for each offender’s preperiod and postperiod. A price, based on one or more authoritative sources or from available relevant data, was assigned to each event and multiplied by the count of that event, leading to a cost in that category for each period for each offender in each group. All prices were expressed in 2004 dollars using the consumer price index or, as appropriate, the medical price index. The first difference in the DID approach is between the preperiod and the postperiod for each offender. The preperiod represents each offender’s baseline “costliness,” and the postperiod indicates the offenders “costliness” in the followup period. The difference shows the degree to which costs for that offender rose or fell over a standard followup time frame. By finding the differences in costs for the preperiod from costs in the postperiod for each offender, the 27 analysis was able to compare postperiod costs after adjusting for the costliness that each offender came in with; that is, each offender served as his or her own control. The second difference in the DID approach is the difference between groups. That is, did the prepost change for all offenders in each group indicate lower or higher per offender costs in one group or the other? Appendix C contains an illustration of a hypothetical DID example. Covariance Adjustment In all studies, the comparison and SACPA group analyses were conducted after controlling for offender background characteristics. These characteristics included age, gender, race/ethnicity, prior drug treatment history, prior criminal history, and the offender’s home county. UCLA used covariance adjustment to remove any extraneous effects of these characteristics on findings. As noted, contextual conditions in the larger society also could have differentially affected the two groups. Adjustments were made for two of the most salient ones possibly affecting SACPA outcomes: national crime trends and the state unemployment rate. Other such contextual trends could be hypothesized to also affect one or the other of the groups differentially over the lagged time frames used. One such trend was the rapid change in welfare benefits over the study period under the impetus of welfare reform policies and practices. This last trend could not be corrected for by covariance adjustment since data shifts due to welfare reform could not be disentangled from those of SACPA. Costs Covered As noted, the analysis covered costs in eight domains (cost categories): five in criminal justice (jail, prison, probation, parole, arrests and convictions), two in social services (drug treatment, healthcare), and one related to taxes paid (taxable earnings and sales tax). The count of events that served as a basis for cost calculation was obtained from California administrative databases. The extent to which such data contain errors potentially affecting reliability and validity varies across data source and depends on the complexity and quality of the data collection, entry, and internal correction processes. Since such data issues in the utilized databases apply uniformly to both the SACPA and comparison groups, the DID approach essentially removes any relative effect of data errors on findings. This section provides a summary of costs in each cost category. These categories cover the majority of cost domains cited in the evaluation’s cost research questions that were formulated throughout the evaluation planning (see Appendix A). It should be noted, however, that the cost analysis could not fully address all of the questions formulated by SACPA stakeholders and advisors. For example, UCLA determined that the number of offenders diverted from prison would likely fill a medium sized prison. However, prison overcrowding in California is at such a high level at present, that the most likely effect of SACPA would simply be a relief on the degree of overcrowding. Whether prison construction can be deferred or eliminated thus becomes a more longterm and broader question, and one beyond the scope of the SACPA evaluation. Jail and prison incarceration costs were based on actual days served in custody (not days sentenced) for each offender. Unlike prison days, where a release date could be obtained from state administrative data, days in jail had to be imputed from sentenced time as reported in administrative data, adjusted by percent of sentence time served, which varied by county depending on local policy and budgets. A survey of counties by ADP and UCLA obtained each 28 county’s best estimate of percent of time served. The price of a jail day also varied by county and was costed accordingly. Jail costs were assigned to each offender based on the offender’s county of conviction. The price of a prison day was assumed to be the same for all offenders confined in any of the state prisons. Because the number of jail and prison days served by SACPA offenders was similar to the full census of a midsize facility, the analysis used the average cost of a jail or prison day rather than the marginal cost 33 Probation and parole costs were based on supervision days for each offender. Dual supervision was not taken into account. The price of a probation supervision day varied by county and was priced accordingly. The price of a parole supervision day was assumed to be the same across counties. All arrests and convictions occurring in the state of California were recorded. 34 Arrest and conviction costs covered felonies and misdemeanors in all crime categories (drug, property, violent, and other) as well as motor vehicle accidents resulting in arrest. In analyses using the taxpayer perspective (which focuses solely on costs to state and local governments), arrest and conviction costs included all case processing costs and that portion of victim services (medical care, ambulance services, mental healthcare, police/fire services, and victim services) likely to have been paid by public sources. Police and sheriffs’ costs were based on arrests because an arrest leads to caseprocessing costs whether or not a conviction later occurs. However, victim service quantities, superior courts, and county prosecutor costs were conservatively costed on the basis of convictions because the formal determination that an offender committed the offense leading to such costs is represented by conviction, not simply arrest. 35 Drug treatment for each offender was costed by treatment modality (outpatient, methadone maintenance, or residential) received by that offender because modalities can vary substantially in price. Days in treatment were obtained from the California Alcohol and Drug Data System (CADDS), maintained by the California Department of Alcohol and Drug Programs. In some cases, discharge information was missing. An imputation of days of treatment for each modality was used in such cases based on the average duration calculated for those having a discharge date. Healthcare service costs covered medical care claims as paid by the state MediCal and Medicaid system. 33 Marginal costs reflect only those costs due to increased staffing, feeding, medical care, and other costs associated with the number of offenders incarcerated. Average costs also include those for facility construction and maintenance. 34 The evaluation did not have access to information on outofstate offending. 35 While the taxpayer perspective is used for the cost analyses, analyses using the broader “social planner perspective” are also possible. The social planner perspective includes costs to the public as well as to state and local governments. For example, in the calculation of costs attributable to new crimes the social planner perspective included case processing and victim service costs included in the taxpayer perspective detailed above as well as property loss or damage, pain and suffering, and quality of life impacts incurred by victims themselves and perhaps in part by private insurers. 29 Taxable earnings as reported to the California’s Employment Development Department (EDD) for each offender were calculated in order to arrive at amounts paid in state income tax and local sales tax. 36 This cost domain is on the “plus” side. That is, the overall cost in the SACPA and comparison groups was reduced to the extent that offenders in those groups paid taxes. Finally, welfare status was obtained from the Department of Social Services. Costs actually paid were not directly available. As noted, these data could not be used for the cost analysis because of the confounding effects due to welfare reform. Some descriptive data is provided, however, in Findings. Analytic Issues Several analytic issues had to be addressed in the cost analysis. Some were decided on the basis of standard practice in econometrics or on consensual professional judgment. Others were not amenable to a single decision, either because none of the plausible alternatives was clearly superior or because the alternatives might have affected findings to a degree too large to be left unexplored. UCLA used sensitivity analyses to address both of these issues. Sensitivity analysis allows possible alternate scenarios to be explored and their effects on cost variability determined. Ideally, results from sensitivity analyses tend to converge on those from the chosen analysis, and add to the confidence in those results. Results reported here were well within the midrange of results using alternate modeling assumptions. The UCLA analysis was conducted using STATA 9, a program that is frequently used in econometric modeling and that has the appropriate features to allow for robust estimation using multiple modeling techniques. A variety of methods are also available to add statistical control to the analyses. The two modeling approaches chosen for this study were: General Least Squares and Generalized Linear Models. We chose to report the results of the General Least Squares analysis here due to ease of interpretation. However, the Generalized Linear Models approach was used in UCLA’s sensitivity analysis and those results did not affect findings to any significant degree. 36 The social planner perspective includes the full value of foregone employment earnings, whereas the taxpayer perspective considers only consumption taxes and the income taxes paid on earnings. 30 Appendix A: Research Questions UCLA used administrative data maintained by state agencies and collected unitcost information from drug treatment, criminal justice, and other sources in order to measure costs and cost savings and to evaluate the adequacy of funds appropriated. However, some of the questions posed by SACPA stakeholders related to Research Question 2 cannot be directly answered by UCLA’s findings given the data available. Counties have made reports of such expenditure allocation, but these data have not been verified by UCLA. Frequently, the counties were not able to precisely specify the amounts allocated to these various services. Thus, the findings inform the broader discussion concerning changes in SACPA and renewal of appropriations (see Conclusions and Recommendations). Research Question 1: Does SACPA lead to cost savings? Subquestions 1.1 to 1.7 cover components of costs and cost savings. The difference in cost for SACPA offenders and comparison offenders will be calculated for each component and combined across all components to determine whether SACPA leads to net cost savings. Subquestion 1.8 pertains to possible averted costs of prison and jail construction, and those costs will be calculated separately. Subquestion 1.1: Drug treatment costs and cost savings. What are the drug treatment costs for SACPA offenders versus comparison offenders? Subquestion 1.2: Services costs and cost savings. What are the healthcare and social service costs for SACPA offenders versus comparison offenders? Subquestion 1.3: Case processing costs and cost savings. What are the law enforcement, prosecution, defense, and court costs for SACPA offenders versus comparison offenders? Subquestion 1.4: Probation costs and cost savings. What are the probation supervision costs for SACPA offenders versus comparison offenders? Subquestion 1.5: Parole costs and cost savings. What are the parole supervision costs for SACPA offenders versus comparison offenders? Subquestion 1.6: New crimes costs and cost savings. What are the costs of new crimes by SACPA offenders versus comparison offenders? Subquestion 1.7. Incarceration costs and cost savings. What are the costs of jail and prison incarceration for SACPA offenders versus comparison offenders? Subquestion 1.8. Construction. Does SACPA lead to a cost savings from prison and jail construction delayed or averted? 31 Research Question 2: Does the enacted SACPA allocation cover the cost of drug treatment, other services, case processing, and supervision of SACPA offenders? Subquestion 2.1: SACPA allocation. What percentage of the cost of drug treatment, other services, case processing, probation supervision, and parole supervision (measured in Subquestions 1.1 to 1.5) is covered by the SACPA allocation? 32 Appendix B: Expanded SACPA Evaluation Conclusions and Recommendations 37 Introduction The main report highlighted the major conclusions and recommendations from the SACPA evaluation. The information in this appendix is intended to further six overarching goals that UCLA recommends should be salient in all policy discussions and planning regarding SACPA. These goals should be taken as the predominant background context for determining potential actions by and across all agencies involved in SACPA implementation and modification. The background goals for improving SACPA include: further improving systems integration across criminal justice agencies and drug treatment systems within the counties; optimizing offender suitability, acceptance of the treatment alternative, and accountability; increasing offenders’ engagement and retention in and completion of drug treatment; improving matching of severity of dependence to intensity of services, with broader and more accessible services for special populations; developing collaborative methods to increase continuity of care and provision of aftercare; and establishing a structure for strategic planning and continuing systems improvement. Attention to these background goals requires multiple and synchronized action in the areas of statewide coordination and collaboration, offender eligibility and alternate strategies for high cost offenders, systems integration, criminal justice, drug treatment, and strategic planning as elaborated below. Suggested mechanisms are presented within the various levels of government and responsible agencies charged with SACPA implementation to achieve improved outcomes in areas identified throughout the evaluation. In framing these recommendations, UCLA relied on the cost analysis and on prior UCLA evaluation reports. UCLA also drew from other published reports on various aspects of SACPA 38,39 and on recent relevant reviews of research findings on treatment for substanceabusing offenders. Statewide Collaboration and Coordination ADP should be given the statutory authority and responsibility to assume greater oversight for SACPA. UCLA recommends that a strategic plan be developed by a committee of ADP representatives along with a group of expert consultants representing criminology and drug treatment researchers, law enforcement, provider organizations, and other stakeholders. 40 The 37 Contextual information on the historical development of the SACPA initiative and its implementation is provided in Appendix D. 38 Cited reports can be accessed at: http://www.adp.cahwnet.gov/SACPA/P36_Reports.shtml. 39 Notably, the Avisa Group report entitled “Proposition 36 Today: A Study of California Stakeholders in 10 Counties” noted that: "Respondents offered a number of suggestions . . . including changing the eligibility criteria to screen out clients presumed or shown to be little interested in recovery and adding graduated sanctions, which they felt would remove clients with little interest in recovery from the program more quickly;" "Shortening waiting times and thereby accelerating the initiation of the treatment process and successfully enrolling more of the Proposition 36 offenders in appropriate treatment;" and "Inadequate access to residential treatment was mentioned by 17 percent of respondents" (in response to open ended question "What aspects of Proposition 36 could work better in your county?" "Many, however, were insistent that increased funding be tied to better measurements of treatment results and other Proposition 36 processes in order to target the increased funding and allocations to more effective practices and programs." 40 During the first five years of SACPA, a Statewide Advisory Group (SAG) was constituted by ADP to perform some of the functions recommended here. The SAG could be reconstituted and realigned to encompass this broader role. 33 committee should have as one major goal the development of a longrange strategic plan, guided by past and ongoing research on SACPA, to improve the program in each successive year. 41 Since the continued improvement of SACPA will likely require the adoption, adaptation, implementation, and evaluation of researchbased elements, the committee should explore ways to approach the federal government and major California and national foundations for resources to assist in this process. Since the value of using administrative databases has been repeatedly shown in evaluation research and demonstrated amply in the SACPA cost report, the committee should devise and implement standard methods for research access to such information across all state agencies. The SACPA cost evaluation was constrained in accessing necessary data by diverse and time consuming requirements of each separate state agency. For example, one agency believed that their interpretation of the relevant law required denial of data access. ADP was then required to add text to a trailer bill that assuaged the agency’s hesitations about the legal status of supplying the data. In addition, repeated attempts to clarify data anomalies were required. A concerted, collaborative effort should be made at the state level to streamline access to and use of this information for authorized policy and evaluation studies. Offender Eligibility and Alternate Strategies for Highcost Offenders An important finding from the UCLA cost analysis was that relatively few SACPA offenders were responsible for a disproportionately large amount of arrest and conviction costs. The “typical,” or medianlevel, SACPA offender did well in terms of postperiod arrest and conviction costs. The typical offender had one drug arrest, no arrests for property, violent, or sex crimes, and no drug, property, violent, or sex crime convictions in the 30 months following his or her SACPAeligible conviction. As such, the typical offender contributed little to arrest and conviction costs. At the other end of the cost distribution, however, a small percentage of offenders (1.6%, N=1010) accounted for a disproportionately large share of these costs. 42 This is a common finding in criminology, and efforts to contain such highcost and highrate offenders are often the focus of policy and practice initiatives (e.g., California’s Three Strikes law). Individuals with five or more convictions in the 30month period prior to their SACPAeligible arrest (some 70% of the highcost offenders) were, under current procedures, just as eligible to be referred to SACPA as offenders with limited criminal justice histories. However, these offenders had twice as many drug convictions in the 30month followup period, 3 times as many property crimes and violent crime convictions, and 11 times as many sex crime convictions as the average, as opposed to median, SACPA offender. The per offender arrest and conviction cost in the follow up period by offenders with five or more prior convictions was 10 times higher than that of the typical offender. 41 Prior research by other authors supports the use of strategic planning. See: Berry, F.S., Wechsler, B. (1995). State Agencies' Experience with Strategic Planning: Findings from a National Survey. Public Administration Review, 55(2), pp. 159168. Douglas C. Eadie, D.C. (1983). Putting a Powerful Tool to Practical Use: The Application of Strategic Planning in the Public Sector, Public Administration Review, 43(5), 447452. Miller, C.C.; Cardinal, L.B. (1994). Strategic Planning and Firm Performance: A Synthesis of More than Two Decades of Research, The Academy of Management Journal, 37 (6), 16491665. 42 Indeed, a study conducted by the RAND Corporation found that many ostensibly lowlevel drug offenders in prison had more serious criminal histories that their “lowlevel” label suggests (Riley, J.K., Rodriguez, N, Ridgeway, G., BarnesProby, D., Fain, T., Forge, N.G., & Webb, V. (2005). Just Cause or Just Because? Santa Monica, CA: RAND.). 34 Alternatives for processing and/or managing this subset of SACPAeligible offenders need consideration. Possible options include changes at the treatment, eligibility, and criminal justice levels (and these are not mutually exclusive). At the drug treatment level, these offenders could receive residential placement for at least 6 months after SACPA diversion, with supplemental interventions designed to deal with highrate offenders. At the eligibility level—and since SACPA eligibility is set in law, legislative action would be required—offenders with 5 or more prior nondrug convictions in the 3 years prior to the SACPAeligible conviction could be deemed inappropriate for SACPA diversion, and alternate options could be exercised. At the criminal justice level, such offenders could be monitored with greater utilization of drug testing results and reporting requirements. Placing these offenders into drug court monitoring should be considered. Further examination of these and other options (e.g., placement in an inprison or in jail drug treatment program) should occur with representation from the criminal justice, drug treatment, and research fields. Systems Integration SACPA implementation required substantial collaboration among local criminal justice, drug treatment, and other agencies in order to process and serve the drug offenders eligible under the law. In preparation for the law’s implementation, most counties established task forces with representation from the affected agencies (including county administrators, drug treatment providers, judges and court administrators, district attorneys, defense attorneys, and probation and parole representatives) to determine the procedures that seemed most practical and efficacious given local circumstances. Accordingly, while many aspects of implementation were similar across counties, considerable variation resulted. Both common and specific procedures by counties were studied using qualitative and quantitative approaches over the multipleyear course of evaluation, and a number of recommendations have been made in prior SACPA reports (see Footnote 1). We also refer the reader to the Legislative Analyst Office report, "The Future of Proposition 36 Funding," of November 2005, and to the 2004 Little Hoover Commission Report, "For Our Health and Safety: Joining Forces to Defeat Addiction.” Specifically, in terms of system integration and SACPA implementation, UCLA noted that assessment rates were higher in counties where assessment took place in or near the court, where offenders were allowed more days to report for assessment, where probation and assessment staff were colocated, where offenders were allowed to “walk in,” and where offenders could complete assessment procedures in one visit. More widespread adoption of these practices would likely increase showrates at assessment centers. In addition, UCLA found that the likelihood of SACPA offenders reporting for assessment and subsequently for treatment were higher in counties using one or more drug court procedures to handle SACPA offenders. Counties should examine the practicality of more specialized courts for the processing of SACPA offenders or the incorporation of efficacious drug court elements. As one potential state resource, the Network for the Improvement of Addiction Treatment (NIATx 43 ), a partnership between the Robert Wood Johnson Foundation, the Center for Substance Abuse Treatment, and other agencies, has shown success in reducing time to 43 The NIATx website is http://www.niatx.org/. Results obtained when using NIATx can be seen at http://chess.chsra.wisc.edu/pathstorecovery/P2RPresentations/ASHR%20Presentation_102105_final.pdf 35 assessment and time to treatment admission, in reducing noshow rates, and in increasing treatment continuation rates through the implementation of a number of practices aimed at improving drug treatment processes. Many of these could also apply to the processing of SACPA offenders by the criminal justice system. Five states, Delaware, Iowa, Oklahoma, North Carolina, and Texas, currently have participatory pilot projects. 44 ADP should explore NIATx methods and assess if they can be reproduced among California’s county drug treatment systems. ADP should also examine the benefits of joining the partnership. Such actions could result in a substantial improvement in outcomes for the SACPA population. Criminal Justice At the criminal justice level, changes in the legislative text defining SACPA processing criteria or eligibility are likely to improve cost outcomes. As noted, a small percentage of SACPA eligible offenders contributed disproportionate amounts to arrest and conviction costs. Possible adjustments range from assignment to 612 months of residential treatment, to intensive supervision, to barring such offenders from SACPA, or to alternatively receiving injail or in prison drug treatment. Along this same line, the assigned probation or parole officer should work intensively with the SACPA offender to ensure that the initial admission visit occurs quickly and that facilitating resources are provided (e.g., travel assistance). To improve compliance with SACPA requirements, offenders must be held accountable, through, for example, probation officer escort to initiate their treatment, increasing court oversight (e.g., drug court monitoring), or brief jail stays (“flash” incarceration) as a possible sanction. 45 Oversight by the county lead agency to hold accountable all SACPA elements (criminal justice supervision, treatment program, and offender) should continue after admission to resolve any difficulties with attendance or other requirements of treatment or supervision. A greater utilization of drug testing results (both by probation and by community programs) and immediately applied sanctions of increasing severity are recommended for problematic or recalcitrant offenders during the period of probationlinked drug treatment. In accordance with research findings on accountability and contingency management, such measures (whether positive or negative) need to be clearly enumerated, the consequences (or benefits) specified, and sanctions (or rewards) swiftly and consistently applied for infractions (or for recovery milestones) of probation/parole or treatment conditions. 46 Drug Treatment Drug treatment resources should be adjusted to better meet the needs of high drugseverity offenders. In particular, residential treatment should be available for those with the highest severity of substance abuse, and methadone maintenance programs 47 should be provided for SACPA offenders with heroin or other opiatedependence disorders who wish to access this modality. Providers should use methods developed and validated in research to improve treatment engagement, retention, services, and outcomes, while attending to culturally relevant differences. Such strategies include decreasing delays from a request (or mandate) for treatment 44 The NIATx outcome data was collected independently of the current UCLA analysis. The benefits of flash incarceration are not yet consistently confirmed in the research literature (see Marlowe et al., below). Importantly, the offender’s perception of fair and impartial use of this sanction weighs heavily in determining success attributable to this method. 46 Marlowe, D. B., and K. C. Kirby. 1999. Effective Use of Sanctions in Drug Courts: Lessons from Behavioral Research. National Drug Court Institute Review 2 (1):1–31 47 A primary benefit of methadone maintenance is the provision of a legal opiate in the context of appropriate counseling and ancillary services. Such treatment leads to better outcomes for opiate users over any other modality. 45 36 to the actual delivery of services, use of contingency contracting and management procedures, placing clients into another program rather than letting them abscond (or discharging them) from current services, aggressive reengagement efforts via telephone if clients disappear, and post treatment telephone followup to check on progress and promote treatment reentry if relapse is detected. As previously noted, NIATx has shown success in reducing time to assessment and time to treatment admission, in reducing noshow rates, and in increasing treatment continuation rates. In addition, the Center for Substance Abuse Treatment (CSAT) Treatment Improvement Protocols (TIPs) describe empirically based techniques and provides manuals for training. 48 Strategic Planning As has been seen in the cost report findings, the major economic benefits of SACPA derived from a reduction in jail and prison time served. The major attendant costs were those for subsequent arrests and convictions and for drug treatment itself. Probation and parole costs were modest, as were healthcare costs and government income from taxes on earnings and purchases. This is not to say that greater probation and parole expenditures would not improve the performance of offenders under SACPA. For example, enhancing supervision could increase the number of referred offenders who actually report for treatment, or attention to results of drug testing and subsequent action could interrupt relapse before it gains full severity. With Proposition 63 funding available to counties as of January 1, 2006, for better support of mental health services, SACPA offenders with cooccurring mental health disorders should be placed into dualdiagnosis services whenever possible. In addition, ADP should negotiate possibilities with federal agencies and local and national foundations (e.g., California Endowment, Robert Wood Johnson) for service and research funding to develop, assess, implement, and evaluate systemic and programmatic improvements for SACPA. Since the California Legislature has provided funding to continue the SACPA program for an additional year (July 1, 2006, through June 30, 2007), UCLA suggests that some of this funding be directed to determine strategies to meet the aforementioned background goals, especially in regard to more centralized state oversight; to greater resources for improved referral procedures into drug treatment and enhanced retention services; to increasing attention to the results of drug testing to achieve better accountability and to identify any emergent relapse for aggressive intervention; and to provide proven aftercare and continuity of care services As noted above, the possibility of joint funding from both SACPA and Proposition 63 should be encouraged as counties modify their SACPA procedures in the coming years. Finally, it is likely that, unless funding is appropriated for multiple years, SACPA policies and practices in the future will undergo considerable scrutiny as the Legislature debates each year of funding as part of the state’s annual budgeting process 49 . During this recurrent process, further study of techniques and strategies to improve outcomes in all the aforementioned areas should be 48 TIP 44, Substance Abuse Treatment for Adults in the Criminal Justice System (2005); TIP 7, Screening and Assessment for Alcohol and Other Drug Abuse Among Adults in the Criminal Justice System; TIP 12, Combining Substance Abuse Treatment With Intermediate Sanctions for Adults in the Criminal Justice System; and TIP 17, Planning for Alcohol and Other Drug Abuse Treatment for Adults in the Criminal Justice System. 49 In its current form, SB 803 allows for multiple year funding of SACPA. Should this aspect of the proposed bill be retained, SACPA policies and practices should be subjected to annual review to ensure incremental improvement in the program. 37 encouraged. 50 It is likely that many empirically based methods that have achieved better results in some settings will require adaptation to the local SACPA policy and practice ecologies of the various counties. Hence, an ongoing formative evaluation process to allow a “quality improvement” cycle of enhancement selection, adaptation, application, specific evaluation, and reassessment should be undertaken in all subsequent years of SACPA funding. Importantly, an outcomes evaluation effort, conforming to the strategic planning process recommended above and similar to that mounted in SACPA’s first 5 years should be funded as well. Results obtained from this evaluation would also feed into the quality improvement cycle of SACPA refinements. 50 The Little Hoover Commission Report, "For Our Health and Safety: Joining Forces to Defeat Addiction," includes "Recommendation 3: The State should implement outcomebased quality control standards for treatment personnel, programs, and facilities and encourage continuous quality improvement." The report also suggested: "Tie provider reimbursement to outcomes: After establishing performance benchmarks and implementing [the California Outcomes Monitoring System] (CalOMS), [ADP] should [consider] reward[ing] high quality treatment providers with higher rates of reimbursement." 38 Appendix C: DifferenceinDifferences (DID) Hypothetical Example The differenceindifferences (DID) approach is a widely adopted method for policy and program evaluation in many fields, especially in cost determination. This section provides an illustration of how to interpret the DID results reported for SACPA. Briefly, assume that we are trying to estimate the effect of SACPA on some hypothetical cost. Assume also that we obtain data from an appropriate source and estimate that for the SACPA group the average cost per offender was $5,000 in the 30 months before the SACPAeligible conviction and $3,000 in the 30 months after the SACPAeligible conviction. The difference between the pre and the postperiod is $2,000. This scenario illustrates the standard prepost design. Here, the $2,000 cost reduction is attributed to the policy. But now assume the same analysis is performed on our comparison offenders. Assume that the average cost per offender was $5,000 in the 30 months before the equivalent of a SACPA eligible conviction and $4,000 in the 30 months after the SACPAeligible conviction. This analysis is shown in Figure 14. For offenders in the comparison group the difference between the pre and the postperiod is $1,000. This represents, in this case, the cost reduction (we could have hypothesized an increase) that we would have expected to observe in the absence of the law. Thus, attributing the full $2,000 reduction illustrated in Figure 14 to SACPA (the result of the standard prepost design) would be misleading. To estimate more precisely the impact of SACPA, we determine the cost difference that would have been expected had the law not been implemented. Figure 15 illustrates the DID estimate as being the difference between the $2,000 cost reduction observed among the SACPA offenders and the $1,000 cost reduction observed among the comparison offenders. Fig 14. Illustration of a Hypothetical Cost Module $10,000 Dollars per offender $9,000 $8,000 $7,000 $6,000 $5,000 $5,000 $5,000 $4,000 $4,000 $3,000 $3,000 $2,000 $1,000 $0 Comparison Pre SACPA Post Post 39 The DID estimate has a clear interpretation. The dollar value estimated shows us how much the average cost per offender differed under SACPA compared with what would have been expected if the law had not come into effect. For the hypothetical cost module in this illustration, the DID estimate leads us to conclude that costs under SACPA were on average $1,000 lower per offender than what average offender costs would have been in the absence of the law. Figure 15. Illustration of a hypothetical differenceindifferences computation $10,000 D ollars p er o ffender $9,000 $8,000 $7,000 $6,000 $5,000 $5,000 $5,000 $4,000 $4,000 $3,000 $3,000 $2,000 $1,000 $0 Comparison Pre SACPA Post Post This approach was applied to the 8 cost areas described in findings section of the main report. In that section, each figure gives the calculated pre and post values, the differences within each group, and the DID between the two groups. 40 Appendix D: Historical Context of SACPA Implementation The context in which the SACPA initiative and law developed merits consideration in understanding how implementation proceeded. The SACPA legislation was prepared, to a large extent, by advocacy groups and used the state initiative process, thereby bypassing the typical legislative process. Opposition to the initiative, especially by many within the criminal justice system, was common. Importantly, little opportunity was created for substance abuse and criminology researchers to be formally involved in framing the law and proposing empirically based strategies for its subsequent implementation. Additionally, while ADP was the agency charged with overseeing SACPA, the responsibility fell to the counties to implement it locally. 51 Accordingly, in the initiative insufficient consideration was given to case flow and efficiency and to accountability mechanisms by law enforcement (the courts, probation, parole) and by drug treatment systems at both the state and county level. This contributed to capacity and service issues that required less than optimal placement for some offenders. For example, lack of residential program slots in many counties required that high drugseverity offenders were placed in outpatient care as the only primary alternative (a placement that may have contributed to the higher arrest and conviction costs found in the SACPA followup). Statewide, more consideration should have been given to the capacity of the law enforcement and drug treatment systems to accommodate the increase in the number of offenders requiring supervision, as required by the law, and of offenders requiring drug treatment. Particularly at the county level, consideration should have been given to whether drug treatment providers were able to adhere to clinically established guidelines and best practices in the treatment of substanceinvolved offenders. 52 Prior SACPA evaluation reports made several recommendations for improving treatment outcomes (e.g., placing opiatedependent offenders in methadone maintenance programs). However, stakeholders did not optimally implement these recommendations in later SACPA years. The overall efficiency and efficacy of SACPA could have been enhanced with more attention to resource allocation and strategic planning. In forthcoming years of SACPA, continuing system improvement is required to address these shortcomings. 51 However, in the survey of county stakeholders, most respondents felt that the flexibility given to counties was a beneficial element of the implementation, a pragmatic observation given the realities of their local treatment and supervision ecologies. 52 ADP issued All County Lead Agency (ACLA) letters to promote best practices, but had little authority to require adoption. 41