United States Government Accountability Office

GAO

Report to the Chairman, Subcommittee
on Crime, Terrorism, and Homeland
Security, Committee on the Judiciary,
House of Representatives

December 2009

JUVENILE JUSTICE
DOJ Is Enhancing
Information on
Effective Programs,
but Could Better
Assess the Utility of
This Information

GAO-10-125

December 2009

JUVENILE JUSTICE
Accountability Integrity Reliability

Highlights
Highlights of GAO-10-125, a report to the
Chairman, Subcommittee on Crime,
Terrorism, and Homeland Security,
Committee on the Judiciary, House of
Representatives

DOJ Is Enhancing Information on Effective Programs,
but Could Better Assess the Utility of This
Information

Why GAO Did This Study

What GAO Found

State juvenile justice systems face
critical problems when it comes to
juvenile delinquency issues such as
reentry—when offenders return
home from incarceration—and
substance abuse. GAO was asked
to review juvenile reentry and
substance abuse program research
and efforts by the Department of
Justice’s (DOJ) Office of Juvenile
Justice and Delinquency
Prevention (OJJDP) to provide
information on effective programs
(i.e., whether a program achieves
its intended goal) and costbeneficial programs (i.e., whether
the benefits of programs exceeded
their costs). This report addresses
(1) expert opinion and available
research on these types of reentry
and substance abuse programs,
(2) the extent to which OJJDP
assesses its efforts to disseminate
information on effective programs,
and (3) OJJDP’s plans to
accomplish its research and
evaluation goals. GAO, among
other things, reviewed academic
literature, and OJJDP’s
dissemination efforts and research
goals. GAO also interviewed OJJDP
officials and a nonprobability
sample of 26 juvenile justice
experts selected based on their
experience with juvenile reentry
and substance abuse issues.

The majority of the juvenile justice reentry and substance abuse experts GAO
interviewed cited evidence that shows cognitive behavioral therapy—
programs that help individuals change their beliefs in order to change their
behavior—and family therapy—programs that treat juveniles by focusing on
improving communication with family members—are effective and cost
beneficial when addressing reentry and substance abuse issues. For example,
two juvenile reentry experts cited studies showing that 1 year after
participating in a cognitive behavioral therapy program, participants were less
likely to commit another offense than nonparticipants. Additionally, experts
cited a study that reported that a family therapy program provides about
$80,000 in savings per participant when accounting for savings from a decline
in crime, such as the cost the police would have incurred. Most experts
indicated that there was limited evidence on the effectiveness and cost
benefits of reentry programs, such as aftercare—programs that assist juvenile
offenders in returning to their communities during the reentry process—and
substance abuse programs, such as drug courts—specialized courts that
provide programs for substance-abusing juveniles and their families.

What GAO Recommends
GAO recommends that OJJDP
develop a cost-effective mechanism
to regularly solicit and incorporate
feedback from the juvenile justice
field on the usefulness of
information in the Model Programs
Guide. DOJ concurred with GAO’s
recommendation.
View GAO-10-125 or key components.
For more information, contact Eileen Larence
at (202) 512-8777 or larencee@gao.gov.

GAO reviewed two OJJDP efforts that provide information on effective
programs across the range of juvenile justice issues, the National Training and
Technical Assistance Center (NTTAC) and the Model Programs Guide. OJJDP
has mechanisms in place to regularly assess the utility of the information
provided by NTTAC, but does not have such a mechanism for the guide.
OJJDP ensures the utility of NTTAC’s information through evaluations in
accordance with federal guidelines that highlight the importance of
regularly soliciting feedback from users. However, OJJDP could better
ensure the utility of the information disseminated by the Model Programs
Guide by having a mechanism in place to solicit regular feedback from
members of the juvenile justice field—for example, program practitioners—
that is specifically related to the guide.
OJJDP has articulated research and evaluation goals to support its mission of
improving the juvenile justice system and is developing plans to assist in
meeting these goals. OJJDP is required under the Juvenile Justice and
Delinquency Prevention Act, as amended, to publish an annual program plan
that describes planned activities under accounts authorized for research and
evaluation activities, among other things. Additionally, the Office of
Management and Budget (OMB) recommended that OJJDP develop a
comprehensive evaluation plan for juvenile justice programs. While OJJDP
has not published an annual program plan since 2002, in December of 2009, it
issued a proposed plan for public comment and aims to publish the final
program plan once public comments are incorporated. Additionally, although
the office has considered developing a comprehensive evaluation plan to
address OMB recommendations, it had not previously done so because of a
lack of resources. However, OJJDP is committed to developing a
comprehensive evaluation plan once the program plan is finalized.
United States Government Accountability Office

Contents

Letter

Appendix I

Appendix II

Appendix III

Appendix IV

Appendix V

1
Scope and Methodology
Background
Experts Cite Evidence from Available Research Indicating That
Cognitive Behavioral Therapy and Family Therapy Programs Are
Effective and Cost Beneficial When Addressing Reentry and
Substance Abuse Issues
OJJDP Has Mechanisms in Place to Ensure Training and Technical
Assistance Meet Users’ Needs, but Regular Feedback on the
Model Programs Guide Would Help OJJDP Better Assess
Information Utility
Finalizing a Program Plan and Developing a Comprehensive
Evaluation Plan Would Help OJJDP Achieve Its Research and
Evaluation Goals and Use Its Limited Resources Effectively
Conclusions
Recommendation for Executive Action
Agency Comments and Our Evaluation

30
36
37
37

Juvenile Justice Reentry and Substance Abuse
Experts We Interviewed

39

Juvenile Justice Experts’ Views on Factors That Can
Help Programs Achieve Intended Outcomes

41

OJJDP’s Enacted Appropriations for Fiscal Years
2007 through 2009

45

Juvenile Reentry and Substance Abuse Programs
OJJDP Funded through Discretionary Grants

47

Experts’ Opinions of Reentry Programs That Lack
Conclusive Evidence of Effectiveness

49

Page i

4
8

12

24

GAO-10-125 Juvenile Justice Programs

Appendix VI

Experts’ Opinions of Substance Abuse Programs That
Lack Conclusive Evidence of Effectiveness
54

Appendix VII

Additional OJJDP Efforts to Disseminate Information
about Effective Juvenile Justice Programs
57

Appendix VIII

Comments from the Department of Justice

58

Appendix IX

GAO Contact and Staff Acknowledgments

60

Table 1: Types of Juvenile Reentry Programs
Table 2: Types of Juvenile Substance Abuse Programs
Table 3: Examples of Net Benefits of Program Interventions within
Reentry and Substance Abuse Program Types We
Reviewed from Four Studies Cited by Experts
Table 4: OJJDP Funds Authorized and Used for Research and
Evaluation, Fiscal Year 2008
Table 5: Juvenile Justice Experts We Interviewed about Juvenile
Reentry or Substance Abuse Programs
Table 6: Juvenile Justice Enacted Appropriations, Fiscal Years 2007
through 2009
Table 7: OJJDP Juvenile Reentry and Substance Abuse Programs
Funded through Discretionary Grants, Fiscal Years 2007
through 2009
Table 8: Additional OJJDP Efforts to Disseminate Information on
Effective Programs

9
10

Tables

Page ii

23
34
39
45

47
57

GAO-10-125 Juvenile Justice Programs

This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.

Page iii

GAO-10-125 Juvenile Justice Programs

United States Government Accountability Office
Washington, DC 20548

December 17, 2009
The Honorable Robert C. Scott
Chairman
Subcommittee on Crime, Terrorism,
and Homeland Security
Committee on the Judiciary
House of Representatives
Dear Mr. Chairman:
Although the juvenile arrest rate is near its lowest point in two decades,
critical problems affecting states’ juvenile justice systems remain, such as
recidivism—the act of committing new offenses after having been arrested
or convicted of a crime—and substance abuse—a pattern of use of illegal,
prescription, or nonprescription drugs leading to significant impairment in
functioning. Reentry, the return of juvenile offenders1 from residential
facilities back into their communities, aims to reduce recidivism by using
programs that promote the effective reintegration of juvenile offenders
and assist them in acquiring the life skills needed to succeed and become
law-abiding citizens.2 According to the Department of Justice (DOJ),
approximately 100,000 juvenile offenders are held in residential facilities
in the United States on a given day, and most of these juveniles will likely
leave these facilities and return home to their communities each year.3
Further, DOJ has estimated that 62 percent of juveniles in residential
facilities self-reported having had at least one prior commitment in the

1
A juvenile offender is a youth under the age of 18 who is too young to be tried as an adult,
and who has been found to have committed an offense that would be criminal if committed
by an adult.
2

Residential facilities house juveniles who are awaiting adjudication (i.e., resolution of a
case by a judge) or have been adjudicated for an offense, or juveniles who are removed
from their homes because of nondelinquency issues, such as being victims of child abuse.
3
This number is based on data from the Census of Juveniles in Residential Placement (the
Census), which takes 1-day population counts of juveniles placed in residential facilities
across states. The number of juveniles placed in residential facilities ranged from about
108,000 in 1999 to around 93,000 in 2006, based on the most recently conducted Census.
The Census does not capture data on juveniles held in adult prisons or jails nor does it
include facilities exclusively intended for drug or mental health treatment, even though
such facilities may house juvenile offenders.

Page 1

GAO-10-125 Juvenile Justice Programs

juvenile justice system.4 Additionally, substance abuse plays a significant
role in juvenile crime.
In addition, in a 2008 annual survey of members conducted by the Federal
Advisory Council on Juvenile Justice,5 reentry of offenders into
communities and schools (28 states) and substance abuse (21 states) were
two of the top three programmatic issues most frequently reported as
affecting these states’ juvenile justice systems.6 Specifically, concerns
were noted about the minimal number of programs available to help
juveniles reenter communities, the large number of inexperienced
practitioners operating programs, and the limited substance abuse
treatment options. Related to research and policy, the most common issue
state officials cited was the need to identify effective practices for juvenile
programs (23 states) followed by the need to enhance the states’
capacities to conduct juvenile research and collect data about juvenile
programs (20 states). Respondents also cited the need for federal agencies
and state governments to collaborate on conducting research and
evaluations to provide a better explanation of (1) practices that have been
evaluated and have been shown to be effective, (2) the need for more

4

This estimate, the most recent available as reported in the Office of Juvenile Justice and
Delinquency Prevention’s (OJJDP) 2006 National Report, is derived from the 2003 Survey of
Youth in Residential Placement, which consists of anonymous interviews with a large,
nationally representative sample of juvenile offenders placed, or committed, in residential
facilities. The universe for this survey is the population of juvenile offenders ages 10 to 20
in all facilities included in the 2003 Census of Juveniles in Residential Placement. There is
no national recidivism rate for juveniles since juvenile justice systems vary across states.
Department of Justice, Office of Juvenile Justice and Delinquency Prevention, Juvenile
Offenders and Victims: 2006 National Report (Washington, D.C., 2006).
5
The Federal Advisory Committee on Juvenile Justice is an advisory body established by
the Juvenile Justice and Delinquency Prevention Act of 1974, as amended, to, among other
things, advise the President and Congress on state perspectives regarding the operation of
OJJDP and on federal legislation pertaining to juvenile justice and delinquency prevention.
42 U.S.C. § 5633(f). The committee consists of appointed representatives from advisory
groups in each of the 50 states, the District of Columbia, and the 5 U.S. territories. As part
of its mandated responsibilities, the committee submits two annual reports—one to the
President and Congress and the second to the OJJDP Administrator. These reports are
informed by data gathered through an annual request for information from the individual
state and territory advisory groups. Specifically, 47 of 55 states and territories (responses
were only requested from 4 of the 5 U.S. territories) responded to the 2008 request.
6

The most frequently cited issue was mental health assessment and treatment (38 states).
Respondents cited concerns about the lack of resources and funding available for mental
health programs, the minimal number of treatment services available, the minimal number
of trained staff operating programs, and the limited mental health treatment options.

Page 2

GAO-10-125 Juvenile Justice Programs

training from federal agencies on how to conduct evaluations, and
(3) increased funding to establish evaluation procedures (23 states).
The Office of Juvenile Justice and Delinquency Prevention (OJJDP) is
DOJ’s office charged with providing national leadership, coordination, and
resources to help prevent and respond to juvenile delinquency and
victimization. OJJDP supports states and communities in their efforts to
develop and implement effective programs to prevent delinquency and
intervene after a juvenile has offended, among other things. For example,
from fiscal years 2007 through 2009, Congress provided OJJDP almost
$1.1 billion to use for grants to states, localities, and organizations for a
variety of juvenile justice programs. In support of its mission, the office
also disseminates information to states and communities related to a
number of juvenile justice issues, including effective reentry and
substance abuse programs, and funds research and evaluations of juvenile
justice programs.
States and localities develop and implement programs that specifically
target facilitating reentry and reducing substance abuse; therefore, it is
important that federal, state, and local agencies that provide grants for
programs, as well as practitioners operating programs, have information
about which ones are effective and cost beneficial. Having such
information will better position these agencies to help ensure that federal,
state, and local funds are well spent. In general, effectiveness is
determined through program evaluations, which are systematic studies
conducted to assess how well a program is working—that is, whether a
program produced its intended results or effects. Additionally, cost-benefit
analyses can help determine if the dollar value of a program’s success—
such as a reduction in recidivism—exceeds the cost of the program. To
help ensure the effective use of grant funds for juvenile reentry and
juvenile substance abuse programs, you asked us to review the available
research as well as OJJDP’s efforts to provide information about effective
programs to the juvenile justice field, that is, program practitioners and
communities. In addition, you asked us to provide information on OJJDP’s
research and evaluation planning. Specifically, this report addresses the
following questions:
•

•

What do expert opinion and available research indicate about the types of
juvenile reentry programs and juvenile substance abuse programs that are
effective or cost beneficial?
To what extent does OJJDP have efforts under way to disseminate
information about effective juvenile justice programs and assess the utility
of the information it is providing through these efforts?

Page 3

GAO-10-125 Juvenile Justice Programs

•

Scope and
Methodology

To what extent does OJJDP have plans in place to accomplish its juvenile
justice research and evaluation goals?
To determine what experts and the available research indicate about the
types of reentry programs and substance abuse programs that are effective
or cost beneficial for juvenile offenders, we reviewed relevant literature,
studies, and federal resources for juvenile justice programs, and
interviewed federal officials and 26 juvenile justice experts.7 Specifically,
to identify the types of programs to review, we conducted a literature
search for studies and articles, including evaluations of juvenile reentry
and juvenile substance abuse programs in the United States that were
published from May 30, 1999, through May 30, 2009. We chose this time
frame, the past 10 years, because it provided us with an overview of the
available research, including unpublished and ongoing studies, which
assesses the effectiveness of reentry and substance abuse programs. We
also consulted with OJJDP officials who coordinate research on juvenile
justice programs and Department of Health and Human Services officials
who oversee substance abuse and adolescent programs to obtain their
recommendations for repositories—online databases that contain
information on effective programs—and research studies and relevant
Web sites for identifying types of reentry and substance abuse programs.8
Using these recommendations, information from relevant literature, and
categories of program types used by OJJDP’s Model Programs Guide, we
identified five types of juvenile justice programs that are used to address

7

GAO defines an expert as a person who is recognized by others who work in the same
subject matter area as having knowledge that is greater in scope or depth than that of most
people working in that area. The expert’s knowledge can come from education, experience,
or both.
8

Repositories and Web sites we identified include OJJDP’s Model Programs Guide,
Substance Abuse and Mental Health Service Administration’s National Registry of
Evidence-based Programs and Practices, University of Colorado’s Blueprints for Violence
Prevention, and the National Institute of Justice.

Page 4

GAO-10-125 Juvenile Justice Programs

reentry issues and five types of programs that are used to address
substance abuse issues for juvenile offenders.9
Specifically related to substance abuse, we focused on substance abuse
programs that involved relapse prevention treatment for juvenile offenders
with substance abuse histories.10 After consulting with experts and
reviewing the literature, we excluded juvenile alcohol abuse programs and
substance abuse programs for the general juvenile population as well as
at-risk juveniles who are prone to, but have not yet developed, substance
abuse problems. For instance, we excluded after school or recreation
programs, conflict resolution programs, and school or classroom
programs. While all of these programs may have a substance abuse
component, this component is not designed to address juvenile offenders’
actual substance abuse problems.
After identifying the types of programs to be reviewed, we looked at online
databases, academic research, and professional organizations to select
subject matter experts—researchers and practitioners—to obtain their
views on the types of programs that have been shown to be effective or
cost beneficial and the basis they used for making such determinations.
We specifically identified researchers who focus on juvenile reentry issues
or substance abuse issues and practitioners who operate programs that
address these issues. We chose 26 experts to interview as a result of this
process. Specifically, we selected 13 individuals with expertise related to
juvenile reentry programs, 7 individuals with expertise related to juvenile
substance abuse programs, and 6 individuals with both juvenile reentry
and substance abuse program expertise. We selected these experts based
on several criteria, including their employment histories related to juvenile
reentry and substance abuse programs and the number of years they spent
studying, evaluating, or managing programs addressing juvenile reentry or
substance abuse issues. We evaluated their experience by reviewing the

9
We used program types outlined in the Model Programs Guide because it describes a range
of potential program types that juvenile offenders may encounter in the juvenile justice
system. Specifically, the Model Programs Guide is a database that in part is designed to
provide information on individual intervention programs within these program types. The
Model Programs Guide categorizes programs by the type of program being administered
(e.g., vocational training or family therapy), the stage at which the program is administered
(e.g., prevention, during incarceration, or after return to the community), and the issue area
the program addresses (e.g., gangs or substance abuse).
10

According to the Substance Abuse and Mental Health Services Administration, relapse
prevention is a variety of interventions designed to teach individuals who are trying to
maintain behavior changes how to anticipate and cope with the problem of relapse.

Page 5

GAO-10-125 Juvenile Justice Programs

studies the researchers had completed and determining the experience the
practitioners had managing the types of juvenile reentry and substance
abuse programs selected for our review. See appendix I for the list of
experts we interviewed.
We asked these experts to provide their views about the effectiveness of
program types (e.g., drug courts), rather than about the effectiveness of
individual intervention programs (e.g., a specific drug court program that
was implemented in one county).11 Because the Model Programs Guide,
like other online repositories, contains information about the effectiveness
of individual intervention programs, it does not provide information about
the effectiveness of program types. As a result, we were interested in
obtaining the experts’ consolidated views of the effectiveness of program
types. We also asked the experts to identify other program types—in
addition to those that we explicitly asked about—that they considered to
be effective or cost beneficial, but no additional program types were
mentioned. In addition, we asked the experts to identify factors that in
their view could help programs to achieve intended outcomes, such as
reducing participants’ recidivism, which are summarized in appendix II.
While the results of these interviews cannot be generalized to reflect the
views of all experts knowledgeable about juvenile reentry or substance
abuse programs, we believe the interviews provided us with a good
overview of the available research and valuable information about what
program types are considered to be effective by subject matter experts. In
addition, while we did not assess the methodological rigor of studies and
evaluations in our review, we corroborated expert testimony by reviewing
and summarizing the studies or evaluations that experts cited as the basis
for their opinions. We also provided the experts with a summary of their
opinions to review in order to ensure that we correctly captured their
views.
To identify the extent to which OJJDP has efforts under way to
disseminate information about effective juvenile justice programs and
assess the extent to which OJJDP ensures the utility of the information
provided, we reviewed documentation, such as OJJDP’s annual reports
outlining information dissemination efforts, OJJDP publications, and a
contract related to disseminating training information on effective

11

For the purposes of this review, we define an intervention program as a specific activity,
treatment, therapy, or service funded at the local, state, or federal level that is intended to
address the reasons behind a juvenile’s delinquent behavior and prevent the juvenile from
committing increasingly serious offenses.

Page 6

GAO-10-125 Juvenile Justice Programs

programs. We interviewed knowledgeable OJJDP officials, such as the
Training Coordinator and communications policy personnel, about
OJJDP’s efforts to disseminate information about effective programs. We
selected two of OJJDP’s efforts through which it disseminates information
about effective programs—the Model Programs Guide and the National
Training and Technical Assistance Center (NTTAC), which provides
training and support to the juvenile justice field in identifying and
implementing effective programs—because they provide information
about effective programs across the range of issue areas in which OJJDP is
involved, including reentry and substance abuse programs. We then
compared these efforts to guidance articulated by the Office of Justice
Programs (OJP), which oversees OJJDP, and in prior GAO reports that
stresses the importance of assessing whether the information
disseminated is meeting the needs of its users.12 We also interviewed
representatives from the two organizations that manage these two
information dissemination efforts. Additionally, we asked the 26 juvenile
reentry and substance abuse experts we interviewed about their views
regarding OJJDP’s information dissemination efforts and their opinions
about the effectiveness of these efforts. Although their views cannot be
generalized to the entire juvenile justice field, we believe that the experts
provided us with a good overview of the utility of the information
disseminated by OJJDP. We did not contact recipients of the information
OJJDP disseminates for their views on the usefulness of the information
provided because of the large volume of recipients and the resulting cost
that would be incurred to obtain this input.
To assess the extent to which OJJDP has plans in place for its research
and evaluation efforts, we reviewed relevant laws related to the office’s
role in supporting research and evaluations of juvenile justice programs.
We also reviewed relevant DOJ and OJJDP documentation, such as annual
reports and strategic plans that contain information on OJJDP’s research
and evaluation goals and plans. We interviewed cognizant OJJDP officials
about the office’s planning efforts related to research and evaluation. We
also reviewed criteria found in standard practices for program
management and our prior products that highlight the importance of

12

Department of Justice, Office of Justice Programs, OJP Information Quality Guidelines
(Washington, D.C.: 2007). GAO, U.S. Public Diplomacy: Actions Needed to Improve
Strategic Use and Coordination of Research, GAO-07-904 (Washington, D.C.: July 18,
2007), and Transportation Research: Opportunities for Improving the Oversight of DOT’s
Research Programs and User Satisfaction with Transportation Statistics, GAO-06-917
(Washington, D.C.: Aug. 15, 2006).

Page 7

GAO-10-125 Juvenile Justice Programs

developing plans to meet goals and help ensure that resources are used
effectively, and then compared these criteria to OJJDP’s stated plans.13
Additionally, we analyzed OJJDP funding and staff data for fiscal years
2005 through 2009 to better understand the resources the office has had
available to support its evaluation activities. We chose these years because
they provide the most recent overview of OJJDP’s research and evaluation
funding.
We conducted this performance audit from April 2009 through December
2009 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to
obtain sufficient, appropriate evidence to provide a reasonable basis for
our findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our findings
and conclusions based on our audit objectives.

Background

Within states’ juvenile justice systems, reentry aims to promote the
effective reintegration of juvenile offenders back into communities upon
release from residential facilities. Reentry is a process that incorporates a
variety of programs to assist juvenile offenders in the transition from
residential facilities to communities. In addition, reentry is intended to
assist juvenile offenders in acquiring the life skills needed to succeed in
the community and become law-abiding citizens and can incorporate the
use of education, mental health, drug rehabilitation, or vocational training
programs. While reentry begins after a juvenile is released back into the
community, to help ensure a seamless transition, a reentry process begins
after sentencing, then continues through incarceration, and into the period
of release back into the community. According to OJJDP, juvenile justice
practitioners and researchers believe that providing supervision and
services to juvenile offenders returning to the community will reduce the
high rate of recidivism among these juveniles. Several types of programs
address juvenile reentry issues, as described in table 1.

13

Program management standards we reviewed are reflected in the Project Management
Institute’s The Standard for Program Management © (2006). Also, see GAO, South Florida
Ecosystem Restoration: A Strategic Plan and a Process to Resolve Conflicts Are Needed to
Keep the Effort on Track, GAO/T-RCED-99-170 (Washington, D.C.: Apr. 29, 1999); South
Florida Ecosystem Restoration: Substantial Progress Made in Developing a Strategic
Plan, but Actions Still Needed, GAO-01-361 (Washington, D.C.: Mar. 27, 2001); Great Lakes:
A Coordinated Strategic Plan and Monitoring System Are Needed to Achieve Restoration
Goals, GAO-03-999T (Washington, D.C.: July 16, 2003); and GAO-06-917, 7, 11.

Page 8

GAO-10-125 Juvenile Justice Programs

Table 1: Types of Juvenile Reentry Programs

Aftercare—programs that focus on the delivery of services and supervision that start
while a juvenile is incarcerated to assist juvenile offenders in returning to their
communities during the reentry process. These programs prepare juvenile offenders to
return to the community by establishing collaboration with the community and marshaling
its resources to help ensure that juvenile offenders receive services that address their
individual needs, such as treatment for a substance abuse problem.
Cognitive behavioral therapy—programs that are designed to identify and provide
juveniles the skills to change thoughts and behavior that contribute to their problems.
Reentry courts—specialized courts that manage the return of juvenile offenders to the
community after they are released from residential facilities. A court manages reentry by
using its authority to direct resources to support the offender’s return to the community
and promote positive behavior, among other things. For example, the court may oversee
a juvenile’s release into the community by assigning a judge to meet with the juvenile
once a month.
Vocational/job training—programs that provide juveniles with employment
opportunities and are intended to improve juveniles’ social and educational functioning
by, for example, increasing earnings, raising self-esteem, and instilling a positive work
ethic.
Wraparound/case management—a strategy designed to keep delinquent youth at
home and out of institutions whenever possible. This strategy involves making an array
of individualized services and support networks available to juveniles, rather than
requiring them to enroll in structured treatment programs, which may not address
individual needs.
Source: OJJDP’s Model Programs Guide.

Substance abuse includes, but is not limited to, the use or abuse of illegal
drugs (e.g., heroin), prescription drugs, and nonprescription drugs (e.g.,
over-the-counter medications available without a prescription, such as
cough suppressant). Treatment of substance abuse may occur in a variety
of different settings, such as in clinics on an outpatient basis or at a
hospital. Treatment can also occur in short- and long-term residential
facilities that range from secure environments where juveniles’ activities
are physically restricted, to group homes, which are nonsecure settings
where juveniles live and receive services in a homelike environment.
According to the Center for Substance Abuse Treatment, juveniles with
addictions to substances can be helped through programs that specifically
target the factors associated with substance abuse—such as a family

Page 9

GAO-10-125 Juvenile Justice Programs

history of such abuse.14 For example, substance abuse intervention
programs, such as cognitive behavioral therapy and family therapy, aim to
change a juvenile’s behavior by focusing on improving a juvenile’s
response to situations that contributed to prior substance abuse.
Substance abuse intervention programs can be provided to juvenile
offenders throughout the juvenile justice system: after sentencing, during
incarceration, and after release back into the community. Whether
treatment occurs while a juvenile is incarcerated or after the juvenile is
released into the community, according to OJJDP, effective intervention
programs can help addicted juveniles to overcome their substance abuse,
lead crime-free lives, and become productive citizens. Table 2 describes
types of programs—in addition to cognitive behavioral therapy and
wraparound/case management, which are discussed in table 1—that
address juvenile substance abuse issues.
Table 2: Types of Juvenile Substance Abuse Programs

Drug courts—specialized courts established within and supervised by juvenile courts to
provide intervention programs, such as cognitive behavioral therapy or family therapy, for
substance-abusing juveniles and their families.
Family therapy—programs using trained therapists to treat juvenile offenders with
substance abuse problems by including families of juveniles in the treatment, focusing on
improving communication and interactions among family members, as well as improving
overall relationships between juveniles and their families.
Mentoring—programs that establish a relationship between two or more people over a
prolonged period of time, where an older, more experienced individual provides support
and guidance to a juvenile. The goal of mentoring is for the juvenile to develop positive
adult contact, thereby reducing risk factors, such as exposure to juveniles who use
substances, while increasing positive factors, such as encouragement for abstaining
from substance use.
Source: OJJDP’s Model Programs Guide.

The Juvenile Justice and Delinquency Prevention Act (JJDPA) established
OJJDP in 1974.15 As the only federal office charged exclusively with
preventing and responding to juvenile delinquency and with helping states

14

The mission of the Center for Substance Abuse Treatment of the Substance Abuse and
Mental Health Services Administration, within the Department of Health and Human
Services, is to promote the quality and availability of community-based substance abuse
treatment services for individuals and families who need them. The center works with
states and community-based groups to improve and expand existing substance abuse
treatment services.

15

42 U.S.C. § 5611.

Page 10

GAO-10-125 Juvenile Justice Programs

improve their juvenile justice systems, OJJDP supports its mission through
a variety of activities. For example, OJJDP administers a wide variety of
grants to states, territories, localities, and public and private organizations
through formula, block, and discretionary grant programs; provides
training and technical assistance; produces and distributes publications
and other products containing information about juvenile justice topics;
and funds research and evaluation efforts.16 In fiscal year 2009, the total
appropriation for juvenile justice programs was about $374 million. See
appendix III for more detailed information on OJJDP’s enacted
appropriations for fiscal years 2007 through 2009.
OJJDP, through its various grant programs, has provided funding to states
and organizations to support juvenile reentry and substance abuse
programs, although the JJDPA does not specifically require OJJDP to fund
them. States generally have the authority to determine how formula and
block grants are allocated and may use these funds to support a range of
program areas, including programs specifically for reentry or substance
abuse. For example, from fiscal years 2007 through 2008, OJJDP reported
that states used approximately $7.1 million in applicable formula and
block grant funds for programs that target reentry and $19 million in
formula and block grant funds for programs that target substance abuse,
representing approximately 1.8 percent and 4.5 percent, respectively, of
such funding for those years. Additionally, from fiscal years 2007 through
2009, OJJDP awarded a total of approximately $33 million in discretionary
grants through four juvenile reentry grant programs and three substance
abuse grant programs. Specifically, in the area of reentry, OJJDP awarded
a total of $25.4 million to 38 grantees under 4 programs, and in the area of
substance abuse, OJJDP awarded a total of $7.6 million to 15 grantees
under 3 programs. See appendix IV for more detail on funding for these
reentry and substance abuse programs.

16

In general, formula and block grant awards provide funds to states in accordance with
statutory requirements. OJJDP allocates some formula and block grants to states on the
basis of states’ juvenile populations, while others may be awarded at a fixed level to all
states. Discretionary grants provide funds to states, units of local government, and
organizations to administer programs. OJJDP awards discretionary grants to recipients
through an application process or based on congressional direction. The term “state”
means any state of the United States, the District of Columbia, the Commonwealth of
Puerto Rico, the Virgin Islands, Guam, American Samoa, and the Commonwealth of the
Northern Mariana Islands. 42 U.S.C. § 5603.

Page 11

GAO-10-125 Juvenile Justice Programs

Experts Cite Evidence
from Available
Research Indicating
That Cognitive
Behavioral Therapy
and Family Therapy
Programs Are
Effective and Cost
Beneficial When
Addressing Reentry
and Substance Abuse
Issues

Of the five reentry program types we reviewed, reentry experts reported
that there is evidence from available research that cognitive behavioral
therapy reduces recidivism. While experts cited a lack of evidence
demonstrating that wraparound/case management, aftercare, and
vocational/job training were effective in achieving results, such as a
reduction in recidivism, they generally provided positive views on the
potential results of these three types of programs, based on their own
experience or knowledge of them.17 Similarly, of the five substance abuse
program types we reviewed, juvenile substance abuse experts reported
that there is evidence from available research that cognitive behavioral
therapy along with family therapy are effective at reducing recidivism and
show successful results at reducing substance abuse. However, expert
opinions regarding other substance abuse program types, such as drug
courts, mentoring, and wraparound/case management, were mixed, with
experts stating that these program types could be effective, they were
ineffective, or there was not enough evidence to determine effectiveness.
Furthermore, both reentry and substance abuse experts cited studies
indicating that cognitive behavioral therapy and family therapy programs
are cost beneficial; however, the experts cited limited evidence for
determining the costs and benefits of the other programs we reviewed.

17
In addition to the four program types discussed, we also reviewed reentry courts.
However, because only 2 of 19 experts provided comments related to reentry courts, we
included their comments in app. V.

Page 12

GAO-10-125 Juvenile Justice Programs

Reentry Experts Cited
Evidence That Cognitive
Behavioral Therapy Is
Effective at Reducing
Recidivism but Concluded
That Other Programs Lack
the Evidence Necessary to
Determine Their
Effectiveness
Experts Cited Evidence from
Available Research
Demonstrating That Cognitive
Behavioral Therapy Is Effective
at Reducing Recidivism

Eleven of the 12 experts we interviewed who provided comments based
on their knowledge and experience with cognitive behavioral therapy
stated that evidence from available research shows that these programs
can be effective at reducing recidivism.18 Cognitive behavioral therapy
intervention programs are designed to identify and provide juveniles with
the skills to change thoughts and behaviors that contribute to their
problems. The underlying principle of these programs is that thoughts
affect emotions, which then influence behaviors. These intervention
programs combine two kinds of psychotherapy—cognitive therapy19 and
behavioral therapy.20 The strategies of cognitive behavioral therapy have
been used to, among other things, prevent the start of a problem
behavior—such as violence and criminal activity—or stop the problem
behavior from continuing. A juvenile offender can receive this type of
intervention program after sentencing, throughout incarceration, or after
returning to the community. For example, a cognitive behavioral therapy
intervention program may provide individual and family services to treat a

18

Twelve out of 19 reentry experts provided specific comments on cognitive behavioral
therapy as a program type. Not all of the reentry experts provided comments on each
program type as their comments were based on their particular area(s) of expertise.

19

Cognitive therapy concentrates on thoughts, assumptions, and beliefs and encourages the
recognition and change of thoughts that contribute to problem behaviors, such as violence,
criminal activity, substance use, or school-based problem behaviors. For instance when a
student has trouble completing a math problem and automatically thinks, “I’m stupid, I’m
not a good student, I can’t do math,” the student is encouraged to replace the negative
thoughts with more realistic thoughts, such as “this problem is difficult, I’ll ask for help.”

20

Behavioral therapy concentrates on specific actions and environments that either change
or maintain behaviors. For instance, when people try to stop smoking they are often
encouraged to change their daily habits—for example, instead of having their daily coffee
upon waking, which may trigger the urge to have a cigarette, they are encouraged to take a
morning walk.

Page 13

GAO-10-125 Juvenile Justice Programs

juvenile offender who has mental health and substance abuse issues. The
treatment can occur during the juvenile’s transition from incarceration
back into the community and help the juvenile lower the risk of
recidivism, connect the family with appropriate community support, assist
the juvenile in abstaining from drugs, and improve the mental health of the
juvenile.
Based on their assessment of the available research, these 11 experts
stated that cognitive behavioral therapy programs have been shown to be
effective. Experts identified two meta-analyses21 of cognitive behavioral
therapy programs that demonstrated effectiveness.22 One such study
concluded that effective cognitive behavioral therapy programs are
characterized by the low proportion of juveniles who dropped out of the
program, as well as the close monitoring of the quality of the treatment
and adequate training for the providers. In addition, this same study also
found that 12 months after treatment, the likelihood of a juvenile who
received cognitive behavioral therapy not recidivating was about one and
a half times greater than for a juvenile who did not receive the therapy.
This study also reported that the effects of cognitive behavioral therapy
were greater for offenders who had a higher risk of recidivism than those
with a lower risk. Specifically, the best results, in terms of recidivism
reductions, occurred when high-risk offenders received more intensive
treatment that targeted criminal thinking patterns. A second study also
reported that among therapeutic interventions, such as skill building,
cognitive behavioral therapy was most effective at reducing recidivism.23
The 12th expert stated that the particular cognitive behavioral therapy
intervention program he was using—aggression replacement training®—

21

A meta-analysis is a study that is a systematic synthesis of quantitative research results. In
other words, a meta-analysis of a substance abuse or reentry intervention program involves
conducting a literature search for all studies that have been conducted on that program,
evaluating the methodological quality of those studies, and then systematically combining
the findings of the studies using statistical procedures to calculate the overall effect the
program has on the various outcomes, such as recidivism and substance abuse.

22

(1) N.A. Landenberger and M. W. Lipsey, “The Positive Effects of Cognitive-Behavioral
Programs for Offenders: A Meta-Analysis of Factors Associated with Effective Treatment,”
Journal of Experimental Criminology, vol. 1 (2005). (2) M.W. Lipsey, “The Primary
Factors that Characterize Effective Interventions with Juvenile Offenders: A Meta-Analytic
Overview,” Victims and Offenders, vol. 4 (2009).

23

Skill-building programs assist juveniles in developing skills to control their behavior or
enhance their ability to participate in society in a positive manner.

Page 14

GAO-10-125 Juvenile Justice Programs

had not been evaluated at his particular program site, so he could not
draw conclusions as to its effectiveness.24

Reentry Experts Indicated
Positive Views of
Wraparound/Case
Management, Aftercare, and
Vocational/Job Training
Programs, but Stated That
These Programs Lacked
Evidence to Demonstrate
Effectiveness

Despite having generally positive views on the results of wraparound/case
management, aftercare, and vocational/job training programs based on
their experience or knowledge of these programs, reentry experts
reported a lack of evaluations that show conclusive evidence about the
effectiveness of these programs. Specifically, of the nine experts who
provided comments on wraparound/case management programs, eight
offered positive opinions about these programs. For example, two of these
experts commented that wraparound/case management can be successful
at reducing recidivism, depending on the quality and availability of
services provided to juveniles. However, two of these eight experts also
stated that there was a lack of evaluations demonstrating the effectiveness
of wraparound/case management programs. One of these experts pointed
us to a study on a specific wraparound/case management intervention
program, Wraparound Milwaukee, that showed potentially promising
results related to a reduction in recidivism rates for juvenile offenders.25
However, another expert cautioned that initial evaluations of
wraparound/case management programs did not conclusively demonstrate
the effectiveness of wraparound/case management programs.26 Finally, the
ninth expert stated that in her experience, wraparound/case management
interventions are not effective because, for example, juveniles are placed
into these interventions based on the availability of program staff and
resources rather than program services being tailored to the individual
needs of each juvenile.

24

Aggression replacement training® is an intervention program designed to alter the
behavior of chronically aggressive juveniles, such as those who have continual problems
managing their anger. The goal of the program is to improve skill competence, anger
control, and moral reasoning.
25

Bruce Kamradt, Stephen Gilbertson, and Nancy Lynn, “Wraparound Milwaukee: Program
Description and Evaluation,” Outcomes for Children and Youth with Emotional and
Behavioral Disorders and Their Families, ed. Michael H. Epstein et al. (Austin: ProEd,
2005), 307.

26

Wraparound Milwaukee is an intervention program for juveniles with serious emotional,
behavioral, and mental health needs and for their families. This intervention program
attempts to meet the mental health, substance abuse, social service, and other supportive
needs of juveniles in the Milwaukee community by identifying the specific personal,
community, and professional services each family needs to care for a juvenile with special
needs.

Page 15

GAO-10-125 Juvenile Justice Programs

In addition, 7 of the 15 experts who commented about aftercare programs
opined that aftercare interventions are important reentry programs, in
part, because they link a juvenile with the community and provide regular
contact with a caseworker. However, 6 other experts said there was
inconclusive evidence to determine whether these programs can be
effective in achieving results. Three of these experts based their opinions
on an evaluation of the Intensive Aftercare Program27 that showed
inconclusive results about program effectiveness.28 Specifically, the study
found no evidence that the program had its intended impact of reducing
recidivism among juveniles who were released back into the community
under supervision in the three states that piloted the program. However,
the evaluation did find that the three states that implemented the Intensive
Aftercare Program model successfully incorporated most of its core
features, which prepared juveniles to transition back into the community.
For instance, these states created new Intensive Aftercare Programs—
specific treatment programs that among other things, prepared juveniles
for increased responsibility in the community, facilitated interaction with
the community, and worked with the juveniles’ schools and families. The
state programs had a large percentage of juveniles involved in various
treatment services. Despite the inconclusive results of the study, 1 expert
credited the aftercare program model with addressing the issue of
juveniles interacting with multiple probation officers throughout the entire
reentry process because aftercare programs, in general, assign one
probation officer to a juvenile as a consistent point of contact. The
remaining 2 of 15 experts opined that aftercare intervention programs had
not been shown to be effective at achieving desired results because, for
example, the treatment a juvenile receives depends on the services
available in the community.
With respect to vocational/job training programs, 10 of the 11 reentry
experts who commented on these programs expressed positive opinions
about the programs’ potential outcomes but noted that there had been
limited research conducted to demonstrate their effectiveness.
Specifically, experts noted that vocational/job training programs could be

27

The Intensive Aftercare Program was a program initiative funded by OJJDP from 1987
until 2000 that was intended to reduce recidivism among juveniles who are released back
into the community under supervision.

28

Department of Justice, Office of Juvenile Justice and Delinquency Prevention,
Implementation and Outcome Evaluation of the Intensive Aftercare Program
(Washington, D.C.: March 2005).

Page 16

GAO-10-125 Juvenile Justice Programs

beneficial if they were applied to older juveniles and if they led to those
juveniles getting jobs. The remaining expert said there is little evidence to
demonstrate the effectiveness of these intervention programs. For a more
detailed description of reentry experts’ opinions about these program
types, see appendix V.

Substance Abuse Experts
Cited Evidence That
Cognitive Behavioral
Therapy and Family
Therapy Are Effective at
Reducing Recidivism and
Can Help to Reduce
Substance Abuse, but Said
That Evidence of
Effectiveness Was Lacking
for the Other Programs
Experts Cited Evidence from
Available Research Showing
Cognitive Behavioral and
Family Therapies Effectively
Reduce Recidivism and
Demonstrate Success at
Reducing Substance Abuse

All of the 13 substance abuse experts we interviewed stated that based on
available research, cognitive behavioral therapy effectively reduces
recidivism and has demonstrated success at reducing substance abuse.
Experts cited six studies to support their opinions, two of which were the
same studies cited by reentry experts that demonstrate that cognitive
behavioral therapy is effective at reducing recidivism.29 Two of the
substance abuse experts noted that few studies have been conducted to
determine whether an intervention program is effective at specifically
reducing substance abuse. However, 3 experts also noted that within the
last decade, newly emerging research has shown promising results with

29

(1) Steve Aos, Marna Miller, and Elizabeth Drake, Evidence-Based Public Policy Options
to Reduce Future Prison Construction, Criminal Justice Costs, and Crime Rates
(Olympia, Wash.: Washington State Institute for Public Policy, October 2006). (2) M.
Dennis, S. Godley, G. Diamond, F. Tims, T. Babor, J. Donaldson, H. Liddle, J. Titus, Y.
Kaminer, C. Webb, N. Hamilton, and R. Funk, “The Cannabis Youth Treatment Study: Main
findings from two randomized trials,” Journal of Substance Abuse Treatment, vol. 27
(2004).
(3) Landenberger and Lipsey, “Positive Effects of Cognitive-Behavioral Programs,” 451.
(4) Lipsey, “Primary Factors,” 124. (5) H. B. Waldron and C. W. Turner, “Evidence-Based
Psychosocial Treatments for Adolescent Substance Abuse,” Journal of Clinical Child &
Adolescent Psychology, vol. 37, no. 1 (2008). (6) H. B. Waldron and Y. Kaminer, “On the
Learning Curve: The Emerging Evidence Supporting Cognitive-Behavioral Therapies for
Adolescent Substance Abuse,” Addiction, vol. 99 (2004).

Page 17

GAO-10-125 Juvenile Justice Programs

respect to cognitive behavioral therapy program types reducing substance
abuse. For example, these experts pointed us to three studies that report
that juveniles who participated in these programs showed reductions in
marijuana use.30
Twelve of the 13 substance abuse experts we interviewed who provided
comments based on their knowledge and experience stated that family
therapy programs are effective at reducing recidivism or decreasing
substance use.31 Family therapy uses trained therapists to treat juvenile
offenders with substance abuse problems by including families of
juveniles in the treatment and focusing on improving communication and
interactions among family members and improving overall relationships
between juveniles and their families. This type of therapy focuses on the
family as it is the primary and sometimes only source for emotional
support, moral guidance, and self-esteem for juveniles. Family habits, such
as failing to set clear expectations for children’s behavior, poor monitoring
and supervision, and severe and inconsistent discipline can often lead to
juveniles engaging in delinquency and substance abuse, according to
OJJDP. For example, family drug use often results in adolescent drug use.
Based on their assessment of the available research, these 12 experts
provided positive opinions about the effectiveness of family therapy, and 7

30

(1) Waldron and Kaminer, “On the Learning Curve: The Emerging Evidence Supporting
Cognitive-Behavioral Therapies for Adolescent Substance Abuse,” 93. (2) Waldron and
Turner, “Evidence-Based Psychosocial Treatments for Adolescent Substance Abuse,” 238.
(3) Dennis et al., “The Cannabis Youth Treatment Study: Main findings from two
randomized trials,” 197.

31

Not all 13 of the substance abuse experts provided comments on each program type as
their comments were based on their particular area(s) of expertise.

Page 18

GAO-10-125 Juvenile Justice Programs

of these experts cited 9 studies that support their opinions.32 These studies
demonstrated, for example, that multisystemic therapy—a family therapy
intervention program that helps parents identify strengths and develop
natural support systems (e.g., extended family, neighbors, friends, and
church members)—is an effective intervention program for reducing
recidivism and substance use because, for example, juveniles who
participated in multisystemic therapy programs engaged in significantly
less criminal activity than did nonparticipants. Specifically, multisystemic
therapy participants had fewer average convictions per year for violent
crimes than those juveniles who did not participate in the program.
Additionally, analyses of drug tests demonstrated significantly higher rates
of drug abstinence for program participants than for nonparticipants. One
study also showed that participants in functional family therapy, another
family therapy intervention program, had 50 percent reductions in
substance use as compared to juveniles who did not participate in the
program.33

32

(1) Aos et al., Benefits and Costs. (2) Dennis et al., “The Cannabis Youth Treatment
Study: Main findings from two randomized trials,” 197. (3) P. Greenwood, “Prevention and
Intervention Programs for Juvenile Offenders,” The Future of Children, vol. 18, no. 2
(2008). (4) A. Sheidow and S. Henggeler, “Multisystemic Therapy with Substance Using
Adolescents: A Synthesis of the Research,” Crossing Frontiers (Brighton: Pavilion
Publishing, 2008). (5) S. Schoenwald, D. Ward, S. Henggeler, S. Pickrel, and H. Patel,
“Multisystemic Therapy Treatment of Substance Abusing or Dependent Adolescent
Offenders: Costs of Reducing Incarceration, Inpatient, and Residential Placement,” Journal
of Child and Family Studies, vol. 5, no. 4 (1996). (6) L. Stambaugh, S. Mustillo, B. Burns,
R. Stephens, B. Baxter, D. Edwards, and M. DeKraai, “Outcomes From Wraparound and
Multisystemic Therapy in a Center for Mental Health Services System-of-Care
Demonstration Site,” Journal of Emotional and Behavioral Disorders, vol. 15, no. 3
(2007). (7) S. Henggeler, W. Clingempeel, M. Brondino, and S. Pickrel, “Four-Year Followup of Multisystemic Therapy with Substance-Abusing and Substance-Dependent Juvenile
Offenders,” Journal of the American Academy of Child and Adolescent Psychiatry, vol.
41, no. 7 (2002). (8) M. Lipsey and F. Cullen, “The Effectiveness of Correctional
Rehabilitation: A Review of Systematic Reviews,” The Annual Review of Law and Social
Science, vol. 3 (2007). (9) Waldron and Turner, “Evidence-Based Psychosocial Treatments
for Adolescent Substance Abuse,” 238.

33

Functional family therapy is a family therapy intervention program that among other
things, attempts to reduce and eliminate problem behaviors, such as substance abuse,
through improving communication skills, parenting, problem solving, and conflict
management while increasing the family’s exposure to community resources to prevent a
reoccurrence of substance use.

Page 19

GAO-10-125 Juvenile Justice Programs

Experts Had Mixed Views on
Effectiveness of Drug Courts,
Mentoring, and
Wraparound/Case Management
Types of Programs

According to the 10 experts who commented on drug courts, 5 stated that
there is a lack of evidence to determine program effectiveness, while
another expert stated that drug courts are ineffective types of programs
because they expose first-time offenders to more serious drug users. The
remaining 4 experts stated that drug courts can be effective if, for
example, they are combined with other effective intervention programs,
such as multisystemic therapy. Similarly, according to the 8 experts who
commented on mentoring, 1 stated that there are too few evaluations to
determine effectiveness, while 4 stated that mentoring programs alone are
ineffective or unsuccessful at achieving desired results and that mentoring
intervention programs are more effective at preventing at-risk juveniles
from engaging in delinquent behavior.34 However, 3 experts thought
mentoring intervention programs could be effective if the programs adhere
to certain factors that have been evaluated and shown to be effective, such
as the mentor being properly trained.35 Finally, experts also had mixed
views on the effect of wraparound/case management types of programs.
Of the 11 experts who commented on these programs, 7 experts stated
that wraparound/case management is effective or can be effective if, for
example, wraparound/case management is combined with another
intervention program that has been evaluated and has shown to be
effective, such as cognitive behavior therapy. Conversely, 4 experts either
stated that these programs are ineffective because, for example, the
intervention programs lack follow-through as there are no consequences if
a juvenile does not show up for treatment, or there is not sufficient
evidence to determine effectiveness. For a more detailed description of
substance abuse experts’ opinions about these program types, see
appendix VI.

34

At-risk juveniles are youths who, because of certain characteristics or experiences, are
statistically more likely than other youths to encounter certain problems, such as legal,
social, financial, educational, emotional, and health problems.

35

Experts we interviewed noted that several factors could help a program be effective at
achieving its goals, such as ensuring that juveniles receive programs based on their
individual needs and ensuring that staff are well trained to correctly implement a specific
program. See app. II for a summary of these most frequently cited factors.

Page 20

GAO-10-125 Juvenile Justice Programs

Reentry and Substance
Abuse Experts Cited
Available Research That
Indicates Cognitive
Behavioral and Family
Therapies Are Cost
Beneficial, but Provided
Limited Evidence of the
Costs and Benefits of the
Other Program Types

While program evaluations establish if a program is effective in producing
its intended results or effects—such as a reduction in recidivism—costbenefit analyses use program evaluation to determine if the dollar value of
a program’s benefits exceeds the costs to deliver the program. For
example, if a program evaluation shows that an intervention program
reduces the number of offenses committed by juveniles from three to one,
a cost-benefit analysis would first determine a dollar value for each of the
offenses. Then, the cost-benefit analysis would estimate whether the
savings of going from three offenses to one offense is more or less costly
than the amount of money required to deliver the intervention program, as
compared to an alternative program the juvenile would have received. The
intervention may not always be more expensive than the alternative. For
example, if the alternative is incarceration, the intervention program may
be less expensive—meaning that the intervention program can be cost
beneficial even if it does not result in a reduction of offenses. By applying
the same cost-benefit analysis techniques to evaluations of different
program types, decision makers can make comparisons among
alternatives and determine which program types offer the greatest benefits
for the least cost. The results of a cost-benefit analysis are often
represented as a net benefit, meaning total benefits minus total cost.
Of the 26 reentry and substance abuse experts we interviewed, 19
provided information related to the cost benefits of the reentry and
substance abuse program types in our review. These 19 experts identified
five cost-benefit analyses of juvenile justice programs consisting of four

Page 21

GAO-10-125 Juvenile Justice Programs

meta-analyses36 and one systematic review.37 The studies demonstrate that
various cognitive behavioral therapy and family therapy intervention
programs are cost beneficial because they are effective at reducing crime
and are expected to produce more benefits than costs compared to the
alternative. For example, in one study, the authors reviewed several
program interventions that fall into the family therapy program type, such
as multisystemic therapy and multidimensional treatment foster care.38
The authors analyzed three program evaluations of multidimensional
treatment foster care and found that this intervention can be expected to
reduce crime outcomes by 22 percent. Based on this reduction in crime,
the authors of the study predict that the intervention provides about
$80,000 worth of benefits per participant. This dollar value reflects the
savings per participant that result from a decrease in criminal activity,
including savings to crime victims, police and sheriff’s office costs, and
juvenile detention costs, among others.
The four studies cited by the experts show mixed or inconclusive results
for drug courts, vocational/job training, and mentoring program types. For
example, one study found that juvenile drug courts are cost beneficial
because they are expected to have a net benefit of $4,622 per program
participant. The other studies could not determine drug courts’ costeffectiveness because they either did not include program evaluations of

36
The four meta-analyses cited by experts were (1) Steve Aos, Marna Miller, and Elizabeth
Drake, Evidence-Based Public Policy Options to Reduce Future Prison Construction,
Criminal Justice Costs, and Crime Rates (Olympia, Wash.: Washington State Institute for
Public Policy, October 2006); (2) Aos et al., Benefits and Costs; (3) Elizabeth Drake, Steve
Aos, and Marna Miller, “Evidence-Based Public Policy Options to Reduce Crime and
Criminal Justice Costs: Implication in Washington State,” Victims and Offenders, vol. 4
(2009); and (4) Damon Jones, Brian K. Bumbarger, Mark T. Greenberg, Peter Greenwood,
and Sandee Kyler, The Economic Return on PCCD’s Investment in Research-Based
Programs: A Cost-Benefit Assessment of Delinquency Prevention in Pennsylvania
(University Park, Pa.: The Prevention Research Center for the Promotion of Human
Development, Pennsylvania State University, 2008).
37

C. McDougall, M. Cohen, R. Swaray, and A. Perry, “Benefit-Cost Analyses of Sentencing,”
Campbell Systematic Reviews (March 2008). A systematic review is a summary of the
results of available research on a given topic. The systematic review examined available
literature about sentencing of both adult and juvenile offenders to determine how many
cost-benefit analyses of these programs exist and the quality of the studies that had been
done. Because the systematic review did not examine the specific program types in this
report, we did not include the review’s assessments of the costs and benefits of sentencing
options in this report.

38

Multidimensional treatment foster care is an intervention that places juveniles with
specially trained foster parents instead of in residential facilities.

Page 22

GAO-10-125 Juvenile Justice Programs

drug court programs or they found mixed results in the program
evaluations analyzed and therefore could not determine the net benefits.
In addition, two studies found that there are too few evaluations of
vocational/job training or mentoring in juvenile justice programs to
calculate if the benefits of these program types outweigh the costs. The
remaining program types in our review—wraparound/case management,
aftercare, and reentry courts—were not analyzed in these studies. Table 3
presents a summary of these studies.
Table 3: Examples of Net Benefits of Program Interventions within Reentry and Substance Abuse Program Types We
Reviewed from Four Studies Cited by Experts
Examples of the net benefits for interventions within program types
Cognitive
behavioral
therapya

Family
therapyb

Evidence-Based Public Policy Options to
Reduce Crime and Criminal Justice Costs:
Implications in Washington State (in 2007
dollars)

$23,015

Benefits and Costs of Prevention and Early
Intervention Programs for Youth (in 2003
dollars)

Studies cited by experts

Drug courts

Vocational/job
training

$88,953

Findings are
mixed

Too few recent
evaluations

Too few
evaluations to
date

$8,805

$24,290

Not included in
study

Not included in
study

$5,073

$14,660

$77,798

$4,622

Too few recent
evaluations

Too few
evaluations to
date

Not included in
study

$79,331

Not included in
study

Not included in
study

Not included in
study

Evidence-Based Public Policy Options to
Reduce Future Prison Construction, Criminal
Justice Costs, and Crime Rates (in 2006
dollars)
The Economic Return on Pennsylvania’s
Investment in Research-Based Programs: A
Cost-Benefit Assessment of Delinquency
Prevention in Pennsylvania (in 2007 dollars)

Mentoring

Source: GAO analysis of four studies cited by experts.

Notes: Dollar values indicate the highest net benefits for interventions for which net benefits were
calculated (i.e., for which both cost and benefit data were available) and that experts categorized
within the program types analyzed in this report.
Net benefits are discounted, or adjusted, to reflect that costs are incurred when a juvenile initially
receives the intervention, but savings may not result for many years.
Net benefits estimates for (1) Evidence-Based Public Policy Options to Reduce Crime and Criminal
Justice Costs: Implications in Washington State, (2) Benefits and Costs of Prevention and Early
Intervention Programs for Youth, and (3) Evidence-Based Public Policy Options to Reduce Future
Prison Construction, Criminal Justice Costs, and Crime Rates are for expected savings per program
participant in Washington state.
The Economic Return on Pennsylvania’s Investment in Research-Based Programs: A Cost-Benefit
Assessment of Delinquency Prevention in Pennsylvania shows estimated net benefits per program
participant in Pennsylvania.

Page 23

GAO-10-125 Juvenile Justice Programs

In cases where there are sizable differences in methodologies between studies, the benefits and
costs reported cannot be directly compared. For example, there can be differences in the
methodological criteria for inclusion, the surveys of literature can cover different time periods or
differing program types, or values for benefits and costs can either come from external literature or
can be drawn from the studies themselves.
a

Net benefits are for aggression replacement training®, which is an intervention program categorized
as cognitive behavioral therapy by the experts.

b

Net benefits are for multidimensional treatment foster care, which is an intervention program
categorized as family therapy by the experts.

In addition, seven experts also commented on reentry and substance
abuse programs that were not included in the cited studies. For example,
three experts opined that wraparound/case management programs may
eventually be proven to be cost beneficial, based on preliminary research
and evaluations. For example, one expert cited an unpublished study of a
wraparound program pilot project that showed that recidivism of program
participants was low, and that program costs were approximately 60
percent of the costs of incarcerating juveniles. Additionally, although
experts did not cite cost-benefit analyses of aftercare program types, four
reentry experts stated that such programs could be cost beneficial if the
intervention program being delivered is effective because the cost of
incarceration is so high. Three experts we interviewed stressed that even
though some intervention programs that have been shown to be effective
are expensive, if they reduce recidivism, they might be cost beneficial
because of the high cost of incarcerating juveniles.

OJJDP Has
Mechanisms in Place
to Ensure Training
and Technical
Assistance Meet
Users’ Needs, but
Regular Feedback on
the Model Programs
Guide Would Help
OJJDP Better Assess
Information Utility

Consistent with the JJDPA, OJJDP has several efforts under way to
disseminate information about effective juvenile justice programs. Two of
these efforts—NTTAC and the Model Programs Guide—provide
information about effective programs for a range of juvenile justice issues,
including reentry and substance abuse issues. Consistent with federal
guidelines for ensuring the utility of information, OJJDP has established
mechanisms to ensure that the information provided through its training
and technical assistance efforts meets the needs of the juvenile justice
field. However, OJJDP could better ensure the usefulness of the
information it disseminates through the Model Programs Guide by having
a mechanism in place to solicit regular feedback specifically related to the
guide from the juvenile justice field.

Page 24

GAO-10-125 Juvenile Justice Programs

OJJDP Disseminates
Information about
Effective Juvenile Justice
Programs through Several
Efforts

According to the JJDPA, OJJDP is authorized, but is not required, to
provide information about juvenile justice issues and programs and to
provide training and technical assistance to help the juvenile justice field
implement and replicate such programs.39 In accordance with this
authority and its mission to support states and communities in their efforts
to develop and implement effective juvenile justice programs, OJJDP
disseminates information related to these programs through a range of
efforts, from those designed to meet the needs of the juvenile justice field
as a whole to those that focus on effective programs in a specific issue
area, such as gang prevention or girls’ delinquency. OJJDP distributes the
broadest range of information on juvenile justice topics through the
Juvenile Justice Clearinghouse (Clearinghouse).40 Through its services, the
Clearinghouse offers, among other things, the latest research findings and
statistics, publications on juvenile justice issues and programs,
announcements of funding opportunities, and other resources prepared by
a variety of researchers in juvenile justice. As part of its efforts, the
Clearinghouse responds to requests for information about effective
programs by directing users to OJJDP efforts that develop and disseminate
information about effective programs, such as NTTAC and the Model
Programs Guide. Thus, we focused on NTTAC and the Model Programs
Guide because they provide information about effective programs across
the range of issue areas in which OJJDP is involved, including reentry and
substance abuse programs. OJJDP also disseminates information about
effective programs in specific issue areas through various centers, such as
the National Youth Gang Center and the Underage Drinking Enforcement
Center. For a more detailed discussion of these centers and other
information dissemination efforts that focus on specific issues, see
appendix VII.

39

42 U.S.C. § 5661(e).

40

The Clearinghouse was established in 1979 to provide individuals and organizations with
access to a comprehensive collection of information and resources on juvenile justice
topics in a centralized location. The Clearinghouse is responsible for coordinating OJJDP’s
mailing list, newsletter, publications and editorial services, library and reference services,
and Web site maintenance. It also coordinates such efforts as the Juvenile Justice
Listserv—a listserv that provides information on juvenile justice and other youth service–
related publications, funding opportunities, and events; News-at-a-Glance—OJJDP’s
bimonthly newsletter; as well as OJJDP’s library services, among other things. The
Clearinghouse is a component of the National Criminal Justice Research Service, which
offers justice and substance abuse information to support research, policy, and program
development by providing access to published reports, research findings, and other
information products.

Page 25

GAO-10-125 Juvenile Justice Programs

NTTAC was established in 1995, in part to provide information about
effective juvenile justice programs—such as programs that address issues
related to reentry and substance abuse—through its training and technical
assistance efforts.41 According to OJJDP, NTTAC works to promote the use
of effective programs in the field through training and technical assistance
programs. Additionally, NTTAC develops training materials and resources,
and customizes the information included in its curricula in an effort to
best meet the needs of its training and technical assistance recipients. In
terms of its efforts specifically related to program effectiveness, NTTAC
provides training and technical assistance for members of the juvenile
justice field on how to develop and sustain effective programs, and to help
the field understand programs that are effective for various juvenile
populations, such as juveniles with mental health issues or female
offenders.
The Model Programs Guide is an online database that contains summary
information about approximately 200 juvenile justice programs, from
prevention programs to reentry programs. It is designed to help
practitioners and communities identify and implement prevention and
intervention programs that have been evaluated and have been shown to
be effective.42 Programs in the Model Programs Guide may focus on a
range of issues, including delinquency, violence, youth gang involvement,
substance abuse, or academic issues, and can include, but are not limited
to, delinquency prevention, community service, drug courts, or family
therapy. To be included in the Model Programs Guide, programs are
reviewed and rated along several dimensions, including such factors as
whether an evaluation of the program established a causal association
between the treatment and the outcome. Users can search the Model
Programs Guide to find programs that meet their specific needs. For

41

According to OJJDP, NTTAC is one of many providers that OJJDP uses for its training and
technical assistance efforts. Whereas NTTAC provides information on a wide range of
juvenile justice issues, OJJDP’s other providers deliver specialized training and technical
assistance on specific topics, such as gang prevention through the National Youth Gang
Center.

42

The Model Programs Guide is one of several online databases that provide information
about programs that have been evaluated and may have been shown to be effective. In
general, these online databases provide information about specific intervention programs
(e.g., multisystemic therapy) rather than types of programs (e.g., drug courts or mentoring).
At the time this report was issued, GAO was in the process of issuing another report that
indicated that these online databases use different criteria for determining whether a
program is effective. See GAO, Program Evaluation: a Variety of Rigorous Methods Can
Help Identify Effective Interventions, GAO-10-30 (Washington, D.C.: Nov. 23, 2009).

Page 26

GAO-10-125 Juvenile Justice Programs

example, users can look for a program that has been shown to be effective
for juveniles with substance abuse problems who are first-time offenders,
or they can search for a program that has been shown to be effective for
juveniles involved in gang activities who are reentering the community.

Evaluations and Needs
Assessment Help OJJDP
Ensure Usefulness of
Information Provided by
Training and Technical
Assistance Efforts, but
OJJDP Could Better
Ensure the Utility of the
Model Programs Guide’s
Information through
Regular Feedback

In accordance with federal guidelines from OJP and prior GAO work,
OJJDP has mechanisms in place to regularly conduct evaluations and is
currently conducting a needs assessment to ensure the usefulness of the
information provided by its training and technical assistance efforts.
However, OJJDP could better ensure the utility of the information
provided by the Model Programs Guide by establishing a mechanism to
solicit regular feedback from the juvenile justice field. We have previously
reported on the importance of regularly soliciting feedback to assess user
needs and satisfaction.43 Specifically, we have reported that without
feedback, an agency lacks valuable information from its users and is
hindered in its ability to make improvements to information products that
are relevant to users. Additionally, OJP has published Information Quality
Guidelines for its bureaus, including OJJDP, that highlight the importance
of ensuring the utility of information to be disseminated to the public by
continuously monitoring information needs, among other things.44
OJJDP has mechanisms in place to regularly assess the usefulness of the
information disseminated by NTTAC to ensure that it meets the needs of
the juvenile justice field. Specifically, OJJDP has established an evaluation
process for NTTAC that is designed to collect the data necessary to
regularly assess the outcome and impact of the training and technical
assistance NTTAC provides to improve the quality of the information it
disseminates. Officials at NTTAC explained that after every training or
technical assistance event, all participants are given an evaluation form to
complete. This form is intended to capture feedback from participants
about the quality of the event, as well as feedback about the referrals and
resources NTTAC provides. Other evaluation forms are also available on
NTTAC’s Web site so that users can provide feedback about NTTAC’s

43

GAO-07-904 and GAO-06-917.

44

OJP developed the Information Quality Guidelines in accordance with Office of
Management and Budget (OMB) guidance, which is found in Office of Management and
Budget, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and
Integrity of Information Disseminated by Federal Agencies, 67. Fed. Reg. 8,452 (Feb. 22,
2002). Utility, as defined by OMB, refers to the usefulness of the information to the
intended users.

Page 27

GAO-10-125 Juvenile Justice Programs

services, as well as feedback about the utility of the Web site. NTTAC then
follows up with a sample of these respondents for more in-depth feedback.
According to NTTAC officials, NTTAC analyzes the data collected from
these forms and then provides them to OJJDP. These officials stated that
OJJDP receives this information on at least a quarterly basis, and uses the
information to make changes to existing curricula and guide future
curriculum development, among other things. In accordance with OJP
guidelines and prior GAO work that highlights the importance of assessing
user needs, these evaluation efforts allow OJJDP to regularly monitor the
usefulness of the information it disseminates in order to develop or modify
its information products.
In addition, OJJDP is conducting a needs assessment to solicit additional
information about the utility of the information it disseminates through
NTTAC’s training and technical assistance efforts. NTTAC is administering
the needs assessment and, according to NTTAC officials, it is designed to
determine the training and technical assistance that would be most helpful
to the field. Specifically, the needs assessment is soliciting feedback from
members of the juvenile justice field about OJJDP’s existing efforts. It is
also requesting information regarding issues of interest to the field, any
current training or technical assistance needs, and the specific challenges
that the juvenile justice field is facing in its work. OJJDP officials stated
that they intend to use the results of the needs assessment to influence the
development of training and technical assistance activities and curricula
and the content of national conferences and workshops.
OJJDP’s efforts to conduct evaluations and a needs assessment are
consistent with comments we received from our expert interviews. We
asked all 26 of the juvenile reentry and substance abuse experts we
interviewed to comment on OJJDP’s overall efforts to disseminate
information about effective programs to the juvenile justice field. Thirteen
experts provided responses, and while they did not comment specifically
on NTTAC or the Model Programs Guide, they commented on the utility of
the information OJJDP provides in general about effective programs. Ten
of 13 experts had negative opinions of how useful the information OJJDP
disseminates is to members of the juvenile justice field. For example, 1
expert stated that practitioners often do not have the time to read research
data disseminated by OJJDP, which prevents them from being able to
effectively use it in their work. The expert added that it would be more
useful if OJJDP disseminated information that was practical and could be
applied in the field. In addition, 2 of these 10 experts suggested that it
would be helpful for OJJDP to obtain feedback from members of the
juvenile justice field about what types of information they would find

Page 28

GAO-10-125 Juvenile Justice Programs

useful. Thus, OJJDP’s needs assessment should help to address this
concern. The remaining 3 experts who commented on OJJDP’s
information dissemination efforts had generally positive opinions, stating
that the information is useful to researchers and practitioners.
With respect to the Model Programs Guide, although OJJDP has ad hoc
mechanisms in place to solicit feedback about the information it provides,
it does not solicit this feedback on a regular basis or use feedback to help
ensure that the information disseminated by the Model Programs Guide is
useful to the field, in accordance with federal guidelines. For example, the
Model Programs Guide’s Program Director gives several presentations
about the guide each year at juvenile justice conferences. Officials who
operate the Model Programs Guide stated that following these
presentations, they request verbal feedback from participants. Officials
also stated that they regularly receive unsolicited feedback through the email address that is listed on the Model Programs Guide’s Web site, which
they respond to on a case-by-case basis. Additionally, officials said that
they collect feedback about the Model Programs Guide through an annual
e-mail survey that is sent to the program points of contact listed on the
guide to obtain updated program information. Although these efforts to
solicit feedback about the Model Programs Guide provide OJJDP with
some information from its users, according to OJJDP officials, because the
guide does not have a systematic feedback mechanism, information
received cannot be analyzed on an aggregate level in order to regularly
assess how the juvenile justice field views the utility of the information
provided by the Model Programs Guide. Further, while the annual e-mail
survey can help OJJDP confirm that the program information featured in
the Model Programs Guide is accurate, it does not provide information
about whether the guide is useful to the field as a whole since OJJDP
sends the survey’s request for comments about the Model Programs Guide
to a portion of the juvenile justice field whose programs are already
published in the guide, which means that the comments it receives about
the Model Programs Guide do not necessarily reflect the opinions of the
juvenile justice field as a whole.
OJJDP officials agreed that they had not established a systematic
mechanism to obtain feedback from the field regarding the usefulness of
the Model Programs Guide and recognized that such a mechanism would
be useful to have in place. Officials also stated that NTTAC’s needs
assessment might be used as a model to build in more consistent
mechanisms for feedback for the office’s broader efforts. Because NTTAC
uses evaluations and is taking steps to conduct a needs assessment to
monitor the information needs of the juvenile justice field, OJJDP is in a

Page 29

GAO-10-125 Juvenile Justice Programs

better position to help ensure that the information provided by NTTAC is
useful to the juvenile justice field. Recognizing that, although there is a
cost associated with gathering and analyzing feedback data, establishing a
cost-effective mechanism to regularly solicit feedback about the Model
Programs Guide should provide OJJDP with the information necessary to
assess whether the information provided by this tool is useful to the
juvenile justice field.

Finalizing a Program
Plan and Developing a
Comprehensive
Evaluation Plan
Would Help OJJDP
Achieve Its Research
and Evaluation Goals
and Use Its Limited
Resources Effectively

OJJDP has articulated research and evaluation goals to support its mission
of promoting effective programs and improving the juvenile justice system.
According to OJJDP, one of its three main goals is to promote
improvements in juvenile justice and facilitate the most effective
allocation of resources by conducting research to understand how the
juvenile justice system works in serving children and families.45 Under the
JJDPA, OJJDP is required to publish an annual program plan that
describes planned activities that are under accounts authorized for
research and evaluation activities and that demonstrate promising
initiatives, among other things.46 This plan is required to be published
annually in the Federal Register for public comment, and is to describe the
activities the Administrator intends to carry out under parts D and E, the
appropriations accounts that in general are available for research and the
development of new programs and initiatives, respectively.47 Specifically,
according to the JJDPA, the Administrator must take into account the
public comments received during the 45-day period and develop and

45
OJJDP has established three goals that it states constitute the major elements of a sound
policy for juvenile justice and delinquency prevention. In addition to its goal for research
and evaluation, OJJDP’s remaining two goals are (1) to promote delinquency prevention
and early intervention efforts and (2) to foster the use of community-based programs and
services for juvenile offenders.
46

42 U.S.C. § 5614(b)(5).

47

Under part D, OJJDP is authorized to conduct research, evaluation, and technical
assistance, among other things. 42 U.S.C. §§ 5661-62. Under part E, OJJDP is authorized to
make grants for developing, testing, and demonstrating promising new initiatives and
programs. 42 U.S.C. §§5665-66. OJJDP has awarded funds under part E pursuant to
congressional direction for a number of years. For example, the explanatory statement
accompanying the fiscal year 2008 appropriations act directed that funds appropriated for
part E programs be provided to specific grantees. Explanatory statement accompanying
the Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 121 Stat. 1844 (2007). For
fiscal year 2009, the appropriations act directed that funds appropriated under part E be
provided to specific grantees, as specified in the explanatory statement. Omnibus
Appropriations Act, 2009, Pub. L. No. 111-8, 123 Stat. 524, 581.

Page 30

GAO-10-125 Juvenile Justice Programs

publish a final plan before December 31 of each fiscal year, describing the
particular activities that the Administrator intends to carry out under parts
D and E. While OJJDP has not published an annual program plan since
2002, it issued a proposed plan in the Federal Register to solicit public
comment in December 2009.48 OJJDP aims to publish the final version once
public comments are incorporated, in accordance with the JJDPA’s
requirements.49 Although the annual program plan is required to describe
the particular activities the Administrator intends to carry out under parts
D and E of the JJDPA, the proposed program plan includes the office’s
priorities with respect to all discretionary funding, including its research
and evaluation efforts. According to the Acting Administrator, this will, in
part, provide complete transparency for all such funding. According to
OJJDP, the development and publication of the annual program plan is a
first step that will lead to a comprehensive evaluation plan as the annual
program plan outlines the agency’s overall research and evaluation goals.
Additionally, the Office of Management and Budget’s (OMB) fiscal year
2006 Program Assessment Rating Tool found that juvenile justice
programs would benefit from evaluations of their effectiveness but noted
that such evaluations are difficult and expensive to do.50 As a result, OMB
recommended that OJP develop a comprehensive evaluation plan for
juvenile justice programs to obtain better information about the programs’
impacts.51 Although OMB’s recommendation was directed at OJP, OJP and
OJJDP officials stated that because OJJDP is the office within OJP
required to conduct juvenile justice evaluations, it is that office’s
responsibility to develop this evaluation plan.
In addition to the above requirement and recommendation, federal
guidelines stipulate the importance of developing a plan to achieve agency

48

74 Fed. Reg. 62,821 (Dec. 1, 2009). OJJDP officials did not provide a reason why the office
has not published annual program plans for fiscal years 2003 through 2009.
49

See 42 U.S.C. § 5614(b)(5).

50

The Program Assessment Rating Tools aimed to assess and improve program
performance so that the federal government could achieve better results. Office of
Management and Budget, “Juvenile Justice Programs,” 2006 Performance and
Accountability Report (2006), http://www.whitehouse.gov/omb/expectmore/detail/
10003813.2006.html (accessed May 19, 2008).
51

The other two recommendations included in OMB’s review were to (1) make juvenile
justice programs’ performance results available to the public through program publications
and the Internet, and (2) include performance information in budget submissions to better
link resources requested to program performance goals.

Page 31

GAO-10-125 Juvenile Justice Programs

goals. As established in the standard practices for program management,
specific goals of an agency must be conceptualized and defined in a plan.52
Specifically, this plan is to contain a description or road map of how the
goals and objectives are to be achieved, including identifying the needed
resources and target milestones or time frames for achieving desired
results. We have also reported on the importance of planning research and
evaluation efforts, in part to ensure that goals are met and resources are
used effectively.53
OJJDP’s Research Coordinator stated that such a road map or plan for
conducting research and evaluation would help better target the agency’s
research and evaluation efforts toward achieving their goals. However,
from 2006 to 2009, OJJDP had not developed such a plan, primarily
because of resource constraints. According to this official, in lieu of having
a comprehensive evaluation plan in place to guide its research and
evaluation efforts, the office’s efforts are influenced by a number of
factors, including whether Congress directs the agency to conduct
research in a particular area or whether ideas are generated internally by
staff or externally by members of the juvenile justice field. For example,
OJJDP staff responsible for the mentoring area may generate ideas about
how available research funds could be used, for example, by evaluating a
particular type of mentoring program. In addition, the office may receive
recommendations from the Federal Advisory Council on Juvenile Justice
or feedback from others in the juvenile justice field. While these factors
have influenced OJJDP’s research and evaluation efforts, they have not
provided a framework for helping the office meet its research and
evaluation goals. Therefore, once the program plan is finalized, OJJDP
intends to develop a comprehensive evaluation plan in accordance with
OMB recommendations to provide direction and priorities for its research
and evaluation efforts. According to the Acting Administrator, OJJDP
intends to use this comprehensive evaluation plan to better align and
target available discretionary funds toward achieving its research and
evaluation goals.
In addition to having a road map to help ensure it meets its goals, it is
important for OJJDP to have a comprehensive plan that lays out how the
office will evaluate its juvenile justice programs. Such a plan would help to
ensure that its limited resources are being used effectively. This is

52

Project Management Institute, The Standard for Program Management © (2006).

53

GAO-03-999T, GAO-01-361, and GAO/T-RCED-99-170.

Page 32

GAO-10-125 Juvenile Justice Programs

important because OJJDP does not currently receive dedicated funding for
research and instead must make trade-off decisions to balance funding to
implement programs with funding to evaluate which programs are
effective. The office has not received dedicated research funding since
fiscal year 2005 when it received $10 million for its part D appropriations
account—the appropriations account specifically available for research
and evaluation efforts.54 Without part D funding, OJJDP has relied on funds
it has set aside from its other appropriation accounts to fund its research
and evaluation activities.55 Specifically, as shown in table 4, OJJDP is
authorized by the appropriations act to set aside up to 10 percent of
certain appropriations accounts for its research and evaluation efforts. In
fiscal year 2008, the last year for which set-aside funding data are
available, the appropriations act authorized OJJDP to set aside over
$23 million for research and evaluation. However, according to OJJDP, the
office set aside approximately $11 million. OJJDP officials stated that this
was, in part, because the JJDPA requires and the agency wants to ensure
that sufficient funds are available to the states for grant programs.56 In
addition, officials explained that some of OJJDP’s accounts are transferred
to other program offices, such as the Office of Community Oriented
Policing Services, so research funds are not deducted from those
accounts.57 Of the over $11 million that OJJDP did set aside, officials
reported that the office used nearly $8 million (or 70 percent) for research
and evaluation. Table 4 shows the amounts authorized to be set aside by

54

OJP’s Performance Budget shows that OJP has not requested part D funding for OJJDP
since fiscal year 2007.

55

Appropriations statutes for fiscal years 2007 through 2009 provided that OJJDP may use
not more than 10 percent of each amount appropriated for research, evaluation, and
statistics activities that benefit the programs or activities authorized, and not more than 2
percent of each appropriated amount for training and technical assistance. See, e.g.,
Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 121 Stat. 1844, 1911-12 (2007).
This provision applied to appropriation accounts under juvenile justice programs, but did
not apply to amounts appropriated for grants and projects authorized by sections 261 and
262, part E, of the JJDPA—which is the account available for developing, testing, and
demonstrating promising new initiatives and projects. See, e.g., id.
56

Under the formula grants program, funds are to be allocated among the states on the
basis of relative population of people under age 18; however, the JJDPA also sets base
amounts for awards to the states, depending on the aggregate amount appropriated for the
program each year and taking into account amounts allocated to the states for fiscal year
2000. 42 U.S.C. § 5632.

57

The Office of Community Oriented Policing Services is a component of DOJ responsible
for helping state and local law enforcement agencies prevent crime.

Page 33

GAO-10-125 Juvenile Justice Programs

the annual appropriations act, as well as the amounts actually set aside
and used by OJJDP.
Table 4: OJJDP Funds Authorized and Used for Research and Evaluation, Fiscal Year 2008

OJJDP appropriations account

Issue area funded by the accounta

Part A – Concentration of Federal Cooperation and coordination
c
between federal agencies involved in
Efforts
juvenile justice issues

Maximum funds
authorized by the
appropriations act for
research and evaluation
10 percent of amount
appropriated)b

Funds set
aside by
OJJDP

Funds used
by OJJDP

65,800

0

0

Part B – State Formula Grantsd

Grants to state and local
governments for juvenile
delinquency programs

7,426,000

5,940,800

3,170,011

Part D – Research, Evaluation,
Technical Assistance and
e
Training

Research and evaluation

0

0

0

N/A

N/A

N/A

7,000,000

907,620

907,479

320,000

320,000

124,623

Part E – Developing, Testing, and Development and demonstration of
new projects and initiatives
Demonstrating Promising New
f
Initiatives and Projects
Youth Mentoring Grants

Mentoring programs

Title V – Local Delinquency
g
Prevention Incentive Grants

Tribal programs, gang prevention,
alcohol prevention, other local
delinquency programs

Secure Our Schoolsh

Improved security at schools

1,504,000

0

0

Victims of Crime Act – Improving Training and technical assistance for
the Investigation and Prosecution professionals involved in
of Child Abuse Programs
investigating, prosecuting, and
treating issues related to child abuse

1,692,000

0

0

Juvenile Accountability Block
Grant

Juveniles already involved in the
juvenile justice system

5,170,000

4,136,000

3,626,917

Project Childsafei

Safe firearm handling and storage
practices

0

0

0

$23,177,800

$11,304,420

$7,829,030

Total

Sources: Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 121 Stat. 1844, 1911-12 (2007); the JJDPA; and GAO analysis
of OJJDP data.
a

The appropriation act allows OJJDP to set aside funds for research, evaluation, and statistics
activities designed to benefit the programs or activities authorized. Consolidated Appropriations Act,
2008, Pub. L. No. 110-161, 121 Stat. 1844, 1911-12 (2007). As such, the accounts from which
OJJDP sets aside funds for research and evaluation are tied to specific issue areas, and therefore all
of the set-asides must be used for research, evaluation, and statistics activities designed to benefit
these specific issue areas. For example, the mentoring appropriation account can only be used for
research and evaluation activities related to mentoring programs.

Page 34

GAO-10-125 Juvenile Justice Programs

b

The appropriations statute for fiscal year 2008 provides that not more than 10 percent of each
amount appropriated may be used for research, evaluation, and statistics activities that benefit the
programs or activities authorized, and not more than 2 percent for training and technical assistance.
Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 121 Stat. 1844, 1911-12 (2007). These
provisions applied to appropriation accounts for juvenile justice programs, but did not apply to
amounts appropriated for grants and projects authorized by sections 261 and 262, part E, of the
JJDPA—which is the account available for developing, testing, and demonstrating promising new
initiatives and projects. See id.

c

According to OJJDP officials, historically, OJJDP has elected not to set aside funds from part A as
the appropriation supports the Coordinating Council on Juvenile Justice and Delinquency
Prevention—a council created, in part, to coordinate all federal juvenile offender programs in
cooperation with state and local juvenile justice programs.

d

Under the formula grants program, funds are to be allocated among the states on the basis of
relative population of people under age 18; however, the JJDPA also sets base amounts for awards
to the states, depending on the aggregate amount appropriated for the program each year and taking
into account amounts allocated to the states for fiscal year 2000. 42 U.S.C. § 5632.

e

Funds for part D have not been appropriated since fiscal year 2005.

f

The provision in the appropriations act allowing the office to set aside up to 10 percent of each
amount appropriated is not applicable to grants and projects authorized by sections 261 and 262, part
E, of the JJDPA and is authorized for developing, testing, and demonstrating promising new initiatives
and projects. Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 121 Stat. 1844, 1911-12
(2007).
g

As described in app. III, the appropriation account for Title V programs includes funds that are carved
out for specific purposes. Pursuant to the appropriations act, these amounts are not available for setasides, as they are under sections 261 and 262, part E, of the JJDPA, whereas funds may be set
aside from the remaining amounts available for Title V delinquency prevention programs.
Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 121 Stat. 1844, 1911-12 (2007).

h

According to OJJDP officials, the office does not set aside funds from this account because the
entire appropriation is transferred to the Office of Community Oriented Policing Services—a
component of DOJ responsible for helping state and local law enforcement agencies prevent crime.

i

In fiscal year 2008, Project Childsafe was not appropriated funds as part of the appropriations for
juvenile justice programs.

Additionally, all of the set-asides from these four accounts must be used
for research, evaluation, and statistics activities designed to benefit the
juvenile justice issues that the accounts specify. For example, set-aside
funds from the youth mentoring grant appropriation account must be used
to research or evaluate mentoring programs. For other accounts, OJJDP
can elect to fund research and evaluation efforts in a number of different
areas. For example, under Juvenile Accountability Block Grants, OJJDP
provides funds to states and units of local government to strengthen the
juvenile justice system. The states can use these funds for 17 different
purpose areas, including establishing programs to help the successful
reentry of juvenile offenders from state and local custody in the
community or for hiring staff or developing training programs for
detention and corrections. Consequently, there are limits on the amount of
funds OJJDP can divert to research and evaluation and on its discretion
over how to use of some of these funds. In fiscal year 2008, the
appropriation act allowed OJJDP to set aside more than $23 million that
could be dedicated to research and evaluation efforts on numerous eligible

Page 35

GAO-10-125 Juvenile Justice Programs

programs. Because OJJDP has to decide how to split set aside funds
between supporting state and local program implementation and program
evaluation, in accordance with federal guidelines, a comprehensive
evaluation plan that in part identifies its funding resources could help
OJJDP make this determination.
According to OJJDP, the office has spent several years considering
developing a plan to provide a road map for how it would meet its
research and evaluation goals. However, officials stated that it has been
difficult to complete a comprehensive evaluation plan to fulfill OMB’s
Program Assessment Rating Tool recommendation because they have not
had the resources available—that is, funding and staffing—to develop the
plan. Specifically, because funds have not been appropriated for part D
since fiscal year 2005, OJJDP has not had a dedicated source of funding
that could be used to develop a comprehensive evaluation plan or to fund
the research identified by such a plan. Additionally, in 2003, OJJDP
reorganized its divisions and, as part of this, dissolved its research
division, as well as the training and information dissemination units.
According to OJJDP, the intention of the former Administrator who
implemented this reorganization was to better integrate these functions
throughout the agency. OJJDP officials stated that those staff who were
dedicated to research and evaluation work were reassigned to other
divisions. Although some of these staff retained the research projects they
had at the time, they also assumed new grant management duties. Also,
over the past 8 fiscal years, OJJDP’s overall authorized staffing level has
decreased from 95 to 76. Specifically, those staff dedicated to research and
evaluation decreased from 10 in fiscal year 2002 to 3.5 in fiscal year 2009.
According to OJJDP officials, the reduction in staff who were dedicated to
research and evaluation has strained the staffing resources that could be
used for developing a comprehensive evaluation plan.
Although OJJDP cited funding and staffing constraints, the Acting
Administrator has made developing a comprehensive evaluation plan a
priority and the office is committed to moving forward with developing
this plan. Following through with its planning efforts will help OJJDP to
meet its research and evaluation goals and better ensure that its resources
are being used effectively as stipulated by federal guidelines.

Conclusions

As the juvenile justice field—including states and local communities—
works to implement programs to lower juvenile recidivism rates and
address juvenile substance abuse, it is important that the field has
information about which programs have been shown to be effective

Page 36

GAO-10-125 Juvenile Justice Programs

through program evaluations. The importance of OJJDP’s goal to research
and evaluate programs to improve juvenile delinquency underscores the
need for a comprehensive plan to evaluate juvenile justice programs, one
that identifies resources to be committed to its research and evaluation
efforts and outlines the details of how OJJDP will accomplish its research
and evaluation goals. OJJDP efforts to publish a fiscal year 2010 program
plan in December are positive steps in developing the comprehensive
evaluation plan that officials have said they are committed to developing.
Having such a plan will provide OJJDP with a road map to help ensure that
it meets its research and evaluation goals, uses its limited resources
effectively, and contributes to identifying effective programs to help
support states and localities. With respect to OJJDP’s efforts to
disseminate information about effective programs, NTTAC’s efforts to
regularly assess the needs for the information it is disseminating through
training and technical assistance are important to helping OJJDP assess
the utility of its efforts and make appropriate improvements. We also
recognize that OJJDP’s efforts to conduct a needs assessment could help
provide important information to NTTAC that can be used in conjunction
with its evaluation efforts. Consistent with federal guidelines from OJP
and prior GAO reports, assessing the utility of the information
disseminated through OJJDP’s Model Programs Guide is also critical to
ensuring that such information meets the needs of the juvenile justice field
so the field can better implement effective programs. Having a mechanism
in place to regularly solicit feedback from the field about the usefulness of
the Model Programs Guide would better position OJJDP to assess whether
the information it is disseminating through the guide on effective programs
regularly meets the needs of its users.

Recommendation for
Executive Action

To help ensure that OJJDP’s Model Programs Guide is regularly meeting
user needs and providing the most helpful information on effective
programs, consistent with federal guidelines, we recommend the
Administrator of OJJDP develop a cost-effective mechanism for regularly
soliciting and incorporating feedback from the juvenile justice field on the
usefulness of the information provided in its Model Programs Guide.

Agency Comments
and Our Evaluation

We provided a copy of this report to the Attorney General for review and
comment. On December 3, 2009, OJP provided written comments, which
are reprinted in appendix VIII. OJP stated that it agreed with our
recommendation and intends to develop a mechanism for regularly
soliciting and incorporating feedback from the juvenile justice field on the
usefulness of the information provided in its Model Programs Guide by

Page 37

GAO-10-125 Juvenile Justice Programs

March 31, 2010. OJP also provided technical comments, which we
incorporated as appropriate.

We are sending copies of this report to the Attorney General, selected
congressional committees, and other interested parties. The report also is
available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please
contact Eileen Larence at (202) 512-8777 or larencee@gao.gov. Contact
points for our Offices of Congressional Relations and Public Affairs may
be found on the last page of this report. Key contributors to this report are
listed in appendix IX.
Sincerely yours,

Eileen R. Larence
Director, Homeland Security and Justice Issues

Page 38

GAO-10-125 Juvenile Justice Programs

Appendix I: Juvenile Justice Reentry and
Substance Abuse Experts We Interviewed

Appendix I: Juvenile Justice Reentry and
Substance Abuse Experts We Interviewed
For the purposes of our review, we selected a total of 26 experts to
interview—13 of whom had expertise related to juvenile reentry programs,
7 of whom had expertise related to juvenile substance abuse programs,
and 6 of whom had both juvenile reentry and substance abuse program
expertise. See table 5 for a list of these experts.
Table 5: Juvenile Justice Experts We Interviewed about Juvenile Reentry or
Substance Abuse Programs
Experts on reentry programs
David Altschuler, John Hopkins University
Troy Armstrong, California State University
Richard Dembo, University of South Florida
Lynn Ellsworth, Creative Strategy Group and Columbia University
Jim Heafner, McLaughlin Youth Center
Peter Jones, Temple University
Barry Krisberg, National Council on Crime and Delinquency
Kevin Minor, Eastern Kentucky University
Shelli Rossman, Urban Institute
Richard Steele, Juvenile Court Judges’ Commission, State of Pennsylvania
Mercer Sullivan, Rutgers University
Christy Visher, University of Delaware
Richard Wiebush, National Council on Crime and Delinquency
Experts on substance abuse programs
Paul Boxer, Rutgers University
Laurie Chassin, Arizona State University
Michael Dennis, Chestnut Health Systems
Scott W. Henggeler, Medical University of South Carolina
Randolph Muck, Substance Abuse and Mental Health Services Administrationa
Lawrence Murray, Columbia University
John Roman, The Urban Institute
Experts on both reentry programs and substance abuse programs
Steve Aos, Washington State Institute for Public Policy
Peter Greenwood, Association for the Advancement of Evidence-Based Practice
Nancy Jainchill, National Development and Research Institutes, Inc.
Bruce Kamradt, Milwaukee County
Doug Kopp, Washington State Juvenile Rehabilitation Administration
Mark Lipsey, Vanderbilt University
Source: GAO.

Page 39

GAO-10-125 Juvenile Justice Programs

Appendix I: Juvenile Justice Reentry and
Substance Abuse Experts We Interviewed

a

During our interview with Randolph Muck, other researchers were present, including H. Westley
Clark, Director, Substance Abuse and Mental Health Services Administration.

Page 40

GAO-10-125 Juvenile Justice Programs

Appendix II: Juvenile Justice Experts’ Views
on Factors That Can Help Programs Achieve
Intended Outcomes

Appendix II: Juvenile Justice Experts’ Views
on Factors That Can Help Programs Achieve
Intended Outcomes
Of the 26 reentry and substance abuse experts we interviewed, 22
experts—as well as research we reviewed—identified several factors that
can help programs achieve intended outcomes, that is, be effective.1 The
following factors, while not an exhaustive list of items for programs to
consider when implementing juvenile justice intervention programs, were
the most frequently cited by the experts we interviewed:
•
•
•
•

maintaining fidelity to the program;
selecting, training, and retaining qualified providers;
conducting needs-based assessments to provide individualized treatment;
and
improving juvenile program participation by engaging and motivating
juvenile and family involvement.
While incorporating these factors into reentry or substance abuse
programs does not guarantee that any particular intervention program will
be successful, existing programs that have been evaluated and found to be
effective have generally included these factors in their designs or
implementation.

Maintaining Fidelity to the
Program

According to 17 of the 22 experts, maintaining fidelity to the program as it
was intended to be implemented can help programs achieve their intended
objectives. This factor focuses on ensuring that core program services or
intervention components are delivered as they were designed, that is, with
fidelity. For example, for cognitive behavioral therapy, this would entail
that core intervention components, such as cognitive and social skills
training, were provided exactly as they were designed to each participant.2
According to one expert’s research, the degree to which an intervention
program is delivered with fidelity is closely related to its effects on
recidivism. Another expert concurred, stating that the more closely core
program services or intervention components are implemented as they
were designed, the more the intervention program will reduce recidivism
rates. For example, one expert emphasized the importance of maintaining
fidelity to the program when replicating the model within a specific

1

Four experts did not provide comments on any factors.

2

Cognitive skills include training on general thinking and decision-making skills, such as to
stop and think before acting, generate alternative solutions, evaluate consequences, and
make decisions about appropriate behavior. Social skills include training in prosocial
behaviors, interpreting social cues, and taking other persons’ feelings into account, among
others.

Page 41

GAO-10-125 Juvenile Justice Programs

Appendix II: Juvenile Justice Experts’ Views
on Factors That Can Help Programs Achieve
Intended Outcomes

community. In particular, another expert explained that some therapists
tend to substitute their own preferred treatment techniques instead of
using those prescribed by the intervention program, which can affect how
effective a program is at reducing recidivism. This is particularly true if the
intervention program being delivered is a program that has been evaluated
and found to be effective. Furthermore, one of these experts stated that
the specific model chosen has less of an effect on intended outcomes than
the manner in which it is delivered. As another expert explained it, a
weaker intervention—one that has not been evaluated and proven to be
effective—may result in decreased recidivism rates, for example, if it is
implemented as designed, while an effective intervention program that is
implemented poorly may have little or no effect on intended outcomes.

Selecting, Training, and
Retaining Qualified
Providers

According to 19 of the 22 experts, selecting, training, and retaining
qualified providers can help intervention programs achieve intended
outcomes. For example, the quality of the services that cognitive
behavioral therapy delivers depends, in part, on the provider’s ability and
whether the provider has been trained on the specific therapies and
components of the intervention program. Three of these experts noted
that if providers are not appropriately trained in the therapy or
intervention being implemented, they may not provide the program as it
was intended, or as one of them noted, may substitute their own preferred
treatment techniques for those prescribed by the intervention program. As
a result, the providers’ failure to deliver the intervention program as it was
designed reduces the ability of the program to achieve intended outcomes.
Furthermore, many intervention programs utilize providers who have
certain educational or clinical experience, such as having a background in
mental health or being a licensed practitioner for the specific therapy
being implemented. One of the 19 experts we interviewed also mentioned
the importance of gaining the support of the juvenile justice community,
as well as agencies’ program management, in the selection and training of
providers.

Conducting Needs-Based
Assessments to Provide
Individualized Treatment

According to 18 of the 22 experts, by assessing a juvenile’s specific
treatment needs, program providers can better design intervention
programs that will be targeted to a juvenile’s individual situation. For
example, 4 experts noted that this can help intervention programs achieve
intended outcomes because individualized treatment is more likely to
affect participants’ individual outcomes since it takes into account
differences such as age, gender, culture, environment, and problem
severity. One expert noted that individualized treatment ensures that

Page 42

GAO-10-125 Juvenile Justice Programs

Appendix II: Juvenile Justice Experts’ Views
on Factors That Can Help Programs Achieve
Intended Outcomes

juveniles do not receive unnecessary treatment, which in some instances
may produce harmful results. According to this expert, providing juveniles
who do not have substance abuse problems some programs, such as
cognitive behavioral therapy, may lead to harmful results because these
juveniles are exposed to others who have more serious addictions. Four
experts also noted that using needs-based assessments to develop
individualized treatment plans can be more cost beneficial than using
standard treatment plans. Specifically, as one of these experts noted, this
is because individualized treatment plans can help ensure that costly
interventions are not provided to juveniles who do not need extensive
services. In addition, 5 experts stated that conducting a risk-based
assessment is important to determining which juveniles are at higher risk
of reoffending in order to focus programming efforts on them.3 One of
these experts cited a study that shows that targeting specific treatment
needs of offenders is correlated with recidivism outcomes, that is,
providing targeted treatment needs is generally related to lower
recidivism.4

Improving Juvenile
Participation by Engaging
and Motivating Juvenile
and Family Involvement

According to 16 of the 22 experts, engaging and motivating juvenile and
family involvement can help to improve a juvenile’s program participation,
thereby helping intervention programs to achieve intended outcomes. For
example, 1 expert noted that successful programs rely on staff members to
gain the trust of juvenile offenders. These programs also recognize that
juveniles may experience program fatigue because they are participating
in numerous programs and that motivation may become an issue. In
addition, this expert noted that after being released into the community,
juveniles and their families may not be motivated to participate in
intervention programs. Additionally, research has shown that encouraging
families to participate in the juvenile’s treatment program can reduce
family risk factors for delinquency.5 Eleven experts also mentioned that
motivating juvenile offenders and their families to participate can assist
juveniles in successfully completing an intervention program. Two of

3

Risk factors for reoffending can include antisocial attitudes and peer associations, selfcontrol and self-management skills, and drug dependencies.
4

Four additional experts commented on assessing an individual juvenile’s treatment needs,
but did not expand on its significance.
5
For example, family risk factors for delinquency can include patterns of high family
conflict, having a parent who has been involved in the criminal justice system, and sibling
delinquent behavior.

Page 43

GAO-10-125 Juvenile Justice Programs

Appendix II: Juvenile Justice Experts’ Views
on Factors That Can Help Programs Achieve
Intended Outcomes

these experts noted that by involving family members in treatment, some
issues that may contribute to juvenile dropout rates, such as a history of
traumatic stress and family members who also abuse substances, can be
addressed within an intervention program.6

6

Five additional experts commented on engaging and motivating juvenile and family
involvement, but did not expand on its significance.

Page 44

GAO-10-125 Juvenile Justice Programs

Appendix III: OJJDP’s Enacted
Appropriations for Fiscal Years 2007 through
2009

Appendix III: OJJDP’s Enacted
Appropriations for Fiscal Years 2007 through
2009
To prevent and respond to juvenile delinquency and help states improve
their juvenile justice systems, the Office of Juvenile Justice and
Delinquency Prevention (OJJDP) administers a wide variety of grants to
states, territories, localities, and public and private organizations through
formula, block, and discretionary grant programs.1 The office also provides
training and technical assistance, produces and distributes publications
and other products containing information about juvenile justice topics,
and funds research and evaluation efforts. Table 6 shows funding by fiscal
year from 2007 through 2009 for the appropriation accounts for juvenile
justice programs.
Table 6: Juvenile Justice Enacted Appropriations, Fiscal Years 2007 through 2009
Dollars in thousands
Funding by fiscal year
Line item
Part A – Concentration of Federal Effortsb
Part B – State Formula Grants
Part D—Research, Evaluation, Technical
Assistance, and Training
Part E – Developing, Testing, and Demonstrating
Promising New Initiatives and Projects
Youth Mentoring Grants
Title V – Local Delinquency Prevention Incentive
c
Grants
Tribal Youth
Gang Prevention
Alcohol Prevention
Project Childsafed
e

2007

2008

2009a

703

658

0

78,978

74,260

75,000

0

0

0

104,674

93,835

82,000

9,872

70,000

80,000

64,171
(9,872)
(24,680)
(24,680)

61,100
(14,100)
(18,800)
(25,000)

62,000
(25,000)
(10,000)
(25,000)

987

0

0

Secure Our Schools

14,808

15,040

0

Victims of Crime Act—Improving Investigation and
Prosecution of Child Abuse Programf

14,808

16,920

20,000

1
OJJDP formula and block grant awards provide funds to states in accordance with
statutory requirements. OJJDP allocates some formula and block grants to states on the
basis of states’ juvenile populations, while others may be awarded at a fixed level to all
states. OJJDP discretionary grants provide funds to states, units of local government, and
organizations to administer programs. OJJDP awards discretionary grants to recipients
through an application process or based on congressional direction. The term “state”
means any state of the United States, the District of Columbia, the Commonwealth of
Puerto Rico, the Virgin Islands, Guam, American Samoa, and the Commonwealth of the
Northern Mariana Islands. 42 U.S.C. § 5603.

Page 45

GAO-10-125 Juvenile Justice Programs

Appendix III: OJJDP’s Enacted
Appropriations for Fiscal Years 2007 through
2009

Dollars in thousands
Funding by fiscal year
Line item
Juvenile Accountability Block Grant Program
Total

2007
g

2008

2009a

49,361

51,700

55,000

$338,362

$383,513

$374,000

Source: OJJDP funding data.

Note: According to OJJDP, the office’s overall budget also includes funds that are transferred from
other appropriations accounts; for example, Juvenile Drug Court programs are administered by
OJJDP, but the funding for these programs is provided by the Bureau of Justice Assistance—a
component of the Department of Justice that in part provides funding, training, and technical
assistance in support of national, state, and local efforts to prevent crime, drug abuse, and violence.
a

Not included in this table is the American Recovery and Reinvestment Act of 2009, Pub. L. No. 1115, 123 Stat. 115, 130, which provided an additional $225 million under the State and Local Law
Enforcement Assistance account that the conference report directed be used over 2 years for
competitive, peer-reviewed grants to units of state, local, and tribal governments and to national,
regional, and local nonprofit organizations to support critical nurturing and mentoring of juveniles,
among other things. H.R. Rep. No. 111-16, at 419.

b

According to the Office of Justice Program’s fiscal year 2010 congressional budget submission, the
Concentration of Federal Efforts program promotes interagency cooperation and coordination among
federal agencies with responsibilities in the area of juvenile justice, as authorized by part A of the
Juvenile Justice and Delinquency Prevention Act, as amended.

c

These three amounts are carved out from the Title V appropriations account in accordance with the
annual appropriations acts. The remaining funds are used for Title V delinquency prevention
programs for juveniles who have had contact with the juvenile justice system, as well as
nonoffenders—juveniles who have not yet had contact with the juvenile justice system but are in need
of preventive services.
d

Project Childsafe is a nationwide program to promote safe firearms handling and storage practices
through the distribution of safety education messages and free gun-locking devices.
e

The Secure Our Schools program provides discretionary grants to states, local governments, and
Indian tribes to provide improved security, such as placement and use of metal detectors, at schools
and on school grounds.

f

The Victims of Crime Act—Improving Investigation and Prosecution of Child Abuse program provides
training and technical assistance to professionals involved in investigating, prosecuting, and treating
issues related to child abuse.
g

Under the Juvenile Accountability Block Grant Program, OJJDP provides funds to states and units of
local government to strengthen the juvenile justice system. These funds can be used for 17 different
purpose areas, including establishing programs to help the successful reentry of juvenile offenders
from state and local custody in the community or for hiring or training programs for detention and
corrections personnel.

Page 46

GAO-10-125 Juvenile Justice Programs

Appendix IV: Juvenile Reentry and Substance
Abuse Programs OJJDP Funded through
Discretionary Grants

Appendix IV: Juvenile Reentry and Substance
Abuse Programs OJJDP Funded through
Discretionary Grants
From fiscal years 2007 through 2009, OJJDP allocated approximately
$33 million through discretionary grants to four juvenile reentry grant
programs and three juvenile substance abuse programs. See table 7 for a
description of these reentry and substance abuse programs and the
amounts OJJDP awarded to grantees.
Table 7: OJJDP Juvenile Reentry and Substance Abuse Programs Funded through Discretionary Grants, Fiscal Years 2007
through 2009
Grant program

Description

Reentry programs
High-Risk Youth Offender Reentry
and Family Strengthening Initiatives

The purpose of this program is to facilitate the successful transition of juvenile offenders to
their families and communities following confinement in a juvenile residential facility. In 2007,
OJJDP awarded a total of $15 million to 19 grantees to develop programs and strategies
aimed at helping juvenile offenders who pose significant public safety risks stay out of the
juvenile justice system, for example, by improving their family relationships.

Second Chance Juvenile Mentoring
Initiative

The purpose of this program is to support the successful and safe transition of juvenile
offenders from residential facilities to their communities. In 2009, OJJDP awarded a total of
$4.7 million to 11 grantees to develop, implement, and expand mentoring programs and
transitional services that specifically match juveniles with mentors during the juveniles’
confinement to facilitate successful community reintegration and reduce recidivism.

Second Chance Act Youth Offender
Reentry Initiative

The purpose of this program is to assist jurisdictions characterized by large numbers of
juvenile offenders returning to their communities after release from residential facilities and
reduce the rate of recidivism for these juvenile offenders. In 2009, OJJDP awarded a total of
nearly $3.7 million to five grantees to develop projects aimed at providing juvenile offenders
with services, including vocational and job placement services, substance abuse treatment,
family strengthening practices, and mentors who work with juvenile offenders during
confinement and after reentry into the community.

Tribal Juvenile Detention and
Reentry Green Demonstration
Program

The purpose of this program is to provide federally recognized tribes with the funds to support
program services for tribal juveniles residing in, or soon to be released from, tribal juvenile
detention centers. In 2009, OJJDP awarded a total of $2 million to three grantees to develop
programs that provide services, such as needs assessments, vocational training, substance
abuse treatment, family strengthening, and aftercare to help successfully reintegrate the
juveniles into the tribal community.

Substance abuse programs
Juvenile Drug Court/Reclaiming
a
Futures Program

The purpose of this program is to build the capacity of states, state courts, local courts, units of
local government, and Indian tribal governments to develop and establish juvenile drug courts
adopting the Reclaiming Futures model. This model embodies three essential elements for
juvenile offenders who are abusing substances: (1) designing a system to coordinate services,
(2) involving the community in creating new opportunities, and (3) improving treatment services
for drug and alcohol abuse. In 2007, OJJDP awarded nearly $1.3 million to three grantees for
programs that applied the Reclaiming Futures model to their juvenile drug courts by helping
youth meet educational goals, identifying juveniles requiring substance abuse treatment, and
effectively engaging youth in treatment by increasing the number and range of available
options.

Page 47

GAO-10-125 Juvenile Justice Programs

Appendix IV: Juvenile Reentry and Substance
Abuse Programs OJJDP Funded through
Discretionary Grants

Grant program

Description

Substance Abuse Prevention and
Intervention Programs

The purpose of these programs is to enhance juvenile justice, child protection, and
delinquency prevention by funding substance abuse programs or strategies that focus on
change at the individual, family, and community levels. In 2007, OJJDP awarded $4.7 million to
nine grantees for programs that aim to prevent or reduce juvenile delinquency associated with
substance abuse problems by offering a range of services, from community-based
interventions, such as academic assistance and job skills training, to individual treatment
services, such as counseling or therapy. These programs incorporate the use of prevention
and intervention programs that use innovative approaches or that have been evaluated and
have been shown to be effective, such as wraparound/case management or family therapy, as
well as those that target both substance-abusing juveniles and their families.

Brief Interventions and Referrals to
Treatment for Juvenile Courts and
a
Juvenile Drug Courts

The purpose of this program is to increase the provision and effectiveness of outpatient
treatment for adolescents with substance abuse disorders by funding juvenile courts and
juvenile drug courts to adopt and expand a combination of two interventions for adolescents in
outpatient treatment settings. These interventions are (1) motivational enhancement therapy,
an intervention program designed to change behavior by helping clients understand their
ambivalence and achieve lasting changes for a range of problematic behaviors, and
(2) cognitive behavioral therapy. In 2008, OJJDP awarded nearly $1.6 million to three grantees
for juvenile drug court programs that adopt and expand the use of these two therapies.
Source: GAO analysis of OJJDP grant programs and funding data.
a

According to OJJDP, funding for the Juvenile Drug Court/Reclaiming Futures Program and Brief
Interventions and Referrals to Treatment for Courts and Juvenile Drug Courts is provided by the
Bureau of Justice Assistance but administered by OJJDP. In addition, OJJDP has funded family drug
court programs and training and technical assistance efforts for drug courts.

Page 48

GAO-10-125 Juvenile Justice Programs

Appendix V: Experts’ Opinions of Reentry
Programs That Lack Conclusive Evidence of
Effectiveness

Appendix V: Experts’ Opinions of Reentry
Programs That Lack Conclusive Evidence of
Effectiveness
Experts Provided Positive
Views about
Wraparound/Case
Management Programs,
but Could Not Cite
Evidence from Available
Research Indicating That
They Are Effective at
Reducing Recidivism

Of the 19 reentry experts we interviewed, 9 had specific experience or
knowledge related to wraparound/case management and 8 experts had
positive comments about the effectiveness of this program.1 In general,
wraparound/case management interventions involve making an array of
individualized services and support networks available to juveniles, rather
than requiring them to enroll in treatment programs that may not address
individual needs. According to OJJDP, the goal of wraparound/case
management programs is to keep delinquent juveniles at home and out of
institutions whenever possible. The basic elements that constitute a
wraparound program include, among other things, (1) a collaborative,
community-based interagency team responsible for designing,
implementing, and overseeing the intervention program in a given
jurisdiction; (2) care coordinators who are responsible for helping
juveniles create customized treatment programs, among other things;
(3) juvenile and family teams consisting of family members and
community members who work together to ensure the juvenile’s needs are
met at home, at school, and in the community; and (4) a plan of care
developed and updated by all members of the juvenile and family teams
that identifies the juvenile’s strengths and weaknesses, targets specific
goals such as improved performance in school, and outlines how to
achieve them.
Of these nine experts, eight provided positive opinions of the results of
wraparound/case management intervention programs. For example, an
expert commented about how in one specific wraparound intervention
program, a single case manager is assigned to a juvenile and is responsible
for determining the services the juvenile is to receive based on his or her
specific needs, instead of enrolling the juvenile into a treatment program
that may not be as beneficial for the juvenile. Two of these eight experts
noted that there was a lack of evaluations demonstrating effectiveness of
these intervention programs but pointed us to a study2 on a specific
wraparound/case management intervention program, Wraparound
Milwaukee, that showed potentially promising results related to a

1

Not all of the reentry experts provided comments on each program type as their comments
were based on their particular area(s) of expertise.
2
Kamradt et al., “Wraparound Milwaukee: Program Description and Evaluation,” Outcomes
for Children and Youth with Emotional and Behavioral Disorders and Their Families,
307.

Page 49

GAO-10-125 Juvenile Justice Programs

Appendix V: Experts’ Opinions of Reentry
Programs That Lack Conclusive Evidence of
Effectiveness

reduction in recidivism rates for juvenile offenders.3 However, these
experts stressed that this study alone did not conclusively demonstrate the
effectiveness of wraparound/case management programs. The ninth expert
stated that in her experience, wraparound/case management interventions
are not effective because, for example, a juvenile is placed into this
intervention program based on the availability of program staff and
resources rather than program services being tailored to the individual
needs of the juvenile. Additionally, three of the nine experts cautioned that
these intervention programs are difficult to implement because of such
issues as a lack of quality services or low retention of juveniles and their
families in the intervention being provided. Specifically, one of these
experts noted that the quality of wraparound services can vary depending
on a community’s resources. In addition, another expert emphasized the
importance of obtaining buy-in from diverse service providers who may be
used to working on their own, such as within the welfare, foster care, and
public school systems.

Reentry Experts Cited
Lack of Conclusive
Evidence Based on
Available Research and
Mixed Views of the
Effectiveness of Aftercare
Programs at Reducing
Recidivism

Of 19 reentry experts we interviewed, 15 had specific experience or
knowledge related to aftercare programs and 6 cited a lack of conclusive
evidence of effectiveness of the program type. Aftercare intervention
programs are intended to prepare juvenile offenders to return to the
community during the reentry process by focusing on the delivery of
services and supervision that start while juveniles are incarcerated and
continue after they return to their communities. Specifically, aftercare
programs collaborate with the community and marshal its resources to
help ensure that juvenile offenders receive services that address their
individual needs, such as treatment for a substance abuse problem. These
intervention programs focus on changing individual behavior thereby
preventing further delinquency. For example, an aftercare program might
incorporate the use of techniques from an intervention therapy, such as
motivational enhancement therapy, to engage juvenile offenders in
treatment and increase their commitment to change.

3

Wraparound Milwaukee is an intervention program for juveniles with serious emotional,
behavioral, and mental health needs and for their families. This intervention program
attempts to meet the mental health, substance abuse, social service, and other supportive
needs of juveniles in the Milwaukee community by identifying the specific personal,
community, and professional services each family needs to care for a juvenile with special
needs.

Page 50

GAO-10-125 Juvenile Justice Programs

Appendix V: Experts’ Opinions of Reentry
Programs That Lack Conclusive Evidence of
Effectiveness

Of these 15 experts, 7 offered positive opinions regarding aftercare
intervention programs, based on their own experience or knowledge of the
intervention programs. For example, 1 expert noted that if aftercare
included intervention programs that were proven to be effective, used
assessment tools that identified the individual needs of the juvenile, and
implemented the therapies as they were designed, then the aftercare
intervention program should be effective at reducing recidivism rates.
Although these experts could not provide examples of studies that had
been conducted to show evidence of the effectiveness of the intervention
programs, all 7 of them agreed based on their own experience or
knowledge that aftercare interventions are important reentry programs, in
part, because they link the juvenile with his or her community and provide
regular contact with a caseworker. Additionally, 3 of these 7 experts stated
that aftercare could be effective depending on the intervention programs
used and if they were delivered as intended. For example, they said that if
aftercare includes intervention programs that have proven to be effective,
such as cognitive behavioral therapy, and identifies the individual needs of
the juvenile, the programs can reduce recidivism. However, 6 of the 15
reentry experts said there was inconclusive evidence to determine
whether these programs can be effective in achieving results. Three of
these experts based their opinions on an evaluation of the Intensive
Aftercare Program that showed inconclusive results about program
effectiveness. Specifically, the study4 found no evidence that the program
had its intended impact of reducing recidivism among juveniles who were
released back into the community under supervision in the three states
that piloted the program.5 However, the evaluation did find that the three
states that implemented the Intensive Aftercare Program model did
successfully incorporate most of its core features, which prepared
juveniles to transition back into the community. For instance, these states
created new Intensive Aftercare Program–specific treatment programs
that among other things, prepared juveniles for increased responsibility in

4

Wiebush et al., “Implementation and Outcome Evaluation of the Intensive Aftercare
Program.”
5

The Intensive Aftercare Program was implemented in three states: Nevada, Colorado, and
Virginia. In Nevada, evaluators found no significant or substantive differences in recidivism
between youth involved in the Intensive Aftercare Program and youth assigned to a control
group receiving traditional services—such as education and individual and group
counseling—except that youth in the Intensive Aftercare Program group were more likely
to be charged with violating parole because of increased monitoring. Evaluators could not
draw conclusions about Colorado and Virginia because of implementation issues, such as
small samples of available participants in those states.

Page 51

GAO-10-125 Juvenile Justice Programs

Appendix V: Experts’ Opinions of Reentry
Programs That Lack Conclusive Evidence of
Effectiveness

the community, facilitated interaction with the community, and worked
with the juveniles’ schools and families. The state programs had a large
percentage of juveniles involved in various treatment services. Despite the
inconclusive results of the study, one expert credited aftercare programs
with addressing the issue of juveniles having to deal with different
probation officers throughout the reentry process because, in general,
aftercare programs assign one probation officer to a juvenile as a
consistent point of contact. The evaluation also stated that in order for the
general aftercare model to be effective, it must not only provide
supervision and services after a juvenile’s release into the community, but
also focus on preparing a juvenile for release. The remaining 2 experts
opined that aftercare intervention programs had not been shown to be
effective at achieving desired results because, for example, the treatment a
juvenile receives depends on what services are actually available in the
community.

Experts Indicated
Potential Positive
Outcomes for Vocational
or Job Training Programs

Of the 19 reentry experts we interviewed, 11 had specific experience or
knowledge related to vocational or job training programs and indicated
potential positive outcomes for these programs. According to OJJDP,
providing juveniles with employment opportunities during reentry is a
common strategy used to try to reduce future criminal behavior.
Vocational or job training intervention programs are intended to improve
juveniles’ social and educational functioning by, for example, increasing
earnings, raising self-esteem, and instilling a positive work ethic. Juveniles
can participate in vocational/job training intervention programs while they
are incarcerated and after they return to the community.
Of the 11 reentry experts, 10 of them had positive comments based on
their experience or knowledge of the program type. Specifically, they said
that vocational/job training programs were potentially beneficial, in part, if
they were applied to older juveniles and if they led to those juveniles
getting jobs. The remaining expert said that there is little evidence to
demonstrate the effectiveness of these intervention programs.

Page 52

GAO-10-125 Juvenile Justice Programs

Appendix V: Experts’ Opinions of Reentry
Programs That Lack Conclusive Evidence of
Effectiveness

Experts Cited No Evidence
for Effectiveness of
Reentry Courts

Reentry courts are specialized courts that manage the return of juvenile
offenders to the community after they are released from residential
facilities.6 The court manages reentry by using its authority to direct
resources to support the offender’s return to the community and promote
positive behavior, among other things. For example, a reentry court would
oversee a juvenile’s release into the community by assigning a judge to
meet with the juvenile once a month. The judge would actively engage the
supervising authority, such as a parole officer, in assessing the juvenile’s
progress. The judge would also oversee sanctions for violations as well as
rewards, like early release from parole, for successful achievement of
goals, such as successfully completing a cognitive behavioral therapy
intervention program.
Of the 19 reentry experts we interviewed, 2 provided comments related to
reentry courts and had differing opinions on their effectiveness. One had a
negative impression of the courts, stating that the reentry courts do not
provide more to a juvenile than a probation officer would. The other
commented that he considers concepts encompassed in reentry courts,
such as intensity of supervision, to be a best practice when it comes to
reentry programs. However, neither was aware of any evaluations of these
types of courts.7

6

Residential facilities are correctional facilities that house juveniles who are awaiting
adjudication (i.e., a trial in a juvenile court) or have been adjudicated for an offense, or
juveniles who are removed from their homes because of nondelinquency issues, such as
being victims of child abuse.
7

None of the remaining 17 experts we interviewed could cite studies demonstrating the
effectiveness of reentry courts or had opinions on this program type.

Page 53

GAO-10-125 Juvenile Justice Programs

Appendix VI: Experts’ Opinions of Substance
Abuse Programs That Lack Conclusive
Evidence of Effectiveness

Appendix VI: Experts’ Opinions of Substance
Abuse Programs That Lack Conclusive
Evidence of Effectiveness
Experts Views Were
Divided on the
Effectiveness of Drug
Courts, with Half
Indicating That Additional
Evidence Is Needed to
Determine Effectiveness

Of the 13 substance abuse experts we interviewed, 10 had specific
experience or knowledge related to drug courts that resulted in mixed
views of the effectiveness of this program type.1 Juvenile drug courts are
specialized courts established within and supervised by juvenile courts to
provide intervention programs, such as cognitive behavioral therapy or
family therapy, for substance-abusing juveniles and their families. Juvenile
offenders assigned to drug courts are identified by a juvenile court as
having problems with alcohol or drugs. The drug court judge maintains
close oversight of each case through frequent—often weekly—status
hearings with the individuals involved. The judge both leads and works as
a member of a team that can comprise representatives from juvenile
justice, social services, school and vocational training programs, law
enforcement, probation, the prosecution, and the defense. Together, the
team determines how best to address the substance abuse and related
problems of the juvenile and his or her family.
Specifically, of these 10 experts, 5 experts described drug courts as having
insufficient evidence to determine program effectiveness. For example, 2
experts mentioned that while some studies show drug courts reducing
substance abuse while juveniles were under court supervision, the results
did not last after juveniles were no longer being supervised by the courts.
Another expert stated that since drug courts tend to be used for juveniles
who have their first or second contact with the juvenile justice system,
they are ineffective at achieving desired results because they expose these
first-time offenders to peers who have more serious substance abuse
addictions and therefore might influence them to continue to abuse
substances. By contrast, the remaining 4 experts stated that drug courts
can be effective at achieving desired results such as reducing substance
abuse if, for example, the juvenile is sent to a community where there are
intervention programs offered that have been evaluated and have been
shown to be effective, such as cognitive behavioral therapy or family
therapy intervention programs. One expert cited a study to support the

1

Not all of the substance abuse experts provided comments on each program type as their
comments were based on their particular area(s) of expertise.

Page 54

GAO-10-125 Juvenile Justice Programs

Appendix VI: Experts’ Opinions of Substance
Abuse Programs That Lack Conclusive
Evidence of Effectiveness

opinion that drug courts supplemented with multisystemic therapy2
resulted in a decrease in substance abuse by juvenile offenders.3

Experts Report Mixed
Views on the Effectiveness
of Mentoring Programs

Of the 13 substance abuse experts we interviewed, 8 had specific
experience or knowledge related to mentoring intervention programs that
resulted in mixed views of their effectiveness. Mentoring programs consist
of a relationship between two or more people over a prolonged period of
time, where an older, more experienced individual provides support and
guidance to a juvenile. The goal of mentoring is for the juvenile to develop
positive adult contact, thereby reducing risk factors, such as exposure to
juveniles who use substances, while increasing positive factors, such as
encouragement for abstaining from substance use. In the substance abuse
field, juveniles in need of sobriety are teamed with older sponsors to serve
as positive role models in helping them become sober.
Of these eight experts, four stated that mentoring programs are ineffective
or unsuccessful at achieving desired results, such as reducing substance
abuse, and that these intervention programs are more effective at
preventing at-risk juveniles from engaging in delinquent behavior.4 Also,
one expert stated that there have been too few evaluations conducted on
mentoring programs to make a general statement about the relative
benefits of mentoring. Conversely, three experts stated that mentoring
programs are effective or can be effective if, for example, mentors are
trained or if mentoring is combined with another intervention program
that has been evaluated and has been shown to be effective, such as
multisystemic therapy.

2

Multisystemic therapy is a type of family therapy that helps parents identify strengths and
develop natural support systems (e.g., extended family, neighbors, friends, and church
members).
3

Sheidow and Henggeler, “Multisystemic Therapy with Substance Using Adolescents: A
Synthesis of the Research.”
4
At-risk juveniles are youth who, because of certain characteristics or experiences, are
statistically more likely than other youths to encounter certain problems, such as legal,
social, financial, educational, emotional, and health problems.

Page 55

GAO-10-125 Juvenile Justice Programs

Appendix VI: Experts’ Opinions of Substance
Abuse Programs That Lack Conclusive
Evidence of Effectiveness

Experts Provided Positive
Views about the Potential
Results of
Wraparound/Case
Management, but Cited
Limited Evidence of Its
Effectiveness

Of the 13 substance abuse experts we interviewed, 11 had specific
experience or knowledge related to wraparound/case management
intervention programs that resulted in mixed views of the program type.
Of these 11 experts, 7 stated that wraparound/case management is
effective or can be effective if, for example, it is combined with another
intervention program that has been evaluated and has shown to be
effective, such as multisystemic therapy. Although these experts had
limited evidence demonstrating the effectiveness of wraparound/case
management, 2 experts cited two studies that show potentially promising
results related to a reduction in recidivism.5 For example, one study
showed that juveniles in wraparound/case management receive a number
of individualized services, such as mental health treatment for those
juveniles who struggle with emotional issues. However, this study stressed
that it is difficult to evaluate wraparound/case management in a controlled
way since treatment plans are individualized for each juvenile. The other 4
experts stated that wraparound/case management intervention programs
are ineffective because, for example, the intervention programs lack
follow-through as there are no consequences if a juvenile does not show
up for treatment, or there is not yet sufficient evidence to determine their
effectiveness.

5

(1) Stambaugh et al., “Outcomes From Wraparound and Multisystemic Therapy in a Center
for Mental Health Services System-of-Care Demonstration Site.” (2) Bruce Kamradt,
Stephen A. Gilbertson, and Nancy Lynn, “Wraparound Milwaukee Program Description and
Evaluation,” Outcomes for Children and Youth with Emotional and Behavioral Disorders
and Their Families: Programs and Evaluation Best Practices, 2nd ed. (Austin: ProEd,
2005).

Page 56

GAO-10-125 Juvenile Justice Programs

Appendix VII: Additional OJJDP Efforts to
Disseminate Information about Effective
Juvenile Justice Programs

Appendix VII: Additional OJJDP Efforts to
Disseminate Information about Effective
Juvenile Justice Programs
In addition to the National Training and Technical Assistance Center and
the Model Programs Guide, OJJDP disseminates information about
effective programs through a variety of other efforts. Specifically, the
office has developed mechanisms to disseminate information related to
effective programs in specific issue areas, such as youth gang activity,
disproportionate minority contact, and girls’ delinquency, as described in
table 8.
Table 8: Additional OJJDP Efforts to Disseminate Information on Effective Programs
Effort

Description

National Youth Gang Center

OJJDP established the National Youth Gang Center in 1995 to assist policymakers,
practitioners, and researchers in their efforts to reduce youth gang involvement and crime by
contributing information, resources, practical tools, and expertise toward the development and
implementation of effective gang prevention, intervention, and suppression strategies. For
example, the center continually identifies and reviews current gang literature and uses an
automated database to compile gang-related legislation. The center also conducts assessments
of the scope and characteristics of youth gang activity in the United States; develops and
disseminates resources for practitioners and communities; and provides training and technical
assistance in support of community-based prevention, intervention, and suppression efforts. For
example, the center annually collects and analyzes gang-related data from law enforcement
agencies across the nation.

Disproportionate Minority Contact
Reduction Database

OJJDP established the Disproportionate Minority Contact Reduction Database in 2006 to
address the issue of a disproportionate number of minorities coming into contact with the
juvenile justice system. The database is a searchable database that assists jurisdictions in
identifying effective programs that may prove useful to address disproportionate minority
contact. The database provides guidelines for the juvenile justice field to assist practitioners in
the field in choosing strategies and developing intervention plans.

Girls Study Group

OJJDP established the Girls Study Group in 2004 to further the juvenile justice field’s
understanding of female juvenile offending and to identify effective strategies for preventing and
reducing female juvenile involvement in delinquency and violence. The Girls Study Group
conducted a literature search and reviewed 61 girls’ delinquency programs to identify risk and
protective factors for girls’ delinquency and to identify effective programs or strategies for
preventing and reducing girls’ delinquency. In July 2009, we reported on the findings of the Girls
Study Group and OJJDP’s efforts related to addressing the group’s findings. Among other
things, we reported that the study group found no effective programs for preventing or reducing
girls’ delinquency, and we recommended that OJJDP develop and implement a plan for
responding to the findings of the Study Group.a

Underage Drinking Enforcement
Training Center

OJJDP established the Underage Drinking Enforcement Training Center in 1999 to build
leadership capacity and increase the effectiveness of states and local communities in their
efforts to enforce underage drinking laws, prevent underage drinking, and eliminate the
consequences associated with alcohol use by underage youth. The center works to achieve this
goal by providing a wide variety of practical training and technical assistance services, such as
electronic seminars where national experts, researchers, and representatives from OJJDP give
presentations on a specific areas of interest, for example, school substance abuse policies or
the effects of alcohol on adolescent brain development.
Source: GAO analysis of OJJDP information.
a

GAO, Juvenile Justice: Technical Assistance and Better Defined Evaluation Plans Will Help to
Improve Girls’ Delinquency Programs, GAO-09-721R (Washington, D.C.: July 24, 2009).

Page 57

GAO-10-125 Juvenile Justice Programs

Appendix VIII: Comments from the
Department of Justice

Appendix VIII: Comments from the
Department of Justice

u.s. Department of Justice
Office of Justice Programs
Office of the Assistant Attorney General

llhshillgtolJ, D.C. 20531

DEC 032009
Ms. Eileen R. Larence
Director, Homeland Security and Justice Issues
Government Accountability Office
441 G Street, NW
Washington, DC 20548
Dear Ms. Larence:
Thank you for the opportunity to comment on the draft Government Accountability
Office (GAO) report entitled "Juvenile Justice: DOJ Is Enhancing Information on
Effective Programs, but Could Better Assess the Utility of This Information"
(GAO-10-125). The Office of Justice Programs agrees with the Recommendation for
Executive Action, which is restated in bold text below and is followed by our response.
To help ensure that OJJDP's Model Programs Guide is regularly meeting user
needs and providing the most helpful information on effective programs,
consistent with federal guidelines, we recommend the Administrator of OJJDP
develop a cost-effective mechanism for regularly soliciting and incorporating
feedback from the juvenile justice field on the usefulness of the information
provided in its Model Programs Guide.
By March 31, 2010, the Office of Juvenile Justice and Delinquency Prevention
(OJJDP) will develop and implement an ongoing mechanism to solicit regular
feedback from the juvenile justice field on the usefulness of the information provided
in its Model Programs Guide. OJJDP's goal will be to implement a process which
ensures that its Model Programs Guide is responsive to the needs of users, accurately
represents the featured programs, and promotes the use of evidence-based programs
and practices in juvenile justice.

Page 58

GAO-10-125 Juvenile Justice Programs

Appendix VIII: Comments from the
Department of Justice

If you have any questions regarding this response, you or your staff may contact
Maureen A. Henneberg, Director, Office of Audit, Assessment, and Management, on
(202) 616-3282.
Sincerely,

cc:

Beth McGarry
Deputy Assistant Attorney for Operations and Management
Jeffrey Slowikowski
Acting Administrator
Office of Juvenile Justice and Delinquency Prevention
Maureen Henneberg
Director
Office of Audit, Assessment, and Management
Richard P. Theis
Audit Liaison
Department of Justice

2

Page 59

GAO-10-125 Juvenile Justice Programs

Appendix IX: GAO Contact and Staff
Acknowledgments

Appendix IX: GAO Contact and Staff
Acknowledgments
GAO Contact

Eileen R. Larence, (202) 512-8777 or larencee@gao.gov

Acknowledgments

In addition to the contact named above, Mary Catherine Hult, Assistant
Director; David Alexander; Elizabeth Blair; Ben Bolitzer; Carissa Bryant;
Katherine Davis; Sean DeBlieck; Allyson Goldstein; Rebecca Guerrero;
Jared Hermalin; Dawn Locke; Lisa Shibata; Janet Temko; and Delia Zee
made key contributions to this report.

(440813)

Page 60

GAO-10-125 Juvenile Justice Programs

GAO’s Mission

The Government Accountability Office, the audit, evaluation, and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO’s
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

Obtaining Copies of
GAO Reports and
Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO’s Web site (www.gao.gov). Each weekday afternoon, GAO
posts on its Web site newly released reports, testimony, and
correspondence. To have GAO e-mail you a list of newly posted products,
go to www.gao.gov and select “E-mail Updates.”

Order by Phone

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site,
http://www.gao.gov/ordering.htm.
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional information.

To Report Fraud,
Waste, and Abuse in
Federal Programs

Contact:

Congressional
Relations

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, DC 20548

Public Affairs

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, DC 20548

Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470

Please Print on Recycled Paper