APPENDIX G

COMPARATIVE EVALUATION GUIDELINES

For the Comparative Evaluation
of the

South Central Correctional Center RFP
and Contract Performance

November 2, 2001

Table of Contents
1:

OVERVIE\V

3

2.

METHODOLOGy

5

2.1.
PERFORMANCE COMPARISON
2.1.1.
PERFORMANCE EVALUATION MEASURES
T)'PES OF MEASURES
2.1.2.
2.1.3.
SCORING

5
5
6
6

3.

PERFORMANCE EVALUATION INFORMATION COLLECTION

7

4.

PERFORMANCE EVALUATION INFORMATION VALIDATION

8

S.

VALUE AND WEIGHTING OF PERFORMANCE MEASURES

8

6.

CONDUCTING THE I>ERFORMANCE EVALUATION

8

7.

COST COMPARISON

9

2

1.

OVERVIEW
The Public Acts of 2001, Chapter 132 amended TCA Title 41, Chapter24 relative to
the comparative evaluation process for private contracts for correctional services.
The new language charged the State of Tennessee with establishing objective
performance criteria and cost criteria for both the State and the private contractor.
These performance and cost criteria are to be used as the basis for a comparative
evaluation between the operations of the state's facilities, as set out in the Request
for Proposal, and the privately operated facility known as South Central Correctional
Center (SCCC). This comparison is to be made after the end of the second full
year of operation, but before renewing the initial contract at the end of the three-year
contract term. The performance evaluation is to be performed by the Select
Oversight Committee on Corrections. The cost comparison is to be compiled by
the Fiscal Review Committee. The contract can only be renewed if the contractor is
providing essentially the same quality of services as the state at a cost of 5% lower
than the state, or if the contractor is providing superior services (greater than 5%) at
essentially the same cost as the state.
TCA 41-24-105 requires the following:
a) Before the awarding of any contact to provide correctional services as defined in
Section 41-24-1 04(2)(F), the state shall establish objective performance criteria
and cost criteria for both the state and private contractor. The performance
criteria shall measure the quality of management, security and safety, personnel
training, inmate programs, and treatment and other topics deemed appropriate.
The performance criteria and cost criteria shall be established and incorporated
as requirements in any proposed request for proposal and any contract and shall
be used as the basis for any comparison between the state and any contractor.
b) For any contractor to provide correctional services as defined in Section 41-24102(2)(F), the initial contract term shall be for a period of three (3) years in order
to allow the contractor sufficient time to demonstrate its performance and to
provide sufficient information to allow a comparison of the performance of the
contractor to the performance of the state in providing similar services.
Provided, however, to allow sufficient time for completion and review of any
comparative evaluation, the initial term may exceed three (3) years by a period
of up to four (4) months as necessary to make the end of the initial term coincide
with the state's fiscal year. The initial contract may include an option to renew for
an additional period of two (2) years, subject to the provisions of this section.
c) After the first two (2) years of operation, but before renewing the initial contract,
the performance of the contractor shall be compared to the performance of the
state for similar services as set out in the contract. The contract may be
renewed only if the contractor is providing essentially the same quality of

3

services as the state at a cost of five percent (5%) lower than the state as set out
in the contract, or if the contractor is providing services superior in quality to
those provided by the state at essentially the same cost as set out in the
contract. For the purposes of this statute and comparison, "essentially the
same" shall mean the difference is no greater than five (5%). For the purpose of
this statute and comparison "superior" shall mean a difference greater than five
percent (5%). The methodology for determining the measurement of five (5%)
differences shall also be set out in the request for proposal and contract.
d) The quality of services provided by the contractor and by the state shall be
compared by the select oversight committee on correction, or, in the absence of
such committee, a committee designated by the speakers of the senate and
house. The committee shall determine the quality of services provided by the
contractor and the state by applying the performance criteria set out in the
request for proposal and contract pursuant to subsection (a) and provide a
numerical score for the state and the contractor. The committee shall report its
determination to the parties responsible for determining whether the contract
should be renewed.
e) The fiscal review committee, or, in the absence of such committee, any other
committee designated by the speakers of the senate and house, shall compare
the cost measures as established in this chapter and set out in the request for
proposal and the contract for service and provide a prisoner per day cost for the
state and the contractor. The committee shall report its determination to the
parties responsible for determining whether the contract should be renewed.
As a means of satisfying the statutory requirement to develop objective
performance and cost criteria, the Select Oversight Committee on Corrections
(SOCC) director, the SOCC consultant, a member of the Fiscal Review Committee,
as well as Department of Correction employees with a wide range of correctional
expertise, initiated the development of a performance evaluation process and
assessment and scoring instruments.
It was decided that Northeast Correctional Complex (NECC) and Northwest
Correctional Complex (NWCC) are the most comparable state facilities to evaluate
and compare with the South Central Correctional Center (SCCC). The two state
operated institutions have been selected for previous comparisons because of the
similarity in age of the facilities, design of the facilities, and inmate populations.
These two facilities continue to be the most comparable. Since the early institution
comparisons were made, consolidation of state institutions resulted in other
facilities being combined administratively with both NECC and NWCC. These
consolidations add some challenges, such as adjusting for differences in the
number of inmates, the comparability of inmate populations, etc.; however the
evaluation process and instruments have been developed to compensate for these
differences through the use of per capita ratios and other scoring measures.

4

As the process evolved, the methodological guidance of the Vanderbilt Institute for
Public Policy Studies was sought and received. The co-director of the Center for
Evaluation and Research Methodology, Dr. Mark Lipsey, assisted in reviewing the
evaluation instrument for consistency of methodology. He confirmed that the
approach adopted by the evaluation group was a fair way of comparing the
performance of all three prisons because the criteria for the comparison apply
equally to all three institutions and because the processes necessary to obtain the
data required to support the resulting measurements are in place.
Resulting from these meetings is the Performance Evaluation process and
instrument that is included as a part of the Pro Forma Contract of the Request for
Proposal under Appendix G. This instrument, identified as a Performance
Evaluation data collection instrument is included with this Overview as Exhibit "A".

2.

METHODOLOGY
Since renewal of the contract is dependent upon an evaluation of the quality of
services as well as the cost to provide those services, it is important that both
performance and cost measures be comparable and quantifiable. The following
methodology was developed with that in mind.

2.1. PERFORMANCE COMPARISON
2.1.1. PERFORMANCE EVALUA nON MEASURES
The comparative evaluation is framed around four (4) major categories. Each
category has a principal goal that expresses the desired result(s) for that particular
category. The categories and goals are as follows:

a) Security and Control
The goal of the security and control category is that the community, staff, and
inmates are protected from harm

b) Inmate Care
The goal of the inmate care category is that the institution provides for the basic
needs, appropriate care, and programming of the inmate population

c) Institution Safety & Physical Plant Conditions
The goal of institution safety and plant conditions is that the institution environment
meets national and state fire and safety codes, health codes and requirements and

5

provides for safe living and working conditions.
with order and efficiency

Emergencies are responded to

d) Institution Administration
The goal of institution administration is that the institution is managed in a
professional and responsible manner
The measurement of how'well the goals have been achieved will be measured in
two (2) ways: Outcome Measures and Mandated Practices. Most terms or phrases
used in the Outcome Measures and Mandated Practices are defined in Department
of Correction policies. An easy reference document with a selection of these
definitions is provided with the Performance Evaluation instrument as well.
Requests for clarification of terms should be made during the Pre-proposal
Conference or should be requested in the form of a Written Comment during the
Request for Proposal process. The State's written responses will become part of
the final contract.

2.1.2. TYPES OF MEASURES
Outcome Measures: Outcome measures measure the results of the institution's
operation. Outcome measures look at specific expected results within each major
category.
Mandated Practices: Mandated practices are those areas of institution operation,
which must be effectively accomplished in order to achieve the goal. These
mandated practices are items, which are done on a continual basis and are
regularly reviewed for compliance with policies, procedures, or other regulations.

2.1.3. SCORING
The scoring for the items in the performance evaluation instrument utilizes a
deductive method. In this method, each institution starts with the same base score
in each of the four categories of interest. Each institution may lose points if it does
not meet baseline standards, but each institution may also gain points if they are in
full compliance with the mandated items.
In the outcome measure section, points are deducted from the base score
according to the severity of the institution's deficiency. The scores in these sections
generally range from 0 to -4, with 0 assigned to the accepted range based on the
average level of the three institutions' previous performance on a particular item and
-4 assigned to extremely deficient performance on the item. On some items in this

6

section, an institution may score greater than --4 points. This range of scoring was
reserved to the most serious items, such as events of escapes, riots, hostages
being taken, homicides, and suicides.
In the mandated practices section, the institutions may be given additional points.
The scores in this section are either 0 or 2, with 2 points awarded for items if an
institution is in 100% compliance and 0 points awarded on those items if an
institution is not in 100% compliance based on the latest annual inspection. The last
category, Institution Administration, is the only exception. In this category, institutions
are awarded 3 points for 100% compliance on each of the mandated items. This
was done in an effort to standardize the percentage of total section points an
institution may gain from its excellent performance on mandated items.
At the end of each category, points from the outcome measure section will be
deducted from the base score and points from the mandated section will be added
to the score. The resulting score will count for a specified percentage of the entire
score.

Category
Security and Control
Inmate Care
Institutional Safety
Institution Administration
TOTAL:

Value Percent
35%
30%
20%
15 %
100 %

This evaluation instrument will penalize institutions for poor (lower expected)
performance but reward institutions for consistently superior performance on the
day-to-day operation of their institution. In essence, this method of scoring takes into
consideration that problematic events may occur even at the best run institutions,
and this instrument seeks to recognize this fact in its attempt to balance the
outcome measures and mandated items. A list of variables and the distribution of
scoring of items is included with this appendix as Exhibit B. The scoring formula to
be used is included as Exhibit C.

3.

PERFORMANCE EVALUATION INFORMATION COLLECTION
The source of the information needed to document performance will be from the
following primary sources:
a) The 2003-2004 Annual Inspection Report
b) Tennessee Department of Correction records
c) Contractor records

7

d) Tennessee Department of Correction and Contractor reports (weekly, monthly,
and annual)
To ensure fairness during the Fiscal Year 2003/2004 Annual Inspection process,
appropriate representatives of both the state and the contractor will participate on
the inspection teams at all three institutions, which are involved in the comparison.
The Select Oversight Committee on Corrections and the Office of the Comptroller
will send a representative to monitor the proceedings. Detailed guidelines for the
inspection process shall be drafted by the Department of Correction with assistance
from representatives from the Select Oversight Committee on Correction and the
contractor as soon as practicable after the awarding of the bid. The guidelines shall
include the make up of the membership of the inspection team, the clarification of
TDOC policy and terms used in the performance instrument, and the details of the
inspection process. The office of the State Comptroller shall review these
guidelines.

4.

PERFORMANCE EVALUATION INFORMATION VALIDATION
The information supplied by the Department of Correction and the Contractor will
primarily be verified from one of two sources:
a) Fiscal Year 2003/2004 Annual Inspection
b) TOMIS Reports
This validation method has been used in previous comparative evaluations.
Reliability on this method will be increased with the use of both state and contractor
representatives on the inspection teams that review all three institutions.

5.

VALUE AND WEIGHTING OF PERFORMANCE MEASURES
Each of the four (4) Major Categories was given the aforementioned proportional
value based upon the relative importance of that category in comparison with the
others. These relative percentage values were arrived at through several
discussions involving the SOCC director and consultant, select Department of
Correction staff, and a representative for the Vanderbilt institute for Public Policy
Studies.

6.

CONDUCTING THE PERFORMANCE EVALUATION
TCA 41-24-1 05(d) states that the quality of services provided by the contractor and
the state are to be compared by the Select Oversight Committee on Corrections.

8

The SOCC will use the performance criteria established in the Request for Proposal
and included in the Contract. The Performance Evaluation instrument and Scoring
instrument will be used for this purpose. The state and the contractor will supply any
information deemed necessary by the Select Oversight Committee on Corrections
to complete the performance portion of the comparative evaluation. The state and
the contractor will each appoint one person who will serve as the contact person for
the SOCC. Those appointed individuals will be responsible for gathering whatever
information is required and transmitting it to the SOCC.
The performance evaluation will result in a numerical score for each institution in
compliance with TCA 41-24-1 05(d). An average score for the two state institutions
will be calculated to arrive at a single state numerical performance score. The state
score will then be compared to the contractor score to determine if the statutory
mandates established in TCA 41-24-105(c) have been met.
An opportunity will be given to both the State and the Contractor to present facts or
evidence to clarify any misunderstandings and correct any perceived
misrepresentations of facts and data. The final draft report will be given to both the
State and the Contractor for comment before it is delivered to the Select Oversight
Committee on Corrections. If either the State or the Contractor choose, they can
submit a written response to the final report which will be included when the report is
submitted to the SOCC.

7.

COST COMPARISON
TCA 41-24-105(e) requires the Fiscal Review Committee to provide a prisoner per
day cost for the State and the Contractor based upon cost measures set out in the
Request for Proposal and the Contract. Those costs are to be used in the
evaluation to determine if the Contractor is providing essentially the same quality of
services as the state at a cost of five percent (5%) lower than the state, or if the
Contractor is providing services superior in quality to those provided by the State at
essentially the same cost pursuant to TCA 41-24-105(c).
The financial information to be compared will be for the Fiscal Year 2003/2004.
This is necessary in order to comply with the statutory mandates which state that the
comparison is to be made after the second year of the Contract, but before any
renewal can occur at the end of the third year. The FY04 information would be the
most current information available at the time of the comparison evaluation and will
match the review period that will be used for the performance evaluation.
The institutions included in the Contract for comparison with the South Central
Correctional Center (SCCC) are Northeast Correctional Complex (NECC) and
Northwest Correctional Complex (NWCC). The two state operated institutions have
been selected for previous comparisons because of the similarity in age of the

9

facilities, design of the facilities, and inmate populations. These two facilities
continue to be the most comparable. Since the early institution comparisons were
made, consolidation of state institutions resulted in other facilities being combined
administratively with both NECC and NWCC. In order to restore a reasonable level
of comparability, adjustments will be made for staff and operating costs.
The cost comparison will review the full costs of the Contractor with the full costs of
the State's comparable facilities (NECC and NWCC). The costs attributable to the
Contractor will include any costs of monitoring the Contract incurred by the State,
which would not have been incurred by the State otherwise. In addition to
monitoring costs, other adjustments and allocations will be made. The cost
comparison will be for the period of July 1,2003, through June 30, 2004.
Allocations will be based on the following:
a) Divide Central Office or Overhead costs between activities involving residential
prisoners and other activities based on direct expenditures for residential
facilities versus direct expenditures for other activities to obtain a percentage of
Central Office or Overhead expenditures applicable to residential facilities.
b) Allocate the amount of Central Office or Overhead expenditures applicable to
residential facilities based on the census for each residential facility to the total
census for all residential facilities.
c) Expenditures for revenue generating activities such as commissary, inmate
labor, inmate telephones, inmates fines, recycling, and art and craft sales at
institutions are to be included in facility expenditures and will be offset by total
revenues collected.
Costs will be allocated to the South Central Correctional Center for:
d) The pro rata costs of the Tennessee Offender Management Information System
(TOM IS), which are applicable to the handling of information on prisoners
assigned to the SCCC facility
e) The amounts expended by the State for monitoring the Contractor's operations
during the 2003/2004 fiscal year
f)

The amounts expended by the State for the benefit of the Contractor during the
2003/2004 fiscal year

g) Any other amounts expended by the State (including any state agency) which
would not have been expended by the State in the absence of the Contract

10

h) State overhead items determined not to be applicable to SCCC will not be
added to the Contract cost
Adjustments will be also made for:

i)

Year-end supply inventories

j)

Equipment items purchased for use in the facilities with a cost in excess of
$1,000 will be deducted from the total cost of operations for all facilities.
Equipment purchased for use by the State's monitors at SCCC will not be
deducted from the State's cost of operating SCCC

k) Expenditures for the use of motor vehicles and motorized equipment purchased
for use by NECC and NWCC will be reduced by the depreciation/replacement
factor included in the reimbursement rate to the Department of General Services
The Fiscal Review Committee will further adjust the reported costs to ensure
comparability in making the comparison of the relative costs of operating the
facilities for the period of July 1, 2003 through June 30, 2004. This would include,
but not be limited to the following:
I)

Any costs that appear to be made ahead of the time needed or are deferred to a
subsequent period if, in the opinion of the committee staff, such costs are in an
amount sufficient to materially affect the comparison

m) The State's or the Contractor's costs for any program or functional areas which it
determines to be not substantially comparable to the operations of the facilities
bein"g compared
n) Any cost items not accounted for in a similar manner

0) Necessary adjustments for population variance to include fixed and variable cost
items for payroll and operational support expenditures
p) The medical component of cost will be adjusted to equalize the costs of each
facility due to the $4,000 stop-loss provision for medical care in the Contract
Requests for clarification should be made during the Pre-proposal·Conference or
should be requested in the form of a Written Comment during the Request for
Proposal process. The State's written responses will become part of the final
Contract.
As required by the Contract, the Comptroller of the Treasury will review all
accounting information submitted to Fiscal Review by the Department of Correction,
and all accounting information provided by the Contractor to Fiscal Review is to be

11

analyzed by an independent accounting firm. The reports generated by those
reviews will be utilized during the evaluation process.
The Fiscal Review Committee staff will calculate the State's and the Contractor's
cost per inmate day. The final draft report will be given to both the State and
Contractor for comment before it is delivered to the Fiscal Review Committee. If
either the State or the Contractor chooses, they can submit a written response to the
final report that will be included when the report is submitted to the Fiscal Review
Committee.

12