Ca Report on Board of Parole Hearings Psychological Evaluations and Mandatory Training Requirements 2010
Download original document:
Document text
Document text
This text is machine-read, and may contain errors. Check the original document to verify accuracy.
Special Report T he B oa r d o f Pa rol e hearings: p s y c h o l o g i c a l e val uat ions and mandato ry Tr a i n i ng Requirem ent s Office of the Inspector General Dav i d R . S h aw Inspector General S tat e o f C a l i f o r n i a J u ly 2 0 1 0 Office of the Inspector General David R. Shaw, Inspector General July 1, 2010 Matthew L. Cate, Secretary California Department of Corrections and Rehabilitation 1515 S Street, Room 502 South Sacramento, California 95814 Dear Mr. Cate: , Enclosed is the Office of the Inspector General's special report concerning the Board of Parole Hearings' (parole board) process for preparing psychological evaluations used for parole suitability hearings and of the parole board's commissioner training program. The purpose of the special report was to review concerns expressed by the Senate Rules Committee for two particular issues: (1) factual errors that may exist in psychological evaluations and (2) certain psychologists who give elevated risk assessment conclusions when compared to conclusions made in prior evaluations. Additionally, the parole board's executive officer formally requested us to examine its new commissioner training program. The Office of the Inspector General found that the parole board lacks reliable data to determine the number of factual errors contained in psychological evaluations and lacks reliable data to determine the number oflow, medium, and high risk assessment conclusions--data that would allow it to perform certain analytical procedures to measure performance. In addition, we found weaknesses in the parole board's oversight of the methods it uses to review psychological evaluations. Specifically, it does not require senior psychologists to use source documentation when conducting their reviews, limiting the effectiveness ofthe reviewers' ability to deteCt . certain mistakes. The parole board also does not actively monitor senior psychologists' activities by requiring them to account for their time by case or by activity. Finally, the parole board failed to provide commissioners, deputy commissioners, and senior psychologists with the sufficient number of mandatory training hours. We would like to tha~ you and your staff for the cooperation extended to my staff in completing this special report. If you have any questions concerning this report, please contact Jerry Twomey, Chief Assistant Inspector General, Bureau of Audits and Investigations, at (916) 830-3600. P.O. Box 348780, Arnold Schwarzenegger. Governor 95834-8780 PHONE (916) 830-3600 SACRAMENTO, CALIFORNIA FAX (916) 928-4684 Mr. Matthew L. Cate, Secretary Page 2 July 1,2010 Sincerely, David R. Shaw Inspector General cc: Martin Hoshino, Executive Officer, Board of Parole Hearings Kim Holt, External Audits Manager, California Department of Corrections and Rehabilitation Contents Executive Summary .................................................................................................................. 1 Introduction............................................................................................................................... 5 Background ............................................................................................................................... 5 Objectives, Scope, and Methodology .......................................................................................10 Finding 1 ...................................................................................................................................12 The Board of Parole Hearings Does Not Have Reliable Psychological Evaluation Data to Effectively Analyze the Number of Reported Errors or the Types of Risk Assessment Conclusions Finding 2 ...................................................................................................................................20 The Board of Parole Hearings Can Strengthen Its Process of Reviewing Psychological Evaluations Finding 3 ...................................................................................................................................25 The Board of Parole Hearings Failed to Provide All Required Training to Its Commissioners, Deputy Commissioners, and Senior Psychologists California Department of Corrections and Rehabilitation’s Response.....................................28 Executive Summary The Senate Rules Committee, a standing committee in the Legislature that reviews proposed changes to rules and other matters relating to the business of the Legislature, requested that the Office of the Inspector General (OIG) review the Board of Parole Hearings’ (parole board) process of preparing psychological evaluations. The committee was concerned about factual errors in psychological evaluations and concerned about certain psychologists whose risk assessment conclusions are elevated compared to conclusions made in previous evaluations. In addition, the parole board’s executive officer requested that the OIG review its new commissioner training program. Findings in Brief The Office of the Inspector General found that the Board of Parole Hearings (parole board) lacks reliable data to determine the number of factual errors contained in psychological evaluations and lacks reliable data to determine the number of low-, medium-, and high-risk assessment conclusions in evaluations. Reliable data would allow the parole board to perform certain analytical procedures to measure performance. In addition, we found weaknesses in the parole board’s oversight of the methods it uses to review psychological evaluations. Specifically, the parole board does not require senior psychologists to use source documentation when conducting their reviews, thereby limiting the reviewers’ effectiveness in detecting certain mistakes. In addition, the parole board does not actively monitor senior psychologists’ activities by requiring them to account for their time by case or by activity. The parole board is responsible for, among other things, determining whether inmates serving life sentences with the possibility Finally, the parole board failed to provide most of its commissioners, deputy commissioners, and of parole can safely re-enter society. In so senior psychologists with the sufficient number of doing, the parole board’s 12 commissioners mandatory training hours. and approximately 70 deputy commissioners preside over parole suitability hearings for eligible adult inmates. When making determinations of suitability, a hearing panel comprised of at least one commissioner and one deputy commissioner weighs a variety of factors, one of which is the inmate’s past and present mental state. Hearing panels evaluate inmates’ mental state by reviewing Comprehensive Risk Assessments prepared by the parole board’s in-house psychologists. These risk assessments, which are commonly referred to as psychological evaluations, include a clinical opinion of the inmate’s potential risk for committing violence if released from prison. In January 2009, the parole board made changes to its process: it began preparing psychological evaluations by adopting standardized risk assessment instruments and reporting formats. The parole board also hired 30 new psychologists in late 2008; however, 14 of those psychologists were hired for a limited term and will no longer be employed by the parole board after June 30, 2010. Nevertheless, the parole board’s executive management believes that it will remain current with its workload of psychological evaluations at least until October 2010. During this review, we found that the parole board’s data concerning the number of factual errors discovered in psychological evaluations during hearings held in 2009 and in the risk assessment conclusions for evaluations in 2009 is not reliable and therefore is less meaningful to State of California • July 2010 Page 1 its management. According to the parole board’s Lifer Scheduling and Tracking System (tracking system), commissioners reported between one and four factual errors in each of the psychological evaluations of 35 inmates who had parole suitability hearings in 2009. However, we question the reliability of this number because we found examples in which commissioners both overstated and understated the number of factual errors in the tracking system. Consequently, the number of evaluations with factual errors, as well as the total number of factual errors, is unknown. In addition, the parole board does not have reliable data with which to effectively analyze risk assessment conclusions. Using a spreadsheet, the parole board began collecting detailed information about psychological evaluations completed in 2009; however, the central repository of this data was not complete or always accurate. As a result, the parole board cannot analyze the data for problematic trends or patterns, such as identifying psychologists who provide risk assessment conclusions that are potentially inconsistent with the types of inmates they evaluate or psychologists who provide a disproportionate number of risk assessment conclusions that are too high or too low. If the parole board had accurate data, it could review summary reports and more effectively measure its quality control processes, identify new training needs, and make necessary procedural changes. Moreover, we found weaknesses in the parole board’s oversight of its methods of reviewing psychological evaluations. For example, the parole board does not require that senior psychologists use source documentation gathered by their subordinate psychologists when they perform supervisory reviews of psychological evaluations. Consequently, the senior psychologists may not be well-equipped to detect misstatements or inaccuracies during their review. In addition, the parole board does not actively monitor its senior psychologists’ activities by requiring them to account for their time either by case or by activity. As a result, the parole board lacks a necessary management tool to measure and monitor its employees’ performance. Although psychological evaluations are only one of several types of relevant information that commissioners consider in making parole suitability decisions, the sheer magnitude of these decisions warrants that commissioners have the most accurate information possible. Furthermore, we found that the parole board did not provide nearly all of its current commissioners and deputy commissioners with at least 40 hours of annual training, as the law requires. Specifically, the parole board did not provide the three current commissioners appointed before 2008 with 40 hours of annual training in 2008, nor did it provide the seven current commissioners appointed before 2009 with 40 hours of annual training in 2009. Furthermore, the parole board provided only three of 16 current deputy commissioners (19 percent) hired in 2009 with 40 hours of training within 60 days of their appointment as required, and provided only two of 70 current deputy commissioners (3 percent) hired before 2009 with 40 hours of annual training as required. Commissioners and deputy commissioners who do not receive training may not be fully prepared to perform their duties and responsibilities, which could affect hearing decisions. If commissioners and deputy commissioners make mistakes because they were not trained in certain areas, those mistakes could result in legal challenges and a weakening of stakeholders’ confidence in the suitability hearing process. Finally, the parole board did not provide all four of its senior psychologists with 80 hours of mandatory supervisory training within one year of appointment to a supervisory position. Consequently, those supervising employees may not be aware of certain techniques for supervising and instructing employees and protecting employees’ rights. Bureau of Audits and Investigations, Office of the Inspector General Page 2 As a follow-up to this report, the OIG will evaluate the risk assessments for a sample of psychological evaluations to review the soundness of the clinical opinions and determine whether the evaluations contain factual errors. We are in the process of retaining an independent licensed psychologist to perform this work, and we will report on the results of that review. Recommendations To strengthen its processes for collecting and using data and reviewing psychological evaluations, the parole board should take the following actions: • Ensure that all of its commissioners who preside over parole suitability hearings receive formal training on how and when to report factual errors in the Lifer Scheduling and Tracking System (tracking system). This action should include ensuring that commissioners clearly articulate on the record the disposition of claimed errors in psychological evaluations and record the result in the tracking system accurately. • Periodically review a report on the number and types of factual errors in psychological evaluations recorded in the tracking system. This review would identify training opportunities for certain psychologists who repeatedly make the same types of errors and would also determine whether changes in the tracking system process are necessary. • Ensure that psychologists report to the chief psychologist all of their risk assessment conclusion information so that the chief psychologist can analyze the data for patterns of conclusions and can identify trends over time. • Ensure that senior psychologists review copies of source documents from the inmate’s central file when reviewing psychological evaluations for parole suitability hearings. This source documentation review will increase the integrity of the review process by helping to identify misstatements or inaccuracies contained within the evaluations. • Develop a review checklist for senior psychologists to follow when conducting reviews of psychological evaluations. This checklist would aid in fact-checking the statements and conclusions in the psychological evaluations and ensure consistency and thoroughness among reviewers. • Develop a time benchmark for reviewing psychological evaluations to ensure that reviewers spend at least the minimum amount of time conducting their reviews. • Require senior psychologists to report to the chief psychologist their time spent reviewing draft evaluations. This information would help the chief psychologist track productivity and manage workload levels. To ensure that its employees receive the required number of training hours, the parole board should regularly monitor training activities and the number of training hours that it provides to all staff. The parole board should then make the necessary scheduling adjustments for those individuals who have not received enough hours. State of California • July 2010 Page 3 Department’s Response The California Department of Corrections and Rehabilitation (CDCR) indicated that it would implement many of our recommendations in the coming fiscal year. The CDCR also stated that it is exploring or has already implemented different methods to ensure that it meets the number of training hours required for commissioners and deputy commissioners annually. Finally, the CDCR stated that it will monitor and document the Board of Parole Hearings’ progress in addressing the report’s recommendations. Bureau of Audits and Investigations, Office of the Inspector General Page 4 Introduction Background Among other responsibilities, the Board of Parole Hearings (parole board) conducts parole suitability hearings for all inmates sentenced to life in prison with the possibility of parole (referred to as an “indeterminate sentence”) to determine whether the inmate can re-enter society or requires a longer period of incarceration to ensure public safety.1 Only inmates sentenced to life in prison with the possibility of parole are subject to suitability hearings, for which they automatically become eligible one year before completing their minimum sentence. Merely being scheduled for a suitability hearing does not indicate an inmate’s suitability for release from prison. Rather, a hearing panel—consisting of one commissioner and one deputy commissioner—determines an inmate’s suitability for parole. The parole board is one of several divisions or boards within the California Department of Corrections and Rehabilitation. The parole board has 12 commissioners who conduct hearings related to adult offenders. These commissioners are appointed by the governor and confirmed by the Senate. The parole board also uses about 70 deputy commissioners for parole suitability hearings. Hearing panels weigh various factors when determining suitability for parole The California Code of Regulations requires that the hearing panel consider all relevant and reliable information available to the panel when determining whether an inmate is suitable for release on parole. When making its determination, a hearing panel weighs a variety of factors, including an inmate’s social history, past and present mental state, criminal history, commitment offenses, past and present attitude toward the crime, any conditions of treatment or control, and any other information bearing on the inmate’s suitability for release. According to state law, regulations, and case law governing parole suitability decisions, a hearing panel shall find an inmate unsuitable for parole if, in the judgment of the panel, the inmate will pose a current, unreasonable risk of danger to society if released from prison. Table 1 on the following page shows examples of factors that tend to indicate suitability and unsuitability for parole. Parole suitability hearing results for January 1, 2007, through December 31, 2009 According to the parole board’s Lifer Scheduling and Tracking System, the number of suitability hearings has increased each calendar year from 2007 through 2009. Figure 1 (on page 7) shows the percentage of parole suitability hearing results by type for the period January 1, 2007, through December 31, 2009. The parole board’s data also shows—as illustrated in Figure 2 (on page 7)—that for the three-year period, the number of parole grants has increased each year, while denials and postponements have decreased overall since 2007. Where a decision of grant or denial was reached, the percentage of 1 An indeterminate sentence is a prison term that is not fixed in advance by the court but instead ranges over an indefinite period, such as “25 years to life.” State of California • July 2010 Page 5 Table 1: Factors Commissioners Consider For Determining the Suitability or Unsuitability of Inmates at Hearings. Suitability Unsuitability • No juvenile record (of assaulting or potentially harming others) • Commitment offense (especially heinous, atrocious, or cruel manner) • Stable social history • Previous record of serious violence • Signs of remorse • Unstable social history • Motivation for crime was significant stress in life • History of sadistic sexual offenses • Crime was the result of Battered Woman Syndrome • Lack of significant history of violent crime • Psychological factors (lengthy history of severe mental problems) • Institutional behavior • Present age • Realistic plans for future • Institutional behavior Source: The California Code of Regulations, Title 15, Section 2402 (c) and (d). suitability hearings in which commissioners granted parole quadrupled, from 4 percent in 2007 to 16 percent in 2009 (refer to Figure 3 on page 8). The parole board recently made changes to improve its process of preparing psychological evaluations Before January 1, 2009, psychological evaluations prepared for parole suitability hearings followed differing report formats and included a variety of means to determine an inmate’s potential for violence if released from prison. Some of the evaluations did not include the use of recognized risk assessment tools and did not have an expiration date. Further, the parole board noted that it did not employ a sufficient number of psychologists to complete its scheduled psychological evaluations, which led to postponements and an increase in hearings conducted outside of the required time period. In late 2008, the parole board hired an additional 30 psychologists, including two senior psychologists and a chief psychologist. And in January 2009, the parole board adopted a new approach for preparing psychological evaluations, one that uses standardized risk assessment instruments and consistent report formats. Generally speaking, this new process includes a mental status examination and an evaluation of the inmate’s remorse, emotional functioning, insight, selfunderstanding, psychiatric symptoms, and risk factors from the inmate’s history, such as the role drugs and alcohol played in the commitment offense. The process also includes an explanation of the commitment offense and the inmate’s attitude regarding it, as well as a history of the inmate’s prison programming. The evaluation process culminates in the psychologist’s opinion, based on available data, of the inmate’s potential for future violence. This opinion is expressed in terms of the inmate’s having a “low,” “moderate,” or “high” risk for future violence. The parole board explains that it uses the psychological evaluation as relevant information to help commissioners make determinations about the inmate’s mental health and attitude towards his or her crime. Bureau of Audits and Investigations, Office of the Inspector General Page 6 Figure 1: Composition of the Results of Parole Suitability Hearings for the Period January 1, 2007 through December 31, 2009. Grants 5% Other* 12% Denials 44% Waivers 7% Postponements 32% Source: The Board of Parole Hearings’ Lifer Scheduling and Tracking System. Unaudited data. 3,500 3,000 Figure 2: Three-Year Comparison of Suitability Hearing Results for January 1, 2007, through December 2,500 31, 2009. 3,500 2,000 2007 3,000 2008 2009 1,500 2,500 1,000 2,000 2007 2008 500 2009 1,500 - Grants 1,000 Denials Postponements Waivers Grants Denials Postponements Other* 500 - Waivers Other* Source: The Board of Parole Hearings’ Lifer Scheduling and Tracking System. Unaudited data. The “Other” category in Figures 1 and 2 includes stipulations, in which an inmate agrees in advance of a hearing that he or she is unsuitable for parole; split decisions, in which the hearing panel disagrees on the decision, resulting in a “tie” vote; cancellations, in which the hearing is cancelled for circumstances such as a change in the inmate’s minimum eligible release date or the inmate’s death; continuations, in which a hearing is stopped and later reheard by the same panel; and a category titled, “No Result.” The “Grant” category in Figures 1, 2, and 3 includes decisions made by the parole board but does not necessarily equate to the number of inmates actually released. State of California • July 2010 Page 7 Figure 3: Percentage Comparison of Parole Grants and Denials at Suitability Hearings in 2007 and 2009. Grants 4% Percentage of Grants and Denials in 2007 96% Denials Grants Percentage of Grants and Denials in 2009 16% 84% Denials Source: The Board of Parole Hearings’ Lifer Scheduling and Tracking System. Unaudited data. The process of preparing psychological evaluations begins when a staff psychologist travels to the prison for a face-to-face interview with the inmate. At this visit, the psychologist reviews the inmate’s central file and medical information and later uses this information to prepare a draft evaluation using a number of standardized risk assessment instruments. These instruments include the following: • Hare Psychopathy Check List-Revised, which is a diagnostic tool used to identify psychopathic tendencies. This tool uses a semi-structured interview, file documentation, and collateral information to measure inferred personality traits and behaviors related to a traditional conception of psychopathy. • Historic-Clinical-Risk-Management-20 (HCR-20), which is an assessment tool that provides an analysis of an individual’s risk potential. This tool consists of a set of structured professional guidelines for the assessment of risk for institutional and community violence in people with mental abnormalities, personality disorders, or substance use disorders. According to the parole board, the HCR-20 has been cross-validated for use in correctional, forensic, and psychiatric settings. • Level of Service/Case Management Inventory, which is a tool designed to assist professionals in managing and treating offenders in correctional and forensic settings. The tool includes both an actuarial estimate of criminal recidivism and a full service case management component. However, the parole board notes that its psychologists use the tool for assessing general criminal recidivism potential (but not violence per se) and not for treatment planning and case management. • Static-99-R, which is an instrument used for assessing risk for sexual and nonsexual violence recidivism. This tool is intended for male individuals who have been arrested for, or convicted of, a sex offense as an adult (and in some cases as a juvenile), or when there is reason to believe that a sex offense against an identifiable victim had occurred. The parole board’s chief psychologist estimates that it takes psychologists an average of 21 hours (spanning approximately 45 days) to prepare an evaluation from start to finish. The staff Bureau of Audits and Investigations, Office of the Inspector General Page 8 psychologist forwards the draft evaluation to a senior psychologist for review and approval. After the senior psychologist approves the evaluation, the parole board sends it to the prison where copies are provided to the inmate and placed in the inmate’s central file. State law permits an inmate the opportunity to review his or her file (which presumably includes a psychological evaluation) at least 10 days prior to any hearing conducted by the parole board.2 According to the parole board’s orientation manual, evaluations completed on or after January 1, 2009, are valid for five years; however, evaluations completed before 2009 are valid for three years from the date of the evaluation or will expire by attrition after use in a hearing occurring after January 1, 2009, whichever occurs first. According to the parole board, it will lose 14 staff psychologists on June 30, 2010, when these employees’ limited term positions expire. After this staffing reduction, the parole board will have 35 psychologists, including four senior psychologists and a chief psychologist. When we spoke to the parole board’s executive officer and chief deputy of program operations about this issue, the executive officer said that due to a number of variables affecting the future pace of suitability hearings, he could not predict if or when the parole board would begin to experience a negative impact from the staffing reduction. The executive officer noted that one of the variables is the commissioners’ availability to conduct suitability hearings. The chief deputy of program operations explained that, due to Marsy’s law, the parole board’s future suitability hearing workload could be less compared to the workload of previous years because the minimum denial period changed from one year to three years.3 He also said that because the parole board schedules the preparation of evaluations 180 days in advance, he believes that even with the loss of 14 psychologists, the parole board will be able to keep up with its psychological evaluations workload at least until October 2010. 2 In re Rutherford, the court ordered, in part, that the parole board make psychological evaluations available to the inmate and the inmate’s attorney at least 60 days prior to the date of the inmate’s scheduled parole suitability hearing. According to the parole board’s Lifer Scheduling and Tracking System for July 1, 2009, through December 31, 2009, the average number of days before the inmate’s suitability hearing date that the psychological evaluation was approved was 159 days. 3 In late 2008, voters passed Proposition 9, the Victim’s Bill of Rights Act of 2008 (known as “Marsy’s Law”), which, among other things, is intended to limit the number of parole hearings that a convicted murderer can receive. Under Marsy’s Law, a convicted murderer can receive a parole hearing no more frequently than once every three years and can be denied a follow-up parole hearing for as long as 15 years. State of California • July 2010 Page 9 Objectives, Scope, and Methodology At the request of the Senate Rules Committee, which is a standing committee in the Legislature that reviews proposed amendments to rules and other matters relating to the Legislature, and pursuant to Penal Code section 6126, the Office of the Inspector General (OIG) conducted a review of the process that the parole board uses to prepare the psychological evaluations for its parole suitability hearings and conducted a review of the parole board’s commissioner training program. The Senate Rules Committee expressed concern about two particular issues: (1) factual errors that may exist in psychological evaluations and (2) certain psychologists who give inmates elevated risk assessment conclusions compared to conclusions made in those inmates’ earlier evaluations. Furthermore, the parole board’s executive officer formally requested that we examine its new commissioner training program. As part of our review, we performed the following actions: • Reviewed relevant laws, rules, and regulations. • Interviewed key members of the parole board’s management team, commissioners, and deputy commissioners. • Interviewed the parole board’s chief psychologist and two senior psychologists to understand the process of preparing and reviewing psychological evaluations. • Interviewed attorneys who represented inmates at parole suitability hearings to discuss their perceptions of the prevalence of factual errors in psychological evaluations. • Observed parole suitability hearings to understand the process. • Reviewed a sample of 38 central files for inmates with recent parole suitability hearings to gain an understanding of the contents of psychological evaluations and to determine whether there were certain types of factual errors in the evaluations. For efficiency, we selected our sample from specific prisons in Northern California instead of from the parole board’s universe of inmates with psychological evaluations. Although our testing revealed one substantial error wherein an evaluation incorrectly listed the inmate’s life crime, we did not use our sample to make projections about the total number of factual errors that may actually exist in psychological evaluations. • Reviewed the parole board’s electronic data concerning the number of errors reported by attendees at parole suitability hearings in 2009. • Reviewed the parole board’s electronic data concerning the risk assessment conclusions of psychological evaluations for parole suitability hearings held in 2009. • Reviewed and analyzed the parole board’s training information concerning the number of training hours the current commissioners and deputy commissioners received in 2008 and 2009. The OIG’s Bureau of Independent Review (BIR) is currently collaborating with the parole board’s executive staff to develop a transparent training curriculum for all commissioners and deputy commissioners. The BIR intends to identify judicial training topics relevant to the parole board’s work and complementary to the parole board’s ongoing internal training. The training curriculum is scheduled to be completed by December 2010. Bureau of Audits and Investigations, Office of the Inspector General Page 10 As a follow-up to this report, the OIG will evaluate the risk assessments for a sample of psychological evaluations to review the soundness of the clinical opinions and to determine whether the evaluations contain factual errors. We are in the process of retaining an independent licensed psychologist to perform this work and will report on the results of that review. State of California • July 2010 Page 11 Finding 1 The Board of Parole Hearings Does Not Have Reliable Psychological Evaluation Data to Effectively Analyze the Number of Reported Errors or the Types of Risk Assessment Conclusions In January 2009, commissioners began tracking the number of factual errors—labeled “administrative” or “substantial”—discussed during parole suitability hearings concerning psychological evaluations. The Board of Parole Hearings (parole board) tracks factual errors in psychological evaluations in its Lifer Scheduling and Tracking System to identify cases that the commissioners believe warrant a secondary review by the chief psychologist. Although the data may serve that purpose, it cannot be used for other useful management purposes, such as determining the total number of psychological evaluations containing factual errors nor determining the total number of factual errors, because the data in the system is not complete or accurate. For example, we found cases in which commissioners overstated the number of factual errors when they reported errors into the tracking system, and we found other cases in which commissioners understated the number of factual errors by failing to report some factual errors into the tracking system. Also in January 2009, the parole board’s chief psychologist began collecting detailed risk assessment results for psychological evaluations using a spreadsheet. Yet this data, too, is unreliable as a management tool because it is incomplete and inaccurate. Consequently, the chief psychologist cannot use the data to effectively analyze patterns or trends in the types of risk assessment conclusions given by psychologists. When the parole board adopted its new process of preparing psychological evaluations in January 2009, it also began requiring commissioners to track the number of factual errors identified in the evaluations during parole suitability hearings, which included classifying each error as either “administrative” or “substantial.” According to the parole board, administrative errors are misstatements of fact, such as occasional references in the evaluation to a wrong inmate name, wrong ethnicity, inaccurate family history, or any similar error. Substantial errors are more serious misstatements of fact that could affect the validity of the evaluation, such as an incorrect crime for which the inmate was sentenced to life in prison or an incorrect number of victims. At the conclusion of every hearing, the commissioners enter into the tracking system a variety of data elements, such as the hearing result, details of the decision, and the numbers of administrative or substantial errors identified in the inmate’s psychological evaluation. If a commissioner reports that an evaluation has three or more administrative errors or just one substantial error, the parole board’s policy requires a secondary review of the evaluation by the parole board’s chief psychologist, a process separate from the suitability hearing. As part of the secondary review, the chief psychologist (or his designee) determines whether the reported errors adversely affect the validity of the evaluation and then either prepares an addendum to the evaluation or orders a new evaluation if he or she deems the errors to be significant. Bureau of Audits and Investigations, Office of the Inspector General Page 12 According to our interviews with six of the 12 current commissioners who preside over suitability hearings, the commissioners collectively believe that the number of factual errors in psychological evaluations is relatively low. As Table 2 shows (on the following page), four of the six commissioners perceived the rate of factual errors in psychological evaluations as less than 5 percent; however, one commissioner perceived the rate of administrative errors to be between 10 and 15 percent. We also asked the commissioners to rate, using a scale between 1 (low) and 10 (high), how reliable they believe the psychological evaluations to be, and all but one reported a score of at least eight. Commissioner Six rated the reliability of the evaluations a five and explained that psychologists need to be more thorough with their inquiries. This commissioner also commented that psychologists seldom look at the central files to ascertain if the inmate is being truthful. Nevertheless, most of these commissioners reported they had a strong positive impression of the evaluations’ reliability. In contrast to the commissioners’ generally favorable remarks, we received mostly negative comments from our discussions with six attorneys who represented inmates at their parole suitability hearings. The attorneys’ perceptions of the frequency of factual errors in psychological evaluations ranged from 20 percent to 100 percent. Further, when asked about their perceptions of the reliability of psychological evaluations, the attorneys gave us responses ranging from “two” to “seven.” The difference between the commissioners’ and the attorneys’ perceptions of the frequency of factual errors in psychological evaluations illustrates the importance of the parole board having reliable data. The parole board does not have reliable data to determine the total number of factual errors in psychological evaluations In January 2009, the parole board began tracking administrative and substantial errors contained in the psychological evaluations discussed at parole suitability hearings. During parole suitability hearings, commissioners use the tracking system to record up to four administrative errors and two substantial errors for each psychological evaluation. The system also includes a hearing comment field where commissioners may type other relevant notes from the hearing that are not otherwise captured in a specific field. These fields serve as a mechanism for commissioners to request a secondary review of the psychological evaluation by the chief psychologist. The parole board also notifies the chief psychologist when commissioners record at least three administrative errors or one substantial error in the tracking system. The chief psychologist reviews the errors in those cases to determine what impact the errors may have on the validity of the psychological evaluations. For the purposes of our audit, we requested a report of all the errors that commissioners recorded during 2009. The parole board produced an ad-hoc report showing that commissioners recorded between one and four factual errors in each of the psychological evaluations of 35 inmates in 2009. However, we have concerns about the reliability of this data because we found instances in which commissioners either overstated or understated the number of errors in the tracking system. For example, in one case, a commissioner reported in the tracking system that an inmate’s psychological evaluation contained an error: the staff psychologist had not completed a Static-99 risk assessment (used for inmates who were arrested for or convicted of a sex offense as an adult). Then in order to trigger a secondary review of this inmate’s case, the commissioner recorded in the tracking system that the evaluation contained two additional errors. He checked two more boxes and in those boxes’ State of California • July 2010 Page 13 Table 2: Perception of Reliability and of the Frequency of Factual Errors in Psychological Evaluations Observed at Suitability Hearings by a Sample of Commissioners and Attorneys Who Represented Inmates at Parole Suitability Hearings. Commissioner or Attorney Commissioner One Commissioner Two Commissioner Three Commissioner Four Commissioner Five Commissioner Six Attorney One Attorney Two Attorney Three*** Attorney Four Attorney Five Attorney Six Perception of the Reliability of Psychological Evaluations* Perception of the Frequency of Factual Errors in Psychological Evaluations “Seven or eight.” Substantial errors: “1 or 2 percent.” Administrative errors: “10 to 15 percent” [The evaluations] are very useful. The newer [evaluations] are much better, since the [parole board] implemented the new format. “Nine point five.” “Less than 2 percent.” The amount of errors has decreased recently. When I first started there were a significant amount of errors. An example of a common error is when the inmate’s name is wrong on the evaluation. “Nine or 10.” Substantial errors: “Less than 2 percent.” Administrative errors: “Less than 5 percent.” [The evaluations] have a high degree of reliability. There is little contradiction within the evaluations. “1 percent.” Because the [evaluations] incorporate findings from multiple validated instruments, and are administered by skilled clinicians, I regard them as likely to be highly reliable. “0 percent.”** The psychologists do a pretty good job. I will usually ask the inmate at the hearing for their feedback on the evaluation and most of them usually agree with it. “5 to 10 percent.” Evaluations are based on what the inmate said at the time to the psychologist, not what is in his [central] file. The inmate can lie and the psychologist will base their opinion on this lie. The psychologist doesn’t typically read the [central] file. “100 percent.” I have never seen an evaluation that did not have factual errors… Not all of the errors are significant, but the psychologist often makes conclusions from incomplete information. “50 percent.” Fairly often, a fair percentage (maybe as high as 50%) of the reports have some factual errors. Many will have relatively minor errors; for example, the date of crime is slightly off, which is not significant to determining overall risk. Many of the factual errors are substantial errors. These include factual findings or findings on risk assessment tools that are stated in conclusory fashion without any explanation. “50 percent.” The errors I encountered varied from simple scrivener’s errors to material errors. The implication is that no one reviewed the reports for errors. “20 percent.” The doctors are not following standard procedures. One doctor only reviewed 15 percent of the [central] file in their report, and we (myself and inmate) identified the missing information from the report in the [central] file. “Up to 20 percent.” The psychologist’s report is subjective, and sometimes they get it right, and sometimes they do not. The psychologists purport to make the evaluation a science, and I do not believe it is science. “95 percent.” Many of the errors are inconsequential to the evaluation as a whole, but are still errors. Some of the errors I see are wrong age and date. Even if the errors are inconsequential, they can still lead to a misunderstanding. “Seven or eight.” “Eight.” “Five.” “Seven.” “Two or three.” “Five.” “Three.” “Five.” “Five.” Notable Comments from Commissioners or Attorneys * Ratings are based on a scale of 1 to 10, with one being the lowest and 10 being the highest. ** Commissioner Five reported that he is a new commissioner and has not yet observed any factual errors in psychological evaluations during a suitability hearing. *** Attorney Three said that she has not represented inmates at parole suitability hearings for over a year. Source: Telephone interviews with commissioners and attorneys Bureau of Audits and Investigations, Office of the Inspector General Page 14 description fields, he wrote, “OK, OK.” When we spoke to the chief psychologist about this case, he explained that the reported errors were not really administrative errors at all but were instead used by the commissioner to have the chief psychologist consider another risk assessment instrument for this inmate. The parole board’s policy specifies that if a psychological evaluation contains three or more administrative errors or one substantial error, the chief psychologist (or his designee) will conduct a secondary review. In this case, the commissioner overstated the number of errors in the psychological evaluation when he reported his concern as errors in the tracking system. The commissioner could have used the hearing comment field to report his concern. By instead reporting his concern as additional administrative errors, he rendered the error-tracking data inaccurate. We also found six examples of errors that commissioners incorrectly recorded in the hearing comment field instead of in the correct administrative or substantial error fields, causing the number of reported errors in the correct fields to be understated. All six of these inmates had their suitability hearings postponed as a result of errors in their psychological evaluations, but the six cases did not appear on the official error report that the parole board prepared for us because the errors weren’t recorded in the proper fields. For example, in one of the cases, the presiding commissioner recorded in the hearing comment field a concern that the psychological evaluation referred to a prior charge of homicide for which the inmate was acquitted, but it was unclear to the hearing panel how, if at all, this information was incorporated into the evaluator’s risk assessment. At the hearing, the commissioner said that based on the new protocol by the forensic assessment division, any time it appears that there may be a substantial error in the [psychological evaluation] we as commissioners are afforded the opportunity to request a [secondary] review with us citing the basis for our request, which I have done, and essentially the question for the Panel and for the secondary review is a determination to what degree and extent, if any, the use of a significant crime, essentially another homicide of which [the inmate] was acquitted, had in [the evaluator’s] risk assessment. Despite the commissioner’s statements that he believed there may be a substantial error in the evaluation, he did not correctly record his concern as an error in the tracking system, but rather documented it in the “hearing comments” field where it would not be counted as an error. In addition, commissioners do not always enter errors discussed at hearings in the system at all. Consequently, we do not know how many other examples of unreported errors actually exist. We expected to see comments on the hearing transcript (record) from the commissioners to ensure that they acknowledged the concerns raised by the inmate or by his or her attorney. Without those comments, it is not clear from reviewing the transcript whether the commissioners disagreed with the claimants and therefore did not record the errors or agreed with the claimants and should therefore have recorded the errors in the tracking system. For example, in one case, an inmate’s attorney said at the hearing, “[The inmate] has indicated to me that there are several gross inaccuracies in the psychological report, to the extent to which [the inmate] believes that the clinician actually confused [the inmate] with someone else.” The attorney further noted that the inmate prepared a multiple-page refutation concerning the accuracy of the report and asked for a new psychological evaluation. Although the attorney did not provide specific examples of the errors nor the total number of errors on the record, the commissioners did not ask for any clarifying information nor did they refute the attorney’s claims on the record. In another example, State of California • July 2010 Page 15 an attorney representing an inmate at his suitability hearing expressed concerns about the inmate’s psychological evaluation and said, “One of the things that stands out as rather significant is there’s reference to another inmate on this report.” As in the previous example, in this case the commissioner did not refute the claims on the record. However, in neither of these cases did the commissioners record the errors in the tracking system. The parole board includes as part of its new commissioner training program a document that briefly describes the process of recording administrative and substantial errors when requesting a secondary review of an evaluation. Yet only two of the six commissioners we interviewed said Table 3: A Sample of Commissioners’ Perceptions Regarding the Sufficiency of Training Received in Performing Duties as a Commissioner and Tracking Factual Errors. Commissioner Perception of the Sufficiency of Training Received in Performing Duties as a Commissioner Commissioner One “Eight or nine.” I felt it was excellent. Commissioner Two “Seven.” Training has changed since I started with the board. There have been several administrative changes that have led to improvement in training. Responses from Commissioners on whether they have received training on tracking factual errors Notable Comments from Commissioners Regarding Their Process for Recording Factual Errors in the Lifer Scheduling and Tracking System (LSTS) No. [Errors] are always entered into the comment section [of LSTS]. No ... [I] was absent the day the [parole board] held a class on the matter. In LSTS … using the ‘Hearing Comment Field.’ [I] record the errors using the check-boxes [for administrative and substantial errors] only if the panel requests a secondary review. Yes … I received training in the beginning of 2009 when updates were done. Based on my judgment … I only enter errors in LSTS if the hearing is postponed and the panel requests a secondary review of the evaluation … If an evaluation has one or two administrative errors, I will not enter the errors in LSTS. Commissioner Four “Nine.” I was one of two people who went through the class for the first time. It was very informative. The class was very effective, dynamic, and challenging. Don’t recall having any. I do not know what is meant by administrative [versus] substantial … as a commissioner, I do not track errors. However, if a hearing were to be continued or postponed for any reason, including an issue with the [psychological evaluation], I would document this in LSTS, and have been trained to do so. Commissioner Five “Eight plus.” I got one-on-one training. Training was very expansive … The whole process made me feel comfortable and prepared to perform my duties at suitability hearings. Yes. When I received my initial training. [Errors are] put on record and put in the comments section of LSTS. Commissioner Six “Three or four.” The initial training [I] received was poor ...Training has improved, but due to the experience level of each commissioner, it is difficult to establish [training] that will suit everyone. No … we don’t ‘track’ errors. [Errors are] put on record and are articulated in the comments section of LSTS. Commissioner Three “Seven to eight.” When I first came on I needed more training … I feel adequately prepared to conduct hearings. Source: Interviews with commissioners. Bureau of Audits and Investigations, Office of the Inspector General Page 16 that they received training specifically related to recording errors identified in psychological evaluations. We learned from interviews with the commissioners that the process of recording errors is inconsistent at best. For example, as shown in Table 3 (on page 16), Commissioner One told us that errors are always entered, whereas Commissioner Three indicated that he uses his judgment when deciding to record errors. In the latter example, the commissioner explained that he only enters errors when the panel decides to postpone the hearing and requests a secondary review of the evaluation. Additionally, the commissioner told us that he does not always record errors if there are fewer than three and the panel does not request a secondary review of the evaluation. Commissioner Four, however, did not understand what we meant by “administrative” and “substantial” errors, commenting that the term “administrative errors” could mean many things. The commissioner did not recall having received specific training about the process of recording errors. Had he received such training, he would have understood the parole board’s terminology. The parole board should ensure that commissioners report in the tracking system all of the factual errors in psychological evaluations unless the commissioners are able to refute the errors on the record. Once the parole board has complete and accurate data regarding factual errors, it should periodically review the same type of summary report it prepared for us. This periodic review would help it gauge the effectiveness of its quality control processes, identify training needs for staff, and make procedural changes as necessary. The parole board does not have reliable data to determine the number and types of risk assessment conclusions for psychological evaluations in 2009 As we discuss in the Scope and Methodology section of this report, the Senate Rules Committee was concerned that certain psychologists give only elevated risk assessment conclusions when compared to previous conclusions. We reviewed a sample of 38 inmates who recently had a parole suitability hearing to gain an understanding of the contents of their psychological evaluations and to look for the existence of certain types of factual errors. However, as we describe in the introduction of this report, the parole board standardized the format and use of risk assessment tools for psychological evaluations beginning in January 1, 2009. Thus, psychological evaluations prepared before that date do not always follow the same approaches as later evaluations and therefore do not lend themselves to simple comparisons with the newer evaluations. Accordingly, we asked the parole board’s chief psychologist whether the parole board collects risk assessment data that could be used for analytical purposes, such as identifying patterns or trends in the types of conclusions provided by psychologists. In response to our query, the chief psychologist said he started capturing detailed risk assessment conclusions for psychological evaluations using a spreadsheet beginning in January 2009. After a staff psychologist completes a draft evaluation, the psychologist enters the risk assessment conclusions into his or her individual spreadsheet and periodically reports the information to his or her senior psychologist, who in turn, sends the information to the chief psychologist. The chief psychologist compiles the information into a single spreadsheet. We obtained the chief psychologist’s data for evaluations prepared between January 1, 2009 and November 30, 2009, consisting of 3,077 evaluations prepared by 44 psychologists. The State of California • July 2010 Page 17 psychologists presented their conclusions using a range of labels, as we show in the text box at right. Psychologists present risk assessment conclusions using the following terminology: In testing this data for completeness, we compared the data from the physical copies of a sample of 40 evaluations to the corresponding • Low data field in the spreadsheet. We could not locate three of the 40 • Low to Moderate evaluations (the missing evaluations were conducted by three different • Moderate psychologists) in the spreadsheet. In addition, we found one instance • Moderate to High in which the overall assessment in the physical copy of the evaluation was labeled as “moderate to high,” but in the spreadsheet, the overall • High assessment was incorrectly recorded as “moderate.” Consequently, we found that the data in the spreadsheet is not complete or always accurate. Without reliable data, the parole board cannot use the information to determine the total number of low-, medium-, or high-risk conclusions or fully analyze the types of risk assessment conclusions provided by its psychologists, and it cannot fully analyze patterns of conclusions associated with particular psychologists or certain trends over time. Despite the data’s limitations, our review revealed that psychologists provided a wide range of conclusions and that no particular psychologist gave only “moderate” or “high” risk conclusions. For example, the psychologists gave an average of 32 “low” or “low to moderate” risk conclusions as their overall assessment. In contrast, the psychologists gave an average of 14 “high” or “moderate to high” risk conclusions as their overall assessment. The average number of “moderate” risk conclusions per psychologist was 24. After we discussed this issue with the chief psychologist, he prepared a written analysis of the psychologists who gave the greatest number of “high” conclusions and discussed the reasons, in his opinion, that their conclusions were appropriate to their workload. Specifically, he pointed out that although these psychologists gave the greatest number of “high” risk conclusions, they also gave “low” risk conclusions on other evaluations, suggesting that these psychologists are capable of identifying low risk when it exists. Furthermore, one particular psychologist tended to evaluate a large number of sex offenders. The chief psychologist noted that “sex offenders do not receive treatment for salient dynamic risk factors in prison and so these problems do not get addressed, which elevates their risk.” He also noted that another of these psychologists evaluated a large number of inmates with “severe and persistent mental disorders which impacts their level of dangerousness.” Recommendations To ensure that the Board of Parole Hearings has and uses reliable data, it should take the following actions: • Ensure that all of its commissioners who preside over parole suitability hearings receive formal training on how and when to report factual errors in the Lifer Scheduling and Tracking System (tracking system). This action should include ensuring that commissioners clearly articulate on the record the disposition of claimed errors in psychological evaluations and record the result in the tracking system accurately. Bureau of Audits and Investigations, Office of the Inspector General Page 18 • Periodically review a report on the number and types of factual errors in psychological evaluations recorded in the tracking system. This review would identify training opportunities for certain psychologists who repeatedly make the same types of errors and would also determine whether changes in the tracking system process are necessary. • Ensure that psychologists report to the chief psychologist all of their risk assessment conclusion information so that the chief psychologist can analyze the data for patterns of conclusions and can identify trends over time. State of California • July 2010 Page 19 Finding 2 The Board of Parole Hearings Can Strengthen Its Process of Reviewing Psychological Evaluations The Board of Parole Hearings (parole board) employs four senior psychologists who review psychological evaluations. These senior psychologists, who report to the parole board’s chief psychologist, each supervise between nine and 12 staff psychologists who prepare psychological evaluations. We found that the parole board can improve its oversight of the senior psychologists’ methods for reviewing psychological evaluations. Specifically, we found that the parole board could require senior psychologists to use source documentation when conducting their supervisory reviews. Not using source documentation limits the senior psychologists’ ability to validate statements and conclusions presented by staff psychologists in their evaluations and to detect factual errors in their subordinates’ work. In addition, the parole board could actively monitor its senior psychologists’ activities by requiring them to account for their time by case or activity. This would aid the parole board in holding its senior psychologists accountable if reviews are not done properly. However, without this information, the parole board lacks a necessary management tool to measure and monitor its employees’ performance. The preparation and review of psychological evaluations is unquestionably challenging, given the complex and subjective nature of assessing an inmate’s potential risk for future violence. We recognize that even with strong controls in place, the process of preparing and reviewing psychological evaluations will be subject to some margin of error. Further, we understand that psychological evaluations are one type of relevant evidence that commissioners consider when making parole suitability decisions. However, because the decision to grant or deny parole is critical, it is reasonable to expect that decision be made using the most accurate information available. Therefore, the changes we recommend are intended to strengthen the parole board’s review process and ultimately lead to fewer errors in evaluations. The parole board does not require its senior psychologists to review source documentation when conducting reviews of psychological evaluations The parole board provides staff psychologists with templates to use when drafting their evaluations. These templates provide the psychologists with a standardized report format and include instructions, examples, and guidance concerning each topic area. When conducting reviews of draft evaluations, the senior psychologists may refer to the templates. In addition, as part of orientation training, the parole board provides senior psychologists with a list of elements that they must look for when reviewing psychological evaluations. As shown in the text box on the following page, this information provides senior psychologists with a review framework and identifies the parole board’s expectations for a well-prepared psychological evaluation. Based on the review guidance provided by the parole board, we anticipated finding that the senior psychologists’ reviews included key source documents to ensure that staff psychologists’ conclusions are supported by evidence and are factually accurate. By agreement between the Bureau of Audits and Investigations, Office of the Inspector General Page 20 prisons and the parole board, employees at the prison make copies of key records from the inmate’s central file for the psychologists’ use. These records include Probation Officer’s Reports providing details of the inmate’s commitment offense (life crime), social factors, and criminal history, as well as the probation officer’s recommendation to the court. Other examples of records provided to the psychologists include chronological lists of rules violation reports and counseling notes prepared for the previous hearing; the last board report that includes the inmate’s case factors; and the Legal Status Summary, which provides information regarding the commitment offense and length of prison term. Surprisingly, although psychologists use these records in preparing their draft evaluations, the senior psychologists do not use them in their reviews of the psychologists’ evaluations. The parole board provided senior psychologists with the following review guidelines: 1. The report was comprehensive and provided adequate summaries of the clinical interview, file records, and inmate’s self-report. 2. Data Critique: The report analyzed the available data, pointed out inconsistencies or discrepancies in the records, changes in the inmate’s accounts and contradictions in the records, and critiqued the conclusions and opinions contained in other psychological evaluations, particularly if the previous conclusions were not supported by the data. 3. The opinions and conclusions pertaining to violence risk and diagnosis were logical, flowed directly from the data, were empirically valid, and well supported. 4. Based on the information provided in the report, the risk assessment instruments appeared to be scored accurately and the interpretations were appropriate. 5. Clinical judgments pertaining to either the overall The parole board’s review guidelines (see increased or decreased level of risk were text box at right) direct senior psychologists well-supported, especially if the actuarial data to ensure that the evaluations adequately suggested a different risk category (i.e. clinical summarize file records and to point out over-ride). data inconsistencies or discrepancies in the 6. The case conceptualization was objective, records, changes in the inmate’s testimony, non-biased and neutral, and was written with and contradictions in the records. From the appropriate audience in mind (i.e. minimal our discussions with two of the four senior clinical jargon). psychologists, however, we learned that when 7. Cultural issues were addressed and integrated the senior psychologists conduct reviews of throughout the report when appropriate. their subordinates’ evaluations, they do not review the source documents that the staff psychologists use. One senior psychologist indicated that he corroborates limited information about the inmate in the parole board’s Lifer Scheduling and Tracking System. Nevertheless, we question how a senior psychologist can effectively identify factual mistakes and omissions of information in an evaluation if the senior psychologist does not refer to the source documents from the inmate’s central file. For example, if the inmate’s records contain a discrepancy, such as conflicting information about the inmate’s life crime, and that discrepancy is not discussed in the psychologist’s evaluation, the senior psychologist would have no independent means of noticing that discrepancy. If copies of the underlying documents were available, the senior psychologists could, for example, review the Probation Officer’s Report to ensure that the evaluation accurately described the inmate’s life crime and the number of victims, or consult rules violation reports and custodial counselors’ notes to ensure that the psychologist properly accounted for the inmate’s behavior while in prison. State of California • July 2010 Page 21 Furthermore, the parole board could also strengthen its process by developing a checklist of specific procedures for senior psychologists’ use in verifying statements and conclusions in the evaluations. An example of a relatively simple procedure would be ensuring that the inmate’s name is spelled correctly and is used consistently throughout the evaluation and confirming that no other inmate’s name is inappropriately included in the evaluation. The senior psychologists could check off the task when complete; the checklist itself could serve as evidence of the review if particular components of the evaluation were later challenged. Certain types of factual errors could be prevented if the senior psychologist reviewed source documentation as part of the review process. For example, we reviewed a parole board tracking report listing factual errors reported by commissioners during suitability hearings in 2009. In one case listed on the report, the commissioner recorded that an inmate had four administrative errors in his psychological evaluation during his suitability hearing. Two of the four errors may have been preventable if the senior psychologist compared the statements in the evaluation to source documentation. One of the preventable errors involved the use of an incorrect inmate name in the psychological evaluation, and the other was a reference in the evaluation to a crime that was not otherwise on the inmate’s record. Furthermore, our review of 38 evaluations spread across four prisons discovered a substantial error in an inmate’s evaluation in which the psychologist incorrectly labeled on the first page of the evaluation the inmate’s life crime as second-degree murder when it should have been first-degree murder. We found this error by comparing the life crime in the evaluation to the inmate’s Abstract of Judgment (located in the inmate’s central file). It is logical to assume that a senior psychologist reviewing this evaluation would be able to replicate our discovery of this error if he or she were independently comparing the evaluation to the underlying source documents used by the staff psychologist who prepared the evaluation. The parole board does not have a time benchmark for reviewing psychological evaluations nor does it require senior psychologists to account for their review time The chief psychologist does not specify a minimum length of time that a senior psychologist should expect to spend reviewing an evaluation nor does the chief psychologist require senior psychologists to account for their time by case or activity. Consequently, the chief psychologist lacks the information needed for determining whether his staff spends an adequate amount of time on their reviews. The chief psychologist primarily oversees the senior psychologists’ work by reviewing their monthly reports of research data and their weekly evaluation scheduling and tracking reports; holding monthly in-person status meetings and weekly conference calls; and conducting additional discussions as needed. We learned from our conversations with the chief psychologist and two of the four senior psychologists that, using the parole board’s framework and guidance, the senior psychologists each have the flexibility to determine the approach they will use in conducting their review and the time they spend on it. The two senior psychologists estimated that they review between 65 and 80 evaluations per month, and although one of them estimated that it takes an average of one hour to review a full draft evaluation, the other estimated that it takes between one and three hours. Bureau of Audits and Investigations, Office of the Inspector General Page 22 However, absent a benchmark or standard against which to compare the actual time spent reviewing each evaluation, the chief psychologist cannot judge whether the senior psychologists are spending too little or too much time in this process. Comparing the actual time spent reviewing evaluations with a benchmark can provide the chief psychologist with more assurance that the evaluations are thoroughly reviewed. Furthermore, given that senior psychologists are not centrally located in an office and instead work most of their time at home, it is important that the parole board have more accountability built into its process for reviewing psychological evaluations. Having to verify a list of specific information and having specified review-time expectations could make the process more consistent and more thorough for all four reviewers. In addition to reviewing evaluations, senior psychologists also perform a number of other activities, such as visiting prisons to observe psychologists’ face-to-face inmate interviews and to meet with prison staff, conducting performance appraisals and other supervisory tasks, participating in conference calls with management and staff, participating in training activities, and performing secondary reviews of evaluations when directed by the chief psychologist. This breadth and variety of supervisory responsibilities further heightens the need for a time-based monitoring system. When we discussed these issues with the parole board’s chief psychologist, he said that the [parole board’s] existing practice of supervisory review of psychological evaluations conducted by licensed psychologists far exceeds the standard of practice in the field. In fact, most licensed psychologists, including those employed by the State, receive little or no clinical supervision of evaluation reports—let alone have each report subjected to supervisory review. In addition, the chief psychologist said that he “believes that [the parole board’s] psychologists receive more individualized supervisory input and oversight of their written work than most psychologists in the state.” The chief psychologist also noted that the individualized nature of psychological evaluations requires that senior psychologists adopt a flexible approach when reviewing evaluations—even when doing so within a fairly structured format. Senior psychologists do not arbitrarily determine the length of their review time. Instead, their review time is determined by the complexity of the case and the supervisory needs of the psychologist. He added the following: It is my position that it is the primary responsibility of the psychologist, and not the senior psychologist, to assure that evaluation findings are based upon accurate reporting of facts. This responsibility is codified in State licensing requirements, professional ethics codes, and specialty guidelines of forensic psychologists. We agree with the chief psychologist’s position that psychologists are responsible for the quality of their work. However, we also believe that as supervisors, senior psychologists are responsible for holding their staff accountable for their work and ensuring that factual errors such as those discussed previously are identified and corrected as soon as possible before the inmate’s suitability hearing. State of California • July 2010 Page 23 Recommendations To strengthen the process for reviewing psychological evaluations, the parole board should take the following actions: • Ensure that senior psychologists review copies of source documents from the inmate’s central file when reviewing psychological evaluations for parole suitability hearings. This source documentation review will increase the integrity of the review process by helping to identify misstatements or inaccuracies contained within the evaluations. • Develop a review checklist for senior psychologists to follow when conducting reviews of psychological evaluations. This checklist would aid in fact-checking the statements and conclusions in the psychological evaluations and ensure consistency and thoroughness among reviewers. • Develop a time benchmark for reviewing psychological evaluations to ensure that reviewers spend at least the minimum amount of time conducting their reviews. • Require senior psychologists to report to the chief psychologist their time spent reviewing draft evaluations. This information would help the chief psychologist track productivity and manage workload levels. Bureau of Audits and Investigations, Office of the Inspector General Page 24 Finding 3 The Board of Parole Hearings Failed to Provide All Required Training to Its Commissioners, Deputy Commissioners, and Senior Psychologists Penal Code section 5075.6 requires commissioners and deputy commissioners who conduct parole suitability hearings to receive 40 hours of annual training. Government Code section 19995.4 requires senior psychologists (because they are supervisors) to receive 80 hours of supervisory training within 12 months of their appointment. However, the Board of Parole Hearings (parole board) failed to provide most of these employees with the required training. Commissioners and deputy commissioners who are not well trained may not be fully prepared to perform their duties and responsibilities, which could affect hearing decisions. If commissioners and deputy commissioners make mistakes because they were not trained in certain areas, those mistakes could put the state at risk of legal challenges and erode stakeholders’ confidence in the parole suitability hearing process. Moreover, the parole board did not provide all four of its senior psychologists with the required supervisory training. As a result, the senior psychologists may not be aware of certain techniques for supervising and teaching employees and enforcing labor policies. Commissioners and deputy commissioners did not always receive the required number of annual training hours Penal Code section 5075.6 requires commissioners and deputy commissioners who conduct hearings for the purpose of considering the parole suitability of inmates to receive—within 60 days of their appointment and annually thereafter—a minimum of 40 hours of training. The law specifies this training to include such topics as the treatment and training programs provided to inmates, parole services, commissioner duties and responsibilities, and the laws and regulations applicable to conducting parole hearings. The parole board noted that it established a new commissioner training program in July 2008. The duration of the program is five weeks, with the first three weeks consisting of classroom instructional training. The parole board maintains a variety of instructional materials, covering such topics as basic introduction to the parole board, ethics, applicable case law, prison resources available to commissioners such as inmate central file and psychological evaluations, familiarization with the Lifer Scheduling and Tracking System, mock hearings, and decision writing. The parole board also provides commissioners with administrative training on such topics as completing time sheets and defensive driver training. The final two weeks of the training program allows commissioners to observe parole suitability hearings to better understand the process. We found that the parole board provided all seven of its current commissioners appointed during 2008 and 2009 with 40 hours of training within 60 days of their appointment. However, the parole board failed to provide any of its current commissioners who were appointed before 2009 with 40 hours of annual training thereafter as required by law. Specifically, according to the parole board’s In-Service Training reports (IST reports), the parole board failed to provide all three commissioners appointed before 2008 with at least 40 annual training hours in 2008 and all seven commissioners State of California • July 2010 Page 25 appointed before 2009 with at least 40 annual training hours in 2009. In 2008 and 2009, the parole board provided these commissioners with an average of 19 and 10 hours of training each year, respectively. In addition, we reviewed training records for deputy commissioners who were panel members for suitability hearings in 2009. According to the parole board’s IST reports, it provided only three of 16 deputy commissioners (19 percent) whom it hired in 2009 with 40 hours of training within 60 days of their appointment. The average number of training hours received by the remaining 13 deputy commissioners was only 28. Furthermore, the IST reports show that the parole board provided only two of 70 deputy commissioners (3 percent) whom it hired prior to 2009 with 40 hours of annual training. The average number of training hours received by the remaining 68 deputy commissioners was only 15. If not trained sufficiently, commissioners and deputy commissioners—even those who have served in the position for a long period of time—may not be as well-equipped or prepared as they could be to make the best parole suitability decisions at hearings. Commissioners and deputy commissioners who do not receive training may not fully understand certain aspects of their duties and responsibilities, which could affect hearing decisions. If commissioners and deputy commissioners make mistakes or issue poorly articulated decisions because they were not trained in certain areas, those actions could put the state at risk of legal challenges and erode stakeholders’ confidence in the parole suitability hearing process. When we brought this matter to the attention of the parole board’s chief deputy of program operations, he said that the parole board recognized that training deficiencies existed in 2009 and had drafted a new training plan. He explained that the new plan addresses training requirements and describes procedures to bring the parole board staff, including commissioners, into compliance with statutes and regulations governing training. He also noted that the parole board has a new training manager who will report quarterly to executive management if there are deficiencies in training. As stated earlier, the Office of the Inspector General’s Bureau of Independent Review (BIR) is currently collaborating with the parole board’s executive staff to develop a transparent training curriculum for all commissioners and deputy commissioners. The BIR intends to identify judicial training topics relevant to the parole board’s work and complementary to the parole board’s ongoing internal training. The training curriculum is scheduled to be completed by December 2010. Senior psychologists did not receive the required number of supervisory training hours within one year of their appointment Government Code 19995.4 requires supervisory employees to receive a minimum of 80 hours of training—40 hours performed by a qualified instructor—within 12 months of their appointment. The law requires the supervisory training to address such topics as the role of the supervisor, techniques of supervision, planning, organizing, staffing and controlling, performance standards, performance appraisal, affirmative action, discipline, labor relations, employment law relating to persons with disabilities, and grievances. The additional 40 hours may be provided as on-the-job training by a qualified higher level supervisor or manager. The Parole Board’s four senior psychologists supervise staff psychologists. However, the parole Bureau of Audits and Investigations, Office of the Inspector General Page 26 board did not provide the 80 hours of supervisor training to any of those senior psychologists within 12 months of their appointment, as required. One senior psychologist, whom the parole board hired in October 2006, reported that he received supervisory training in February 2009, about 15 months after the 12-month period expired. However, these training hours were not listed in the senior psychologist’s IST roster. The chief deputy of program operations acknowledged that the parole board did not provide the other three supervisors with the required training. He explained that this is partly because the parole board focused senior psychologists on completing their high-volume workload. Nevertheless, without proper supervisory training, there is greater risk that staff could be unaware of employment-related matters such as performance standards and processes related to discipline and grievances. Recommendation To ensure that its employees receive the required number of training hours, the parole board should regularly monitor training activities and the number of training hours it provides to all staff. The parole board should then make the necessary scheduling adjustments for those individuals who have not received enough hours. State of California • July 2010 Page 27 California Department of Corrections and Rehabilitation’s response to the special report Bureau of Audits and Investigations, Office of the Inspector General (page 1 of 1) Page 28 SPECIAL REPORT SPECIAL REPORT INMATE CELL PHONE USE ENDANGERS PRISON SECURITY AND PUBLIC SAFETY The Board of Parole hearings: psychological evaluations and mandatory Training Requirements OFFICE OF THE INSPECTOR GENERAL OFFICE OF THE INSPECTOR GENERAL DAVID R. SHAW INSPECTOR GENERAL David R. Shaw STATE OF CALIFORNIA MAY 2009 INSPECTOR GENERAL Jerry Twomey Chief assistant inspector general William Shepherd Deputy Inspector General, In-Charge Bryan Beyer Deputy Inspector General, Senior Christina Animo Deputy Inspector General Ryan Baer Deputy Inspector General STATE OF CALIFORNIA July 2010 WWW.OIG.CA.GOV