OBSERVATIONS REGARDING THE POLICY RESEARCH ASSOCIATES FINAL REPORT
(Excerpted from REPORT OF THE BELLEVUE HOSPITAL CENTER OUTPATIENT COMMITMENT PILOT PROGRAM prepared by:Howard Telson, M.D. Richard Glickstein, Esq. Manuel Trujillo, M.D., MARCH 1, 1999 Bellevue Hospital Center, Department of Psychiatry, 462 First Avenue, New York, NY 10016)
PRA released its Final Report on the OCP pilot on December 4, 1998. It contains PRA’s findings from its client outcome study, program implementation evaluation, and patient, family and provider interviews and focus groups.
Because of its role in developing and implementing the OCP, Bellevue is able to evaluate the PRA study from a unique perspective. Bellevue was responsible for ensuring an adequate number of referrals, and, ultimately, of eligible and appropriate patients, so that PRA’s stated study size requirements for statistical significance could be met. The CT was also responsible for ensuring that all appropriate patients were referred to PRA for participation in the study. The CT also ensured that, during their time in the research, patients in the control group received the same priority access to consultation and services as did patients with court orders.
The following analysis sets forth Bellevue’s view of the PRA report. These observations highlight specific areas of agreement and disagreement between Bellevue’s clinical and implementation experiences and PRA’s research analysis.
AREAS OF SUBSTANTIAL AGREEMENT BETWEEN BELLEVUE AND PRA
1. The PRA report accurately states that “the terms and conditions for successful compliance under OCP were negotiated between providers and program clients on an ongoing basis, proving the court orders to be flexible in interpretation.”
PRA found that the court orders provided a structure around which providers and patients constantly negotiated the terms of treatment. This is consistent with the intent of MHL Section 9.61, which allows the court to order categories of services, and contemplates that providers will attempt to solicit compliance and monitor noncompliance and dangerousness. This is also consistent with Bellevue’s observation that outpatient commitment orders often engage severely mentally ill individuals with histories of noncompliance in a dialogue about the need for treatment when they are still outpatients, and therefore assist in soliciting patient compliance in the community and preventing relapse.
2. PRA’s conclusion that the CT’s service coordination and resource mobilization functions “seemed to make a substantial positive difference in the post- discharge experiences of both the experimental and control groups” is important.
The PRA report states that both groups of patients in the research study showed improved clinical outcomes because a statistically significant lower proportion of patients in both groups were hospitalized in the 11 months of follow-up as compared to the year preceding the target admission. Both groups had access to a wide array of community services, which were coordinated through the CT. Bellevue’s clinical experience has been that a wide array of services and their effective coordination are necessary to establish good patient outcomes.
Bellevue has also found that in many cases the court order and the authority of the legal system contribute to positive patient outcomes. To the extent that patients in the control group and their providers felt that the CT had access to and imposed the authority of the legal system on them, control patients may have benefitted from the outpatient commitment pilot in ways similar to those of patients with court orders. In analyzing the research findings, it is important to acknowledge that patients in the control group received the maximum amount of resource mobilization as well as monitoring in the community that was legally possible.
3. PRA reached the important conclusion that “under the auspices of a pilot outpatient commitment program the Bellevue Coordinating Team was able to mount an effective service coordination and resource mobilization effort that proved very popular with community providers.”
It is indeed Bellevue’s experience that the CT is popular with providers because it operates under the auspices of a pilot outpatient commitment program that has the legal authority to closely monitor patients, to attempt to ensure that they have access to treatment and, to the extent possible to solicit their compliance. Service coordination involves treatment planning and the sharing of medical information between multiple programs and agencies. While case management programs typically attempt to perform this function, the OCP is especially effective because of its legal authority.
More importantly, the CT is able to “mobilize” resources because providers feel empowered by the legal system to work with patients who have historically been noncompliant and service refusing. Sometimes even the most assertive outreach is ineffective in the face of severe symptoms of mental illness; outpatient commitment orders allow providers to maintain engagement efforts without feeling that they are intruding inappropriately on a refusing patient’s privacy.
Since Bellevue could not insure that all patients participating in the study would have court orders, it was essential that the CT made its consultation and coordination services available to all providers who agreed to participate. These services have been appreciated, and it appears that they lead to improved outcomes and quality of care.
The purpose of the study was to determine the degree of added benefit provided by outpatient commitment. In this regard it is worth noting that the PRA report shows trends which suggest that the CT has even greater value when it has the authority of a court order. The report indicates that when patients had court orders 77% of their providers were very satisfied with the support of the CT vs. 59% of providers of control patients. The report also indicates that 71% of providers reported that OCP helped their work with court ordered patients vs. 56% of providers of control patients. Fewer than 10% of providers for both groups of patients felt that OCP hindered their work with the client.
It is also important to note that providers of patients in the control group to some extent felt that the CT had access to and imposed the authority of the legal system both on them and their patients. The degree to which this phenomenon affected the PRA study is not explored, and may be significant. If there had been two coordinating teams, one with access to outpatient commitment orders, and one only doing hospital discharge planning and service coordination, this methodologic flaw would have been eliminated, and the specific effect of the court order might have been clearer.
AREAS OF PARTIAL AGREEMENT BETWEEN BELLEVUE AND PRA
4. PRA’s conclusion that a comprehensive outpatient commitment program was difficult to implement is generally consistent with Bellevue’s experience.
Bellevue had anticipated difficulties in implementing a pilot outpatient commitment program in New York City. Given the amount of time it took for outpatient commitment legislation to be passed, and the many sensitive clinical, legal and ethical issues that had to be resolved, it is clear that program implementation was a process that had to unfold through time and experience.
The PRA report correctly states that the lack of a MHL Section 9.61 hospital transport procedure during the research period was one factor that made the OCP different from the Legislature’s intended model. PRA also concludes that in extrapolating from evidence, it must be remembered that the program tested was not a “fully executed, clinicians-working-with-the-law-enforcement officers” outpatient commitment program. PRA also indicates that this contributed to its difficulty in measuring “any leverage or negative consequences of coercion that may have been added by MHL Section 9.61 over existing emergency intervention procedures.”
In spite of the time it took to fully implement the 9.61 enforcement mechanisms, however, the Bellevue OCP has functioned effectively. The pilot has shown that many government and provider agencies can collaborate to help patients. The program has credibility in the eyes of providers and has brought leverage to bear on the problems of noncompliance. Furthermore, in its assurance of due process and access to MHLS, the program is attuned to patients’ rights and the balance that must be struck between autonomy and social control in the area of mental health treatment.
PRA is not completely accurate in indicating that the program was difficult to implement because “community treatment resources (especially for co-occurring mental disorder and substance abuse) . . . proved insufficient to the demand.” While the availability of different specific resources waxed and waned over time, the OCP was fortunate to have had access to a wide range of services throughout New York City during the whole pilot. Any difficulties regarding access were experienced by the entire mental health system, and did not only affect OCP. Furthermore, while it is true that resources for mentally ill chemical abusers (MICA’s) are insufficient, it is important to recognize that the problem is not just one of the numbers. Most programs for MICA’s require patients to express a desire to stop using substances, and such services were as available to OCP patients as any others. However, many OCP patients express neither a desire nor a willingness to achieve abstinence. Developing services for MICA’s who do not want substance abuse treatment presents many complex challenges to our treatment system and our entire society, which are far from being solved.
5. Bellevue agrees with PRA’s conclusion that OCP court procedures are often informal, but interprets the legal process of outpatient commitment differently.
PRA documents the fact that initial 9.61 hearings became routine after the first year of the pilot. Physicians most often presented their case to the court with affidavits, and patients most often consented to the orders. Renewals also became informal, and patients very often communicated their consent to their MHLS attorney on the telephone.
It is possible that a different procedure for granting orders might alter the effect of outpatient commitment. PRA indicates that, based on their interviews, patients who received court orders appeared to understand their outpatient commitment status despite abbreviated court hearings. Furthermore, all patients who contested the hospital’s request for an outpatient commitment order or renewal had the right to a hearing. While there indeed were very few hearings in relation to the total number of orders granted, PRA hardly mentions the hearings that did take place.
PRA states that the “easy informality” of the hearings served to “mute perception of the essentially coercive nature of the intent behind such proceedings.” Yet this view fails to appreciate the degree to which the success of outpatient commitment depends on enlisting the patient’s cooperation in developing a treatment plan. MHL Section 9.61 requires that the patient, and, at the patient’s request, a person significant to the patient, be given an opportunity to actively participate in the development of the treatment plan.
The hearings can perhaps be better understood as the formalization of the outpatient commitment process, which is a mandate for the patient to comply with treatment in order to prevent deterioration and relapse. When PRA questions “the putative symbolic or moral power that may be ascribed to the legal vehicle (the court hearing)” it does not appreciate that the authority of outpatient commitment is based not only on the hearing, but on the entire process that comes with it. The orders provided a structure around which treatment issues were continuously negotiated between patients and providers.
It is also possible that the lack of 9.61 enforcement procedures during the research period affected the nature of the hearings. As PRA noted, during the first hearing the testifying psychiatrist had difficulty describing how the OCP plan differed from previous discharge plans for the patient. It is possible that the testimony would have been quite different had the Medication Guidelines and the 9.61 hospital transport procedure been in place. It is also likely that a statute with less sensitivity to patient choice and more of an emphasis on coercion than MHL Section 9.61 would result in more formal and contested hearings.
AREAS OF DISAGREEMENT BETWEEN BELLEVUE AND PRA
6. PRA’s conclusion that “the court order itself had no discernible added value in producing better outcomes” must be questioned in light of the limitations of the study. Further, PRA’s conclusion is inconsistent with some of its study’s own results, as well as much of the clinical experience of Bellevue and community providers.
A. The PRA report acknowledges that “the modest size of our study group” is a “limit on our ability to draw wide-ranging conclusions.
The report also states that an analysis of “for example, which subjects if any might have been more likely to benefit from a court order is not possible, because the size of the subgroups becomes too small to reach acceptable levels of statistical rigor.”
B. The “experimental” and “control” conditions were much more similar than had originally been contemplated.
This was the case both because there was no MHL Section 9.61 transport procedure in place, and because of the likelihood that some of the OCP’s legal authority affected the control group. Furthermore, the PRA report describes how much the difference between the conditions was misunderstood by patients and providers, especially before 1997, when the paperwork was modified to address the problem. The similarities of the two conditions limits the meaning of the comparison.
C. PRA minimizes its own statistical trends which suggest that outpatient commitment did reduce hospital days considerably.
The research showed that patients in the court-ordered group spent a median of 43 days in the hospital during the study year, while patients in the control group spent a median of 101 days in the hospital. PRA states that there is “no statistically significant difference” between these numbers, but fails to provide an adequate analysis of this trend. There is also no discussion of the trend showing that non- substance abusing psychotic patients were re-hospitalized far less if they had outpatient commitment orders (25%) than if they were in the control group (45%).
D. PRA does not adequately examine the “trend for controls to be transferred to state hospitals more frequently than the court-ordered group.”
PRA explains that “nothing in the baseline or follow-up measures of symptoms and functioning, or the ethnographic study of OCP, or indeed in the modalities of case management to which they were assigned, suggests that differences in the clinical picture or service packages of the two groups might explain the much longer hospital stays and greater likelihood of state transfers of the controls.” PRA hypothesizes that the court order may have permitted the CT “to act in such ways as to reduce both the number rehospitalized and their likelihood of being transferred to a state psychiatric facility,” but does not provide evidence for this conclusion. Furthermore, by speculating that the difference was only due to the CT’s behavior, PRA ignores the very important possibility that patients with outpatient commitment orders decompensate less severely in the community than control patients. The way in which court orders affect both provider behavior and patients’ clinical courses are questions for future research.
E. The randomization was not completely successful.
The report indicates that the court-ordered group of psychotic patients had a statistically significant higher percentage of substance abusers than psychotic patients in the control group. Therefore, although otherwise similar, the two groups of patients in the study are not comparable.
This is particularly notable since PRA found that substance abusers had a statistically significant greater rehospitalization rate than non substance abusers in both court-ordered and control patients.
PRA also found that patients with court orders were significantly less likely than controls to have been homeless prior to the referring hospitalization. This demographic difference may have had implications regarding differences in treatment planning between court-ordered and control patients.
F. The PRA report does not evaluate patients in relation to their own histories.
This is especially important because, as noted above, the two groups were different in important respects. PRA did not measure and evaluate the number of days patients spent in the hospital in the 18 months prior to the study. PRA also did not assess which services patients already had in place, or which services patients had failed to comply with, prior to inclusion in the study. This information is crucial in assessing the specific effectiveness of outpatient commitment as compared to community services alone, and without such an analysis a comparison of two non-equivalent groups has very limited value.
G. PRA’s findings only reflect a proportion of program participants.
Court-ordered patients completed five or eleven month follow-up interviews 77% of the time, and control patients completed them 66% of the time. Many patients were therefore not studied at all. It is likely that these patients’ attitudes and responses to treatment are different from those of the patients whom PRA studied.
Further, PRA states that “we were somewhat more able to locate and interview the experimental subjects, who less often moved out of the New York City area and were easier to locate.” The reason for this phenomenon is not explored, although PRA notes that “a logistical regression analysis showed no biasing across the two study groups as a result of differential attrition,”
H. The PRA study is limited to patients who were determined to be capable of consenting to participate in the research, and who then consented.
Approximately 15% of patients who were approached by PRA refused to consent to participate in the study. These patients all were brought to court, and they all received outpatient commitment orders and were followed by the CT. The report states that PRA had limited knowledge of these patients, but then goes on to say that research refusers “were no more uncooperative, resistant or dangerous” than patients who consented to the research. In the judgment of the CT this conclusion is questionable.
It is also worth noting that PRA judged a small number of patients to be incapable of consenting to the research, or not appropriate for the study.
I. PRA does not discuss the effect on its conclusions of limitations in its own ability to perform data collection.
The report indicates that “our research team determined some subjects to be too impaired to do interviews.” While roughly 10% of patients were included in this group, the report does not analyze the nature or the effect of the impairment. The report also states that “some subjects refused to do the interviews . . . even after agreeing to do them at the Baseline Interview.” Although outpatient commitment was specifically developed for patients who consent and subsequently refuse, there is no further discussion of how these patients might differ from patients who agreed to all of the interviews.
J. PRA does not fully consider clinicians’ views of the effectiveness of outpatient commitment.
Many providers throughout the community have consistently stated to the CT that outpatient commitment orders promote patient compliance with treatment. The PRA report states that many providers believed that the court order was potentially helpful in improving patient compliance. PRA also notes that once the research recruitment ended, “Bellevue clinicians and community providers are not as reluctant to do the extra work required by the OCP now that they are certain that the patient will be going to court” and not be randomized into the control group.
In a number of instances, however, PRA minimizes the positive views of providers regarding outpatient commitment. PRA incorrectly describes the “high demand for renewals” among providers as being exclusively due to the support and assistance received by the OCP Coordinating Team. In fact, providers consistently described the value of the court orders themselves when requesting renewals.
PRA also understates the extent to which psychiatrists believe that outpatient medication orders are appropriate and valuable. The PRA report notes that the CT encouraged psychiatrists to request that medication be included in outpatient commitment petitions and assisted in the preparation of such medication requests. The PRA report minimizes the belief among many inpatient psychiatrists that medication noncompliance was the heart of the problem for many patients, and that an outpatient medication order might improve medication compliance in the community. Although approximately two-thirds of initial requests for outpatient commitment included medication as a category of service, the PRA report only discusses reasons that the psychiatrists do not include orders, not why they do.
7. The PRA report is confused in its discussion of the relationship between violence and the OCP.
The PRA report states that the OCP “did not target patients perceived as being at high risk of violence in the community.” This is only accurate to the extent that it refers to patients who remain at high risk even while they are receiving treatment in the hospital. Such patients, however, were never defined as the target population for outpatient commitment. This is clear in Dr. Geller’s Clinical Guidelines for the Use of Involuntary Outpatient Treatment,and was the position of all agencies responsible for implementing the New York pilot. Indeed, it would have been unethical to allow mentally ill individuals presently at high risk for violence to be discharged to the community. It is also unclear how many such patients would be capable of providing informed consent for a randomized research study in which only half received outpatient commitment orders.
PRA seems to confuse current with past dangerousness. PRA is incorrect in stating that a “standard of low perceived dangerousness seems to have been consistently applied for all OCP referrals.” Patients were required to have a history of two involuntary hospitalizations in order to even be eligible for OCP, and these require dangerousness to self or others. A high proportion of patients referred to the OCP had histories of agitation, threatening behavior and fights with friends, relatives, providers or strangers. Patients were never excluded from the OCP due to prior histories of violence or incarceration. The majority of patients had histories of repetitive dangerousness, but of the kind which were due to treatment noncompliance and which resulted in psychiatric hospitalization rather than arrest. Many patients who were known to be at high risk of dangerousness, when untreated, in the community, were referred to and followed by the OCP.
For similar reasons PRA is not accurate in its discussion of the CT’s view of Larry Hogue. PRA quotes the OCP Director as discussing Larry’s Hogue’s case over the course of the pilot, and stating that “Mr. Hogue would not have been approved for referral to court.” However, PRA does not elaborate fully on the way in which the case was used for instruction. In fact, Mr. Hogue was often discussed as an example of a patient who had done well in the community for long periods, at different times, on a conditional release legal status. He was presented as an example of a mentally ill individual who was dangerous to others when noncompliant with treatment and intoxicated on cocaine, but who was not dangerous when complying with treatment and abstinent. Therefore, Mr. Hogue was often discussed in the context of explaining the spectrum of legal interventions that might be used to assist dangerous mentally ill individuals to accept treatment and live safely in the community.
8. The PRA report contains a number of other inaccuracies.
First, PRA misrepresents the message that was given to patients regarding the relationship between outpatient commitment and clinical services. PRA states that “it was not uncommon for patients to labor under the (usually uncorrected) impression that renewing the court order would secure them continued enhanced services, and that, without it, those same services would be withdrawn.” This was not the case; patients were told clearly by the CT that access to services was unrelated to the outpatient commitment order. The message was the same at the time of renewing orders, and whenever patients contacted the CT with questions on this issue.
PRA also incorrectly states that “basically, no one is put forward for a court hearing by the Coordinating Team who has not already agreed to participate in the OCP.” Patients generally are required to agree to participate in clinical services before they can be accepted, but patients were by no means required to accept the order. The CT requested outpatient commitment orders in every case in which a patient had a community treatment plan in place but refused to consent to the order.
The PRA report is also inaccurate in its discussion of the case of Billy Boggs. PRA correctly notes that she was a gravely disabled mentally ill homeless woman who became a focus of public attention. However, the PRA report states that she “successfully fought efforts in the early 1980's by city officials to have her hospitalized. In fact, Billie Boggs was hospitalized at Bellevue on the Project HELP unit soon after it was established in 1987. Most importantly, the clinical staff at Bellevue successfully secured a court order to retain Billie Boggs in the hospital because of the continued risks she posed to herself as a result of her mental illness. It was only the hospital’s petition to the court to administer medication over the patient’s objection that was denied.
[ Back to NY-TAC Index ] [ Op-eds ] [ NY Laws ] [ Issues ] [ Join NY-TAC ]
The information on /newyork is provided as a public service by the NY Treatment Advocacy Coalition. For more information, to join, or to make a donation, write to NY Treatment Advocacy Coalition, Suite 4B, 250 West 27th St., New York, NY 10001 or call (212) 366-0527. To support our efforts, make checks payable to Treatment Advocacy Center. To become a free member and receive updates, please send your name, (organization's name, if any) address, phone, fax, and e-mail address to the above address, along with the following signed statement: "I HAVE RECEIVED INFORMATION ABOUT THE NY TREATMENT ADVOCACY COALITION, SUPPORT IT'S MISSION, WANT TO RECEIVE UPDATES AND BE LISTED AS A MEMBER AND SUPPORTER." Unfortunately, requests for free memberships without this information and statement can not be honored. Contents of all material on the Coalition's web site is copyrighted and rights are reserved by the NY Treatment Advocacy Coalition unless otherwise indicated. However content may be reproduced, downloaded, disseminated, or transferred by nonprofit organizations that support our mission for educational purposes if correct attribution is made to the NY Treatment Advocacy Coalition. Please feel free to call with questions on neurobiological disorders, treatment laws or the benefits of medication compliance at 703.294.6001 or 212.366.0527. Send questions via e-mail to stanleyj@psychlaws.org. Media inquiries to press@treatmentadvocacycenter.org. Technical comments on the web site (www.psychlaws.org) can be sent to Webmaster@psychlaws.org.