Limitations of Policy Research Associates report on Bellevue Assisted Outpatient Treatment Program
The legislation enacting the Bellevue Pilot Project included a requirement to evaluate the project. Unfortunately NYC contracted with Policy Research Associates (PRA, Henry Steadman) to conduct the research. Policy Research Associates has a history of conducting highly criticized research. The common theme seems to be a bias against Assisted interventions. (See NATIONAL REVIEW July 20, 1998, pp. 36-37)
Certainly biases against Assisted Treatment seem to have been included in the report, and given great play, while information supporting Assisted Treatment was excluded or minimized. In addition, a lack of specificity in the report prevents re-evaluating the data to gain greater learning.
However, it is the position of the New York Treatment Advocacy Coalition that at least some of the shortcomings of the report prepared by Dr. Steadman and Policy Research Associates may in fact have been the result of flawed legislation and/or implementation, and not all are due to flawed or biased design and reporting.
1. The non-compliant were excluded.
The program was supposed to test whether or not outpatient assisted treatment would help individuals who are non-compliant with treatment. But 100% of the non-compliant were excluded from the program. In order to participate in the PRA study, the participants had to be compliant. They had to agree to be placed in outpatient assisted treatment or a control group. Hence, those who would not agree to go into treatment (the non compliant, those the program was designed for) were specifically excluded from the study.
2. Individuals who lacked capacity or competency were excluded.
Outpatient Assisted Treatment is especially useful to those who because of their illness, may be too psychotic or out of touch with reality (“lack competency” “lack capacity”) to help themselves. 100% of these individuals were excluded from the PRA study because they were unable to give the “informed consent” needed to participate in the research.
3. The violent were specifically excluded
Outpatient Assisted Treatment is especially useful to those with a history of violence associated with non-compliance. 100% of these individuals were excluded from the PRA study. The New York Treatment Advocacy Coalition was extremely surprised that according to Dr. Steadman and Policy Research Associates, the violent were excluded from both the control and study groups. If true, this was in contravention to the stated purpose of the legislation. The Policy Research Associates report talks about the long time it took to enact this legislation and says:
(Page 17) “When the focus shifted to overt threats to public safety the legislative effort garnered sufficient support. The type of individual held up as evidence for the necessity of such a law was Larry Hogue: a homeless veteran who was both actively psychotic and a crack cocaine addict, and was alleged to terrorize this Upper Westside neighborhood of Manhattan. To the mounting frustration of neighborhood organizers, neither police nor mental health officials seemed able to do anything to contain or control his behavior. OPC was pushed with the State Legislature as a public safety measure targeted to prevent dangerous behavior in the community.
“Ironically, Mr. Hogue is highly unlikely to ever have been eligible for OPC. Early in the program’s start-up phase, in response to the understandable concerns of Bellevue staff concerning liability, and in many clarifying documents since (I have not seen these-dj) Dr. Howard Telson, Program Director, made a point of noting that owing to his dangerousness, Mr. Hogue would not have been approved for referral to court. This standard of low perceived dangerousness seems to have been consistently applied for all OCP referrals throughout this phase of OCP.
“…(Page 50) At the outset of the OCP, both the Program Director when engaging Bellevue unit staff and community providers and other senior Bellevue administrative staff who participated in our project’s review by the NYU IRB explicitly state that violent people would not be included in this program (emphasis added).
The report suggests the violent were excluded to get providers to agree to participate. If so, this is another unfortunate consequence of artificially enhancing services in order to make it a study.
4. The subject group had a lot more MICA (“mentally ill chemical abuser”) patients than the control group.
The “MacArthur Study of Violence by people discharged from Acute Psychiatric Inpatient Facilities and by others in the same neighborhoods” co-authored by Dr. Steadman showed that “substance abuse raised the rate of violence among discharged psychiatric patients”. In the Bellevue study, the court-ordered group had a higher percentage of substance abusers than the control group. Hence, one would expect higher violence rates among the court-ordered group. Since the two groups showed equivalent violent rates, this showed court ordered treatment was effective. This point was not adequately addressed in the Policy Research Associates Report.
5. Everyone (controls and subjects) received “Enhanced Services”
Everyone wants individuals with neurobiological disorders to receive enhanced services. But most people who go through the “revolving door”, do not. Yet in this study, it was legislatively mandated that everyone receive enhanced services. The study therefore did not address the real world issue of, does outpatient commitment help those who don’t get enhanced services or those who become violent and psychotic without it? For that reason, the report could only conclude what we all know: that if you give people extra services, they do better.
Contrary to the assertions of Dr. Steadman, extensive research exists in many other localities (Washington, DC, North Carolina, Ohio, etc.). and all conclude Assisted Treatment has beneficial effect.
6. Doing the study, got in the way of what was supposed to be studied
The effect of OPC would have been much more significant, had Bellevue not been forced to attach a study to the program. The study got in the way in such profound ways that we are surprised an Institutional Review Board (IRB) approved it. Basically, no one was ‘involuntarily’ ordered to treatment. The only ones in the program were those who ‘agreed’ to be in it.
Because they were doing a study, the program had to get ‘informed consent’ from everyone. That is not something that would have had to happen had there been no study. Doing a study and getting informed consent meant consumers had to have competence and ‘agree’ to be randomly outpatient committed or assigned to a control group. This resulted in those who lacked capacity being excluded from the program. In addition, those with violent or criminal backgrounds would not be likely to agree to be in a study and were therefore excluded from the study (although not from OPC).
Of the remainder who had capacity and agreed to participate, if they were ‘agreeing’ to outpatient commitment, were they really being committed? When the subject cases came in front of the judges, the patients AND judges were confused as to why they were having a hearing to ‘order’ something the patient was ‘agreeing’ to. Judges and patients were even more confused when the patients and doctors came to them for renewals of the orders the patients were agreeing to. Why bother if the patient was agreeing?
7. Of the 567 people referred for inclusion in the study, only 152 actually participated.
Many of those who didn’t make it into the study eloped. (Again, some of this was an unfortunate consequence of making this a study and enhancing services. Enhancing services increased hospital stays as patients had to remain in the hospital while housing was found. Many decided to elope. Had their been no study requiring enhanced services, fewer patients would have had time to elope). Others went to state hospitals because the Unit Doctors found it faster to transfer them to the state then to wait for acceptance by the OCP which couldn’t accept them until they found enhanced services. These individuals may have been the Larry Hoags who usually go from city hospital to state hospital where they are declared ‘cured’ and released to streets). 13% of the patients enrolled in the study eloped after enrolement and were excluded.
8. No one was medicated over objection
Until the end of October 1998, there was no mechanism available to enforce compliance with treatment orders. The procedure that was finally put in place is too cumbersome to be effective and was never used.
Bellevue Hospital Staff Analysis of Limitations of PRA Report
(Posted 2/1999)
[ Back to NY-TAC Index ] [ Op-eds ] [ NY Laws ] [ Issues ] [ Join NY-TAC ]
The information on /newyork is provided as a public service by the NY Treatment Advocacy Coalition. For more information, to join, or to make a donation, write to NY Treatment Advocacy Coalition, Suite 4B, 250 West 27th St., New York, NY 10001 or call (212) 366-0527. To support our efforts, make checks payable to Treatment Advocacy Center. To become a free member and receive updates, please send your name, (organization's name, if any) address, phone, fax, and e-mail address to the above address, along with the following signed statement: "I HAVE RECEIVED INFORMATION ABOUT THE NY TREATMENT ADVOCACY COALITION, SUPPORT IT'S MISSION, WANT TO RECEIVE UPDATES AND BE LISTED AS A MEMBER AND SUPPORTER." Unfortunately, requests for free memberships without this information and statement can not be honored. Contents of all material on the Coalition's web site is copyrighted and rights are reserved by the NY Treatment Advocacy Coalition unless otherwise indicated. However content may be reproduced, downloaded, disseminated, or transferred by nonprofit organizations that support our mission for educational purposes if correct attribution is made to the NY Treatment Advocacy Coalition. Please feel free to call with questions on neurobiological disorders, treatment laws or the benefits of medication compliance at 703.294.6001 or 212.366.0527. Send questions via e-mail to stanleyj@psychlaws.org. Media inquiries to press@treatmentadvocacycenter.org. Technical comments on the web site (www.psychlaws.org) can be sent to Webmaster@psychlaws.org.