| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ALL ISSUES | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Regular Article |
Deceased.
Received October 18, 1999; revised December 13, 1999; accepted December 16, 1999. From the Department of Psychiatry, Cornell University Medical College, and the New School–Beth Israel Center for Clinical Training and Research, New York, New York. Address correspondence to Dr. Markowitz, 525 East 68th Street, Room 1322, New York, NY 10021; e-mail: jcmarko{at}mail.med.cornell.edu
 |
Abstract |
|---|
 Top Abstract IntroductionMETHODSRESULTSDISCUSSIONReferences |
|---|
Key Words: AIDS/HIV • Adherence Monitoring
|
Introduction |
|---|
|
TopAbstractIntroductionMETHODSRESULTSDISCUSSIONReferences |
|---|
Adherence monitoring measures fidelity to a manualized technique rather than the competence,1,2 much less excellence, of the therapist. Adherence to the therapeutic model provides a basic measure of the type of treatment the therapist has delivered, the bedrock on which other therapist factors, such as competence, can be measured. Adherence thus can offer the limited but important assurance that subjects receiving a treatment (e.g., interpersonal psychotherapy [IPT], cognitive-behavioral therapy [CBT]) are in fact getting the treatment advertised. This represents a significant improvement over studies where one has no idea what occurred in therapists' offices.
Why report what might be considered the background validation of a psychotherapy trial? An important reason is that no benchmark measure exists for psychotherapy adherence. Different instruments have been developed for differing therapies on different occasions. The number of published studies of psychotherapy adherence in outcome trials is small.2–10 When such instruments have appeared in treatment studies, they have generally been used once, without replication. Yet as Klein said of psychotherapy research in another context, "Replications are the soul of scientific advance."11
Perhaps the most highly developed measure, and a key example of adherence monitoring, is the Collaborative Study Psychotherapy Rating Scale (CSPRS) developed by Hollon12 for the landmark, multisite National Institute of Mental Health (NIMH) Treatment of Depression Collaborative Research Program (TDCRP).13,14 Hill et al.,15 using an elegant balanced incomplete block design, found that blinded raters using the CSPRS could differentiate among therapists delivering IPT, CBT, and clinical management (CM, accompanying medication or placebo) to 180 depressed subjects. We now report an initial attempt to replicate the results of this key study. We used an adaptation of the CSPRS to measure psychotherapy adherence in a treatment study of patients seropositive for the human immunodeficiency virus (HIV) who were suffering from depressive symptoms.
We hypothesized that 1) the minor changes required to adjust the CSPRS to the needs of the current study would not compromise its effectiveness in discriminating among therapies. Another hypothesis, crucial to the validity of outcome data from this research, was that 2) therapists would generally adhere to their prescribed therapies, and hence that each treatment cell would score highest on its own CSPRS subscale. Because the therapeutic alliance has been frequently correlated with treatment outcome,16 we also wondered whether 3) facilitative conditions (FC), a modality-nonspecific CSPRS subscale of therapist warmth and other supportive behaviors, might provide a predictive measure mediating treatment outcome for the study.
|
METHODS |
|---|
|
TopAbstractIntroductionMETHODSRESULTSDISCUSSIONReferences |
|---|
Treatments were delineated in manuals for each intervention. The CBT and IPT manuals modified the standard delivery of these treatments of depression to address the particular concerns of depressed HIV-positive patients.19 The supportive therapy used in SP and SWI, defined as non-IPT and non-CBT, was akin to the client-centered therapy of Carl Rogers21 with additional psychoeducational components about depression and HIV. Although SP therapy may have been somewhat hindered by proscription of interpersonal and cognitive techniques, it was by no means a non-treatment, and indeed was associated with lowered symptoms on outcome.22 The SWI condition combined this approach with the medical model derived from the TDCRP clinical management manual of Fawcett et al.,23 which stresses a biochemical rationale and assesses medication side effects. Imipramine was titrated upward on a dosing schedule to a peak of 250 mg daily unless limited by side effects. CBT and IPT involved 16 sessions of 50 minutes within a 17-week envelope. The SWI and SP conditions ranged between 8 and 16 sessions, as needed, of 30 to 50 minutes' duration.
Therapists were psychiatrists, psychologists, social workers, and psychiatric nurses experienced in working with depressed patients and knowledgeable about HIV.19,22 They were trained during a pilot phase by experts in each modality and monitored throughout the study to avoid drift in technique. Monitoring included individual supervision of audiotaped sessions and ratings of randomly selected sessions by independent and reliably trained adherence monitors. Data were analyzed on 2 CBT, 4 IPT, 8 SP, and 4 SWI therapists, who treated 12, 14, 15, and 15 depressed HIV subjects, respectively.
Subjects were HIV-positive depressed outpatients, mainly gay and bisexual white males, who were not too acutely medically ill to begin treatment and who had a score of 
15 on the 24-item version of the Hamilton Rating Scale for Depression (Ham-D)24 and a clinical diagnosis of probable DSM-III-R mood disorder.22
Measures
Outcome measures included the 24-item version of the observer-rated Ham-D and the self-report Beck Depression Inventory (BDI).25 An independent rater scored the Ham-D at intake, at midtreatment, and just before termination of the 16-week course. Therapists also rated the Ham-D at alternate sessions. Subjects completed the BDI before the start of each session.
The CSPRS, developed for the TDCRP, is the sixth version evolving from a lengthy psychometric testing process by Hollon.12 Its 96 items, rated on a 7-point, Likert-type scale, have defined anchor points (1="not at all" to 7="extensively"). The CSPRS includes subscales for CBT (28 items), IPT (28 items), and CM (20 items); two modality-tangential scales (4 CBT, 4 IPT); and modality-nonspecific scales titled "facilitative conditions" (FC, 8 items; measures of nonspecific therapist qualities such as empathy and warmth) and "explicit directiveness" (ED, 4 items; how much the therapist guides the subject). In adapting the scale for this study, we expanded the 96 items of the CSPRS to 104 items to make the "CSPRS-HIV." The eight new items reflected the addition of SP therapy (e.g., "The therapist echoed the patient's concerns as a means of facilitating the patient's discussion of those concerns"). They were incorporated without the involved psychometric testing that heralded the original instrument.12
Raters
Independent raters were four predoctoral psychology graduate students who developed reliability after approximately 40 hours of training. Training, conducted initially by the first author (J.C.M.) and subsequently by the two initial trainees (including P.A.S.), included reading the CSPRS manual12 and rating and discussing up to 16 pilot tapes, randomly chosen but covering all interventions, with ongoing supervision and feedback. Because the design of the two studies differed, and because raters used the 104-item adaptation (CSPRS-HIV) of the 96-item CSPRS, no attempt was made to calibrate their ratings with tapes from the TDCRP. Raters met periodically with the trainer to discuss tapes and prevent drift.
Two complete audiotaped sessions were rated per therapist–subject dyad: an early session, randomly chosen from sessions 3–6, and a late one, from sessions in weeks 9–12 of treatment. (When on occasion the selected tape was unavailable or inaudible, an adjacent session was assessed.) This contrasts with the four tapes per patient (sessions #1; 4; 7 or 8; and 14, 15, or penultimate) assessed by Hill et al.15 We also made no attempt to replicate the design of Hill and colleagues in randomly assigning raters to tapes. Instead, in a more naturalistic design, raters were alternately assigned tapes as they became available from the treatment study. Study raters were blind to treatment modality and to early versus late session status.
Statistical Analysis
Data analysis employed intraclass correlations26 to assess interrater reliability, following the model of Hill and colleagues,15 using a single random effect in which different subsets of raters rated each subject. Following the TDCRP analyses, analysis of variance (ANOVA) was used to assess for treatment adherence by intervention and multiple regression analysis to assess therapist FC effects on outcome, controlling for intake score and intervention.
|
RESULTS |
|---|
|
TopAbstractIntroductionMETHODSRESULTSDISCUSSIONReferences |
|---|
|
|
|
|
Because all therapists met criteria for adherence, the effect of adherence on outcome was not explored. We hypothesized that "facilitative conditions" (FC), measuring therapist qualities such as warmth, supportiveness, and empathy, might reflect general competence independent of treatment adherence, and thereby predict treatment outcome. Multiple regression analysis was employed, controlling for initial BDI score or Ham-D score and for treatment intervention. For individual therapists, there was no significant effect of FC on treatment outcome as measured by BDI and Ham-D scores, although 15% of BDI outcome (F=1.52, df=10,41, P=0.17) and 20% of Ham-D variance (F=1.44, df=10,41, P=0.20) were explained by FC.
|
DISCUSSION |
|---|
|
TopAbstractIntroductionMETHODSRESULTSDISCUSSIONReferences |
|---|
This replication of the TDCRP findings, under slightly altered conditions, confirms our initial two hypotheses. Demonstration of interrater reliability for adherence monitors and adherence for therapists validates the outcome results from this treatment study.14 Thus adherence here demonstrates both discrete adherence of the therapist to a particular modality (i.e., treatment integrity) and avoidance of the techniques of other modalities (i.e., treatment differentiability).28 Some overlap of treatments is inevitable due to the so-called common factors of psychotherapy.29
Our final hypothesis, however, was not confirmed. Although some variation appeared in therapist FC, it did not affect treatment outcome. It may be that FC does not measure, and might even confound (I. Elkin, personal communication, July 1996), true therapist competence. Or perhaps when good therapists are all adherent, outcome measures reflect differences in treatment technique rather than therapist differences.
One limitation of this research is the relatively small patient sample, although it proved more than sufficient to yield statistically significant results. Differing numbers of therapists and cases per treatment cell may have influenced the variance within cells, yet all therapists were rated adherent in each cell. Raters may have noted the shorter length of SP and SWI sessions, but raters were asked to rate 104 adherence items, not to guess the treatment modality.
Our SP subscale, psychometrically untested and measuring a relatively featureless therapy, still had high intraclass correlation scores at least comparable to those for CM on the original CSPRS. The time-by-session interaction noted for IPT and SWI probably reflects a decrement in therapist activity as recovering patients in these more potent therapies learned their treatment rationale: if the patients, rather than their therapists, voiced therapeutic shibboleths, scoring of therapist adherence would fall. By contrast, the SP therapists, whose subjects showed less improvement, may have redoubled their treatment efforts in later sessions.
Monitoring of psychotherapy adherence is a young science; the terms do not even appear in the index of a major recent reference text.30 It is reassuring that adherence technology can be translated from a state-of-the-art, multisite NIMH-designed treatment program to a different locus and design. The similarity of the rating scores for interventions in the two studies suggests either the ease and consistency with which these specified treatments can be learned or, perhaps, a systematic bias of the rating instrument. Hill and colleagues called the CSPRS "robust"15 (p. 77). Our clinical impression, too, is that the CSPRS accurately differentiated among the treatments our highly skilled therapists delivered.
|
Acknowledgments |
|---|
The authors thank Bonnie Gitlin, M.A., Stephen Hartman, M.A., and Michael Moffatt, M.A., for their superb adherence monitoring. This work was supported in part by Grant MH46250 from the National Institute of Mental Health and by the Reader's Digest and New York Community Trust. It was presented in part at the annual meeting of the Society for Psychotherapy Research, York, England, June 1994.
|
References |
|---|
|
TopAbstractIntroductionMETHODSRESULTSDISCUSSIONReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  L. K. Leslie and M. L. Wolraich ADHD Service Use Patterns in Youth J. Pediatr. Psychol., July 1, 2007; 32(6): 695 - 710. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. C. MARKOWITZ Controlled Trials of Psychotherapy Am J Psychiatry, January 1, 2003; 160(1): 186 - 187. [Full Text]
|
|
|
|
 |
|
 J. C. Markowitz, L. A. Spielman, M. Sullivan, and B. Fishman An Exploratory Study of Ethnicity and Psychotherapy Outcome Among HIV-Positive Patients With Depressive Symptoms J Psychother Pract Res., October 1, 2000; 9(4): 226 - 231. [Abstract] [Full Text]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ALL ISSUES | SEARCH | TABLE OF CONTENTS |
