J Psychother Pract Res
HOME HELP FEEDBACK SUBSCRIPTIONS ALL ISSUES SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Høglend, P.
Right arrow Articles by Bentsen, H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Høglend, P.
Right arrow Articles by Bentsen, H.
Related Collections
Right arrow Psychodynamic Therapy
J Psychother Pract Res 9:190-199, October 2000
© 2000 American Psychiatric Association

Regular Article

Assessment of Change in Dynamic Psychotherapy

Per Høglend, M.D., Ph.D., Kjell-Petter Bøgwald, M.D., Svein Amlo, M.D., Oscar Heyerdahl, M.D., Øystein Sørbye, M.D., Alice Marble, Psy.D., Mary Cosgrove Sjaastad, M.D. and Håvard Bentsen, M.D., Ph.D.

Received December 3, 1999; revised May 15, 2000; accepted June 9, 2000. From the Department of Psychiatry, University of Oslo, Norway. Address correspondence to Dr. Høglend, Department of Psychiatry, University of Oslo, P.O. Box 85, Vinderen, N-0319 Oslo, Norway.


   Abstract
Top
 Abstract
Introduction
 METHODS
 RESULTS
 DISCUSSION
 CONCLUSIONS
 References
 
Five scales have been developed to assess changes that are consistent with the therapeutic rationales and procedures of dynamic psychotherapy. Seven raters evaluated 50 patients before and 36 patients again after brief dynamic psychotherapy. A factor analysis indicated that the scales represent a dimension that is discriminable from general symptoms. A summary measure, Dynamic Capacity, was rated with acceptable reliability by a single rater. However, average scores of three raters were needed for good reliability of change ratings. The scales seem to be sufficiently fine-grained to capture statistically and clinically significant changes during brief dynamic psychotherapy.

Key Words: Rating Scales • Outcome • Brief Psychotherapy


   Introduction
Top
 Abstract
 Introduction
METHODS
 RESULTS
 DISCUSSION
 CONCLUSIONS
 References
 
Some researchers have claimed that symptom measures or measures of global change after psychotherapy account for almost all of the variance found in specific outcome measures.1,2 Others emphasize the need for mode-specific outcome scales measuring factors such as quality of interpersonal relationships, insight, or self-esteem.3–5 It is a common clinical observation that some patients may achieve symptom relief by life restrictions, with little or no improvement in areas such as interpersonal functioning, insight, or tolerance for affects. Conversely, patients may sometimes tolerate increased levels of symptoms as their aspirations increase and new problem-solving strategies are tried out. Instruments for measuring symptoms or global functioning may therefore offer limited information about the complex composition of changes that potentially can occur.6

Theory-related, or so called mode-specific, instruments for measuring dynamic changes, developed by pioneers such as Karush et al.,7 May and Dixon,8 Kernberg et al.,9 Bellak et al.,10 and Semrad et al.,11 have been criticized for being too abstract, cumbersome, or unreliable or too highly correlated with symptom measures.

Idiographic (individualized) methods developed by Malan,12 Luborsky,13 Horowitz,14 and Perry15 provide important clinical information with regard to limited areas of psychological functioning. However, individualized measures have weak psychometric properties for group designs. Methods for post-treatment change ratings have been developed by Sifneos16 and Sandell.17 Change estimates from ratings made after therapy tend to be too highly correlated with post-treatment status,18 and such ratings may also be difficult to compare across cases.4

Later developments of batteries of dynamic scales such as the Patterns of Individual Change Scales (PICS),19 Scales of Psychological Capacities (SPC),20,21 and Karolinska Psychodynamic Profile (KAPP)22 have scales with poor to excellent reliability, and some aspects of their validity have been tested. These batteries are quite comprehensive and have many scales. The scales have only three to seven descriptive levels, which may impair their sensitivity for changes. The PICS scales, which have seven descriptive levels, could not capture statistically significant changes during brief psychotherapy with ordinary neurotic patients.23

On the basis of 20 years of clinical and research experience with brief dynamic psychotherapy, we have developed a new set of dynamic scales. We have been influenced by the work of several of the above-mentioned research groups. Thus, resemblances are intentional.

Like most other batteries of dynamic scales, our scales do not measure personality traits or typologies. They describe internal predispositions, psychological resources, capacities, or aptitudes that can be mobilized by the individual in order to achieve adaptive functioning and life satisfaction. Unlike most other batteries, our rating scales cover the entire range of functioning, from superior to extremely poor. Our intention has been to make the scales "fine-grained" enough to capture reliable changes during brief dynamic psychotherapy.

The scale format has been modeled after the Global Assessment Scale (GAS),24 with ten descriptive levels and scale points ranging from 1 to 100. The use of a well-known scale format should make the scales easier to learn. The descriptive levels are linked as closely as possible to the way mainstream psychodynamically oriented clinicians interpret and work with clinically observable phenomena.

Value judgments, especially with regard to higher levels of functioning, are unavoidable with scales of this type. The decision to select five dimensions is based on clinical experience and literature. Psychoanalytic theories give limited assistance in the task of choosing dimensions. Our ambition has been to construct as few scales as possible and still maintain a reasonable comprehensiveness. Several related psychological resources have therefore been incorporated within the same scale.

The content validity and Guttman scale structure have been tested with Q-sort methodology performed by a large number of psychotherapists from Norway, Finland, and Germany.25 A few global scales with many descriptive anchor points are easy to use, and such scales have been demonstrated to be among the most powerful in detecting change.24 Several studies have indicated that global scales rated by experts may be equal and sometimes superior to test batteries with many subscales.26 However, reliability and predictive validity depend on the issue under study.

Current functioning within the last 3 to 4 months should be rated on the basis of a semistructured dynamic interview that includes interpersonal functioning, tolerance for affects, insight, and the capacity to handle both the ordinary vicissitudes of life and more challenging psychosocial stressors (problem-solving capacity). The five scales are described in Appendix A.

The present study tests the interrater reliability of the five scales, the reliability of change ratings, the discriminability from global functioning (Global Assessment of Functioning [GAF])27 and subjective distress (Global Severity Index [GSI] from the Symptom Checklist-90),28 and the scales' sensitivity for change during brief dynamic psychotherapy.


   METHODS
Top
 Abstract
 Introduction
 METHODS
RESULTS
 DISCUSSION
 CONCLUSIONS
 References
 
Patients
The patients are the first 50 cases included in a large-scale experimental study of patient–therapist interaction (transference) in brief dynamic psychotherapy. There were 24 men and 26 women. Their ages ranged from 26 to 58 (median 35) years. Twenty-six were married, 17 never married, and 7 divorced. The majority were professional, middle-class individuals with an average formal education of about 15 years. DSM-IV Axis I diagnoses27 were mostly adjustment disorders, anxiety and affective disorders, and problems not due to a mental disorder. Twenty-four patients (48%) had one or more Axis II disorders—primarily dependent, avoidant, obsessive-compulsive, or depressive personality disorder. The patient sample had, on the average, mild to moderate symptoms and dysfunctions. The mean GAF score at the initial dynamic evaluation was 60 (SD=7.4, range 44–79). The mean GSI score was 1.02 (SD=0.61, range 0.04–3.13). The distribution of mean pre-treatment scores indicated that the sample of 50 patients was a group of mildly to moderately disturbed individuals representative of typical outpatients offered psychotherapy. The range of the pre-treatment scores of the five dynamic scales covered the area of functioning from relatively severe and chronic disturbances to mild and intermittent problems of living (range 41–81). Informed consent was obtained for all subjects in this study.

Therapists
The therapists were 6 psychiatrists and 1 clinical psychologist. They all had long experience in practicing dynamic psychotherapy (range 10–25 years). Eachtherapist worked in a different institution. They hadreceived formal education in psychoanalytic psychotherapy from four different training institutes. All of the therapists were also clinical evaluators. Because the group of raters had such long experience, no pilot training using the dynamic scales was offered, only didactic lessons.

Evaluation
After history-taking and assessment of background variables, each patient was interviewed by one clinician in the presence of two or three other clinicians. When necessary, several of the clinicians posed additional questions after the interview to ensure adequate coverage of the patient's level of functioning in all areas. The group interviews lasted 60 to 100 minutes and included some trial interpretations. Ratings on the five dynamic scales and GAF were done independently by each clinician, before any discussion of the case. Half of the assessments were done by clinicians who had not been present at the dynamic interview. Their assessments were based on the audiotapes from the interviews. The patients filled out the SCL-90-R along with many other self-reports. At present, 36 patients have been reevaluated one year after the start of therapy. Most of them were in treatment for about one year and had recently ended therapy. The therapies were manualized (P. Høglend, unpublished manuscript). Adherence checks were done on several occasions for each case (sessions 7 and 16 plus randomly drawn sessions) in order to secure treatment integrity; details have been published elsewhere.29 The patient and therapist could agree jointly to end therapy before one year if sufficient progress was achieved. The number of treatment sessions ranged from 28 to 40 (median=36).

Data Analysis
The seven raters assessed 50 patients before therapy and 36 of the same patients after therapy. Three of the raters have assessed all of the interviews, and four others have assessed varying numbers of interviews. This design allows several versions of intraclass reliability estimates to be calculated.30 Intraclass correlation coefficients (ICC) are derived from analysis of variance components. Because our design is unbalanced (i.e., all raters did not assess all subjects), we used restricted maximum likelihood approaches. And because we did not assume lack of rater bias, we chose a two-way analysis of variance, random model (random effect of rater, random effect of subject). Average pre- treatment scores on each scale were compared with average post-treatment scores, by use of paired t-tests, on the subsample of 36 patients evaluated before and after therapy. Rating of change is generally more unreliable than status ratings.30 Therefore, repeated-measures analysis of variance with time (pre- and post-treatment) and raters (the three raters with full data sets) as factors were performed on the same subsample in order to analyze in greater detail the differences between raters in assessing change. The interrater reliability of raw change and residual gain scores is also reported for a summary measure of the dynamic scales.

After the intercorrelation matrix of average scores on all pre-treatment variables had been examined, a factor analysis of the variables, with maximum likelihood extraction, was computed.31


   RESULTS
Top
 Abstract
 Introduction
 METHODS
 RESULTS
DISCUSSION
 CONCLUSIONS
 References
 
Out of the whole group of seven clinicians, the interrater reliability estimates for single raters—who rated a variable number of the 50 patients at pre-treatment and of the 36 patients at post-treatment—are shown in Table 1.


View this table:
[in this window]
[in a new window]
  		TABLE 1. Interrater reliability estimates (intraclass correlations; ICC), with 95% confidence intervals (CI), for single raters randomly drawn from a group of seven raters
 
The lower bounds of the confidence intervals were unsatisfactory (<0.50) for three of the single scales at pre-treatment: tolerance for affects, insight, and problem-solving capacity. In scientific studies, such results of reliability estimates for single scales are unsatisfactory. Ideally, the lower bounds of the confidence intervals should be >0.70. With average scores of the three raters who rated all subjects, the intraclass reliability estimates (average measure ICC, two-way, mixed model [fixed effect of rater, random effect of subject]), we achieved ideal results for all single scales, at pre-treatment as well as post-treatment.

Table 2 presents mean scores on all measures at pre-treatment and post-treatment for the 36 patients evaluated on both occasions.


View this table:
[in this window]
[in a new window]
  		TABLE 2. Changes from pre-treatment to post-treatment (average scores of all raters; n=36)
 
The largest amount of change during individual psychotherapy, and the highest ratios of patients with reliable changes according to the Reliable Change Index criteria by Jacobson and Truax,32 tended to be in the areas of insight and tolerance for affects.

Repeated-measures analysis of variance, with the dynamic scales as dependent variables and time and raters as factors, showed five significant main effects for Time and one significant main effect for Raters, as shown in Table 3.


View this table:
[in this window]
[in a new window]
  		TABLE 3. Repeated-measures analysis of variance with time (1, 2) and raters (1, 2, 3) as factors (n=36)
 
These findings indicated that there were no significant differences among the raters in their assessment of pre- to post-treatment changes on the dynamic scales, except for insight. On this scale one rater (K.P.B.) tended to rate patients higher than one of the other raters (Ø.S.) both before and after treatment. Three significant interaction effects (TimexRaters) were found, with tolerance for affects, insight, and problem-solving capacity as the dependent variables. With regard to insight, one rater ( P.H.) tended to rate the patients lower at pre-treatment, but not post-treatment, compared with one other rater (K.P.B.). The pattern was similar for the two other scales.

For four of the dynamic scales, 21% to 43% percent of the variance of average scores (r2) was shared variance with GAF. There was, however, a very high overlap between problem-solving capacity and GAF. The two variables shared 64% of the variance. Given the reliabilities of the two variables (>0.80), our findings may indicate that they measure nondiscriminable constructs. The two variables shared more than 71% of the reliable variance33 in this study. All other scales shared less than 48% of the reliable variance with GAF (and GSI).

A factor analysis was computed to evaluate whether or not the dynamic scales can be differentiated from global functioning and subjective distress. The number of factors was determined by the eigenvalues-greater- than-unity rule. Table 4 shows the results of the factor analysis.


View this table:
[in this window]
[in a new window]
  		TABLE 4. Factor analyses of dynamic scales and symptom scales (N=50)
 
The results were easy to interpret. However, because of the small sample size these results may be unstable and should be considered preliminary. Problem-solving capacity loads on both factors, but it is higher on the dynamic factor. The five dynamic scales constitute a dimension that is probably discriminable from a general symptoms-and-dysfunction dimension. We have decided to name this dimension Dynamic Capacity. Cronbach's alpha was 0.89. A simple weighted sum-score of the five scales represents this dimension well. Dynamic Capacity shared 42% of overall variance and 47% of the reliable variance with GAF33 in this study. The reliability estimates for single raters on Dynamic Capacity were 0.71 and 0.80 at pre- and post- treatment, respectively. The lower bounds of the confidence intervals of the reliability estimates for Dynamic Capacity were 0.60 (pre-treatment sample) and 0.72 (post-treatment sample). This was comparable to the lower-bound estimates obtained for GAF, 0.58 and 0.69 (see Table 1). Average scores of two raters are needed to secure lower-bound reliability estimates above 0.70. Average scores of three raters are needed to secure lower-bound reliability estimates above 0.80. Repeated-measures analysis of variance with Dynamic Capacity as the dependent variable and time and single raters as factors showed a significant main effect for time (F=77.48, df=1,35, P<0.000), no significant main effect for raters (F=0.90 , df=2,34 P<0.90, not significant), but a significant interaction effect (F=11.10, df=2,34, P<0.000). Average scores of three raters are needed in order to secure adequate reliability of raw change scores (ICC=0.75, lower bound of the confidence interval 0.56) and residual gain scores (ICC= 0.81, lower bound of the confidence interval 0.67).


   DISCUSSION
Top
 Abstract
 Introduction
 METHODS
 RESULTS
 DISCUSSION
CONCLUSIONS
 References
 
The dynamic scales seem to measure a construct that may prove discriminable from general symptoms and dysfunction. This is consistent with findings reported by Weiss et al.,3 Sundin and Armelius,34 and Perry et al.35 The reliability estimates for Dynamic Capacity were equal to those for the well-established GAF scale. In a different psychotherapy study, three pairs of raters, after pilot training on three cases, have so far rated 10 patients at pre-treatment. Their interrater reliability was comparable to the results from this study. The single- rater reliability estimate for Dynamic Capacity was 0.79 (A. G. Hersoug, personal communication, 1997).

The reliability estimates for individual scales reported in this study tend to be similar to or higher than reliability estimates for individual scales from other studies using new batteries of dynamic scales.3,19,21,22,36 The cited studies report that a number of individual scales had reliability coefficients below 0.50. We believe that our favorable results for individual scales are at least partly due to our scale format, which includes more descriptive levels and considerably more rating options than other dynamic scales. It is also possible that the comprehensive evaluation interviews secured more complete data for reliable assessments. Our scales are fine-grained enough to capture statistically significant and reliable changes during brief dynamic psychotherapy with ordinary neurotic outpatients. The highest ratios of patients with reliable changes were found in the areas of insight and tolerance for affects. This pattern of change is consistent with the techniques of exploratory dynamic psychotherapy, which specifically aim at endowing patients with greater insight and heightened awareness of their affects. However, this finding is only a weak indication of specificity, since we have no untreated control group or nondynamic alternative treatment group in this design. This is the first psychotherapy study to present reliability estimates of change ratings with dynamic scales.

Insight was the most difficult scale to rate reliably, especially at pre-treatment. Dynamic insight and tolerance for affect are measures of intrapsychic functioning that are closely connected to psychoanalytic theory, requiring more clinical inference or intuitive judgment in the assessment procedure.37 Personality features of different raters may contribute to rater bias. However, insight is considered a central curative factor in dynamic therapy and should be included in instruments for assessment of dynamic changes.

The raters were not blind with regard to whether the evaluation was pre-treatment or post-treatment. Time of evaluation is also frequently apparent from the content of the audiotaped interview. This may have influenced the ratings. We have not been able to detect any systematic differences between therapists rating their own patients and other evaluators' ratings.

The group interview setting may have influenced the interview process and also the clinical inferences upon which the ratings were based. We believe that having more than one interviewer for each patient secures more adequate coverage of all relevant aspects of patient functioning. Dynamic interviews, necessarily less structured than diagnostic interviews, may be unduly influenced by idiosyncratic "matches" between a single interviewer and the patient, and this may lead to a less than adequate interview when the aim is reliable assessment of complex human behavior. On the other hand, the independence of the raters may be compromised when several of them interview the patient on the same occasion. However, we detected no significant differences in the reliability estimates of the ratings between raters present at the interview and raters who only listened to audiotapes of the interview.


   CONCLUSIONS
Top
 Abstract
 Introduction
 METHODS
 RESULTS
 DISCUSSION
 CONCLUSIONS
References
 
Our findings are preliminary. The interrater reliability of five dynamic scales for assessment of changes beyond symptoms during and after psychotherapy was satisfactory. This is the first study that also analyzes in greater detail the reliability of change ratings with dynamic scales. The scales may prove to be discriminable from general symptom measures, and they are fine-grained enough to capture statistically significant changes during brief dynamic psychotherapy. A summary measure of the five scales can be rated with adequate reliability by a single evaluator, but average scores of three raters are needed for reliable change ratings such as those used in many treatment studies.38


   Acknowledgments
 
This research was supported by the National Council for Science and Humanities, Norway; Norwegian Council for Mental Health; and Diakonhjemmet Hospital, Oslo, Norway.



View larger version (74K):
[in this window]
[in a new window]
  		Appendix
 

   References
Top
 Abstract
 Introduction
 METHODS
 RESULTS
 DISCUSSION
 CONCLUSIONS
 References
 
  1. Green BL, Gleser GC, Stone WN, et al: Relationships among diverse measures of psychotherapy outcome. J Consult Clin Psychol 1975; 43:689–699[Medline]
  2. Mintz J: Measuring outcome in psychotherapy: psychodynamic versus symptomatic assessments. Arch Gen Psychiatry 1981; 38:503–506[Abstract]
  3. Weiss DS, DeWitt KN, Kaltreider NB, et al: A proposed method for measuring change beyond symptoms. Arch Gen Psychiatry 1985; 42:703–708[Abstract]
  4. Høglend P, Sørlie T, Sørbye Ø, et al: Long-term changes after brief dynamic psychotherapy. Acta Psychiatr Scand 1992; 86:165–172[Medline]
  5. Connolly MB, Strupp HH: Cluster analysis of patient reported psychotherapy outcomes. Psychother Res 1996; 16:30–42
  6. Høglend P: Personality disorders and long-term outcome after brief dynamic psychotherapy. J Personal Disord 1993; 7:168– 181
  7. Karush A, Esser BR, Cooper A, et al: The evaluation of ego- strength. J Nerv Ment Dis 1964; 130:322–349
  8. May PR, Dixon WJ: The Camarillo Dynamic Assessment Scales, I: measurement of psychodynamic factors. Bull Menninger Clin 1969; 33:1–11[Medline]
  9. Kernberg O, Burstein ED, Coyn L, et al: Psychotherapy and psychoanalysis: final report of the Menninger Foundation's psychotherapy research project. Bull Menninger Clin 1972; 36:1– 275[Medline]
  10. Bellak L, Hurvich M, Gediman HK: Ego Functions in Schizophrenics, Neurotics and Normals. New York, Wiley, 1973
  11. Semrad EV, Grinspoon L, Fienberg FE: Development of an ego-profile scale. Arch Gen Psychiatry 1973; 28:70–77[Medline]
  12. Malan DH: The Frontier of Brief Psychotherapy: An Example of the Convergence of Research and Clinical Practice. New York, Plenum, 1976
  13. Luborsky L: Measuring a pervasive psychic structure in psychotherapy: the Core Conflictual Relationship Theme, in Communicative Structures and Psychic Structures, edited by Freeman N, Grand S. New York, Plenum, 1977, pp 367–395
  14. Horowitz MJ: States of Mind, 2nd edition. New York, Plenum, 1987
  15. Perry JC: Wishes and Fears: A Standard List and Guidelines for Assessing Dynamic Motives. Boston, The Cambridge Hospital, 1989
  16. Sifneos P: Short-term Dynamic Psychotherapy. New York, Plenum, 1979
  17. Sandell R: Assessing the effects of psychotherapy, II: a procedure for direct rating of therapeutic change. Psychother Psychosom 1987; 47:37–43[Medline]
  18. Streiner DL, Norman GR: Health Measurement Scales. Oxford, UK, Oxford University Press, 1995
  19. Kaltreider NB, DeWitt KN, Weiss DS, et al: Patterns of Individual Change Scales. Arch Gen Psychiatry 1981; 38:1263–1269
  20. DeWitt KN, Hartley DE, Rosenberg SE, et al: Scales of Psychological Capacities: development of an assessment approach. Psychoanalysis and Contemporary Thought 1991; 14: 343–361
  21. Sundin E, Armelius B-C, Nilsson T: Reliability studies of scales of psychological capacities: a new method to assess psychological change. Psychoanalysis and Contemporary Thought 1994; 17:591–615
  22. Weinryb RM, Røssel RJ, Cosberg M: The Karolinska Psychodynamic Profile, II: interdisciplinary and cross-cultural reliability. Acta Psychiatr Scand 1991; 83:73–76[Medline]
  23. Horowitz MJ, Marmar CR, Weiss DS, et al: Comprehensive analysis of change after brief dynamic psychotherapy. Am J Psychiatry 1986; 143:582–589[Abstract/Free Full Text]
  24. Endicott J, Spitzer RL, Fleiss JL, et al: The Global Assessment Scale: a procedure for measuring overall severity of psychiatric disturbance. Arch Gen Psychiatry 1976; 133:766–771
  25. Bøgwald K-P, Høglend P: A new procedure for content validation of dynamic scales. Paper presented at Society for Psychotherapy Research, 30th Annual Meeting, Braga, Portugal, 1999
  26. Regehr G, McRae H, Reznic R, et al: Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med 1998; 73:993–997[Medline]
  27. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, 4th edition. Washington DC, American Psychiatric Association, 1994
  28. Derogatis LR: SCL-90: Administration, scoring and procedures manual for the revised version. Baltimore, Clinical Psychometric Research, 1983
  29. Bøgwald K-P, Høglend P, Sørbye Ø: Measurement of transference interpretations. J Psychother Pract Res 1999; 8:264–273[Abstract/Free Full Text]
  30. Dunn G: Design and analysis of reliability studies. Statistical Methods in Medical Research 1992; 1:123–157[Medline]
  31. Mardia KW, Kent JL, Bibby JM: Multivariate Analysis. New York, Academic Press, 1989, pp 275–276
  32. Jacobson NS, Truax P: Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991; 59:12–19[Medline]
  33. Lund T: Some metrical issues with meta-analysis of therapy effects. Scand J Psychol 1988; 29:1–8[Medline]
  34. Sundin E, Armelius B-C: Mental health and psychic structure: an empirical study (DAPS report 67). Sweden, Umeå University, 1996
  35. Perry JC, Høglend P, Shear K, et al: Field trial of a diagnostic axis for defence mechanisms for DSM-IV. J Personal Disord 1998; 12:56–68[Medline]
  36. Haver B, Svanborg P, Lindberg S: Improving the usefulness of the Karolinska Psychodynamic Profile in research: proposals from a reliability study. Acta Psychiatr Scand 1995; 92:132–137[Medline]
  37. Høglend P, Engelstad V, Sørbye Ø, et al: The role of insight in exploratory psychodynamic psychotherapy. Br J Med Psychol 1994; 67:305–317
  38. Cronbach LJ, Furby L: How should we measure "change"—or should we? Psychol Bull 1970; 74:68–80



This article has been cited by other articles:


Home page
 Am. J. PsychiatryHome page
P. Hoglend, K.-P. Bogwald, S. Amlo, A. Marble, R. Ulberg, M. C. Sjaastad, O. Sorbye, O. Heyerdahl, and P. Johansson
Transference Interpretations in Dynamic Psychotherapy: Do They Really Yield Sustained Effects?
Am J Psychiatry, June 1, 2008; 165(6): 763 - 771.
[Abstract] [Full Text] [PDF]

Home page
 Am. J. PsychiatryHome page
P. Hoglend, S. Amlo, A. Marble, K.-P. Bogwald, O. Sorbye, M. C. Sjaastad, and O. Heyerdahl
Analysis of the Patient-Therapist Relationship in Dynamic Psychotherapy: An Experimental Study of Transference Interpretations
Am J Psychiatry, October 1, 2006; 163(10): 1739 - 1746.
[Abstract] [Full Text] [PDF]

Home page
 AssessmentHome page
R. F. Bornstein, K. J. Geiselman, E. A. Eisenhart, and M. A. Languirand
Construct Validity of the Relationship Profile Test: Links With Attachment, Identity, Relatedness, and Affect
Assessment, December 1, 2002; 9(4): 373 - 381.
[Abstract] [PDF]

Home page
 J. Psychother. Pract. Res.Home page
A. G. Hersoug, P. Hoglend, J. T. Monsen, and O. E. Havik
Quality of Working Alliance in Psychotherapy: Therapist Variables and Patient/Therapist Similarity as Predictors
J Psychother Pract Res., October 1, 2001; 10(4): 205 - 216.
[Abstract] [Full Text]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Høglend, P.
Right arrow Articles by Bentsen, H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Høglend, P.
Right arrow Articles by Bentsen, H.
Related Collections
Right arrow Psychodynamic Therapy

HOME HELP FEEDBACK SUBSCRIPTIONS ALL ISSUES SEARCH TABLE OF CONTENTS