Jones and Yarhouse have also responded to Chapman's criticisms:
http://www.exgaywatch.com/wp/2007/11/re ... f-ex-gays/Response to Dr. Patrick M. Chapman’s Critique of ‘Ex-Gays’ – Part 1
November 30th, 2007 David Roberts
Response to Part 1 of Dr. Patrick M. Chapman’s Review of “Ex-Gays”, posted on Ex-Gay Watch, November, 2007, by Stanton L. Jones and Mark A. Yarhouse.
The greatest compliment that be paid to any work of scholarship is for it to receive serious consideration and generate discussion. Thus, we are pleased to see the review by Dr. Chapman of our book, Ex-gays?: A Longitudinal Study of Religiously Mediated Change in Sexual Orientation. Chapman raises important issues, but in the end, we must conclude that his review fails to establish the serious flaws he claims in our study.
Response to “Part 1: Introduction and Methods”
We applaud Chapman for correctly summarizing the main questions we examined in the study, for a reasonable brief summary of the study’s methodology, and particularly for granting us some credulity in saying that “They claim the ex-gay organization [Exodus] did not exert any control or power over their results and conclusions (p. 127), and there is currently no reason to believe otherwise.” Minor points of disagreement with his summary and commentary include the following:
Our interest was not triggered by “the conflicting views of science [versus the claims of our] conservative Christian acquaintances;” but rather by the conflict between a) the prevailing and hardening consensus of mental health opinion that change is utterly impossible, based on a very mixed scientific record, versus b) the actual scientific record and the anecdotal claims of people we know. Regarding the actual scientific record, note for instance the recent publication by a respected scholar of a report of some notable plasticity in “female same-sex sexuality” in a minority of women followed in a longitudinal study (Lisa Diamond, Perspectives on Psychological Science, 2(#2), 142-161. Diamond rightly concludes “the more we learn, the more we do not understand,” p. 142. She also, it must be said, would not regard her findings as providing support for change as understood in this study, but on the other hand, her results do challenge a simple “sexual orientation is utterly and always unchangeable” stance). And Chapman in his review gives weight to the anecdotes of people he knows, and his own story, so once again we raise the question why only certain anecdotes are privileged as worthy of consideration in this debate.
Chapman implicitly dismisses “behavior modification” as trivial, but we see insufficient justification to take this step. Some of our subjects experienced more than mere behavior modification, and even behavior modification can be very meaningful if it empowers a person to live in closer accord with her freely chosen core values.
The core of Chapman’s criticism of the study in Part 1 is that our study is somehow not truly prospective. We would agree that if our study is not prospective then it is disingenuous to claim that it is, and the scientific value of the study is considerably weakened. This charge, in other words, is truly significant. Let’s look carefully, then, at the basis for Chapman’s claims.
First, Chapman claims that “technically the study is not prospective because 41 individuals were involved in the Exodus program for one to three years prior to the study (p. 121).” The logic of this argument is not compelling. We are utterly explicit that some of the subjects (the 41 “Phase 2 subjects” in the change process with their current Exodus ministry for 1 to 3 years) had been in the change process longer than others (the 57 “Phase 1 subjects” in the change process for less than 1 year). We continue to maintain that the results for the Phase 2 subjects are worthy of inclusion and consideration, but we always report analyses of the Phase 1 population by itself for precisely the concern Chapman articulates: If the reader insists on a tighter understanding of “prospective,” then you can narrow the focus to the Phase 1 results. These results were not as positive as those for the population as a whole, but were still statistically significant and meaningful, with Phase 1 subjects represented in all six categories of outcomes. Again, for Chapman to focus on the 41 Phase 2 subjects and then pronounce the whole study as not prospective makes no more sense than declaring that the results of our study are irrelevant for men because there were 26 women in the study.
Chapman’s second concern is more interesting and merits serious discussion. He argues that our study is not prospective because “the claim that participants were at the start of their change process is misleading.” He then cites several pieces of data indicating that subjects had previously tried to use other methods to change their sexual orientation before starting their current Exodus involvement (including through involvement in other religious ministries and professional therapy), and then concludes “Suggesting the individuals in this study are ‘starting the change process’ is incorrect. Perhaps this was their first attempt with Exodus ministries but that is not the same as ‘starting the change process.’”
Chapman seems to be arguing for an extremely literalistic understanding of “starting the change process.” Our research question was the possibility of change through involvement in an Exodus ministry, and so we focused on persons between zero and 3 years into that change process. Chapman is arguing for a much more rigorous standard: that the only proper way to study change is to locate and study what we might call “change virgins,” people who had never attempted change at all. We would argue that such a standard is unreasonable for several reasons:
First, such a standard is rarely applied in the study of other intervention methods with other targets of intervention. We urge that our study be examined according to the standards applied to all psychological studies of change, and not by ad hoc standards with few parallels in the general literature. We compare our results in the book with the pattern of results for the STAR*D treatment study of chronic depression, but the very idea that you would screen out all subjects who had previously sought help to change their depressive patterns to get a sample of “change virgins” is not credible. If your goal is to study the effectiveness of a particular intervention method, why would you screen out of your study persons who had previously sought change by other means, especially when it is common in these ministries to work with people who have attempted to change before?
Second, to erect such a requirement for the validity of a study of change of sexual orientation would be to make such a study impossible to conduct. How would you find a pure sample of “change virgins” who had never attempted change? If people are distressed by their sexual orientation for religious, moral or other reasons, isn’t it likely that those person would try a variety of formal and informal means to change that orientation?
Most importantly, if our research question is that of the possibility of change through involvement in an Exodus ministry, why would prior or even concurrent involvement in other methods of change serve as a barrier to involvement in the study? If we are studying the effectiveness of anti-depressants in treatment of depression or of interpersonal therapy on marital relationships, what is the relevance of the subjects having previously received pastoral counseling for depression or having attended a marriage encounter weekend to enhance marital satisfaction?
So in the end, in response to Chapman’s criticism that “Perhaps this was their first attempt with Exodus ministries but that is not the same as ‘starting the change process,’” we would simply reply that by our saying that these subjects were “starting the change process,” we were implicitly and explicitly saying “starting the change process in this particular Exodus ministry.” Hence, we believe that this study meets reasonable standards as a prospective study of individuals seeking sexual orientation change through the Exodus change process. Chapman’s criticisms fail to establish the contrary.
http://www.exgaywatch.com/wp/2007/12/re ... ys-part-2/Response to Dr. Patrick M. Chapman’s Critique of ‘Ex-Gays’ – Part 2
December 6th, 2007 David Roberts
Response to Part 2 of Dr. Patrick M. Chapman’s Review of “Ex-Gays”, posted on Ex-Gay Watch, November, 2007, by Stanton L. Jones and Mark A. Yarhouse.
Response to “Part 2: A Focus on the Results — Examining if Change is Possible”
Here in Part 2 Dr. Chapman’s criticisms turn more severe. First he asserts that ours is not a long term study. Again, his logic is questionable, and the problem of incomplete citation of our argument is significant. Chapman says “In the opening chapter Jones and Yarhouse honestly and correctly state this study cannot establish if long-term, permanent and enduring change occurs because that would require a long-term study (p. 17).” What we actually say on page 17 is that “this study will not establish that permanent, enduring change has occurred; only a very long-term study can demonstrate that.” Our point was not that our study was not a long-term study, nor that our study was inadequate to produce evidence suggesting that change was not impossible. Our point instead was that if you want to show that change is permanent, then logically you have to study subjects throughout their lifespans to death to insure the change was permanent. So our study cannot show that change is permanent, but even so a three to four year span of time is scientifically meaningful and qualifies as “long term.”
Chapman’s subsequent criticisms share a common characteristic that must be noted: Chapman imagines that he blunts our argument that change is possible for some by pointing out contrary pieces of isolated evidence that change did not happen for certain people or did not happen in certain ways he considers important. Science, in contrast, operates by examining all relevant data for trends, and then applies that data to the evaluation of hypotheses.
Our hypothesis regarding change was that “change is impossible.” The relevant data for falsification of that hypothesis is evidence that change is possible for some. Imagine the argument that “it is impossible to sustain life through heart transplant operations.” A scientist studies 100 heart transplants, and finds one year post-operation that 67% of transplant patients are still alive. Does the death of 33% constitute evidence in support of the argument “it is impossible to sustain life through heart transplant operations”? Of course not: If heart transplants are not supposed to help people, then the relevant data is data that falsifies the hypothesis, i.e., evidence of people surviving. Chapman’s selective citation of our data is the equivalent of focusing on the negative cases in this example. This is explicit, as Chapman argues that our conclusion that change is possible for some “is unwarranted because . . .” and then cites a series of evidences of incomplete change.
It was very surprising for Chapman to build the core of his argument around selectively citing the 3 tables (7.4 through 7.6; pp. 239-240) that show no change (which we openly admit) while completely ignoring the other tables on the related variables that show significant change (7.1 through 7.3; pp. 238-239) AND while completely ignoring all of the other variables measured (the balance of Chapter 7) on which statistically significant change and effect sizes ranging from small to large were demonstrated. It was in response to the broader pattern of evidence that we concluded that “change is possible for some” again and again through the book. Chapman says that “This study is littered with biased and sloppy scholarship,” but actually provides no evidence of this. Chapman and others who want to engage this work fairly need to respond to the overall pattern of our findings which, in contrast to the hypothesis that “change is impossible,” found many statistically significant changes and meaningful effect sizes on almost all of the measures of sexual orientation. How can an exclusive focus on those few instances where statistically significant change was not found be justified?
Chapman then turns to a rebuttal of our qualitative categorization of outcomes, focusing first on those we termed “Success: Conversion.” His core complaint is that some of these individuals report various forms of recurring homosexual attraction even as they also report satisfying heterosexual adjustment. Should individuals who report any sort of continuing homosexual attraction be considered to have changed? We discuss this matter throughout the book, but focus on it on pages 235-237 and 373-374, concluding that it is an unreasonable standard to deny that an individual has changed significantly if they experience any residual of homosexual desire. Chapman takes the stance that any signal of homosexual attraction indicates full and enduring homosexual orientation; this strikes us as a naïve and dichotomous understanding of sexual orientation. Further, such standards are not applied to other efforts at psychological change, and we believe they cannot and would not be so applied. Marital couples continue to struggle with conflict; persons with addictions continue to experience cravings. Put differently, the same sorts of standards that recognize significant change with other psychological patterns that are the subject of change attempts should hold for the area of sexual orientation as well.
Chapman then dismisses our conclusions about those who experienced a decrease in the potency of their homosexual desires and were able to embrace chastity, and who themselves considered this a successful outcome to the change process. Chapman suggests that we “accept asexuality as a functional opposite of homosexuality. Based on the depression analogy it appears that Jones and Yarhouse would declare a person ‘healed’ from depression if they ceased to have any and all emotions, for the person would no longer be intensely and persistently sad. I suspect the psychological community would define success in other ways.”
This is an important argument, to which we would respond in two ways. First, these individuals did not find themselves to be either devoid of all emotion entirely nor to be utterly asexual in the sense of being emotionally dead. Instead, their common testimony was of experiencing a diminishing of unwanted, powerful same-sex attractions, and that that decrease enhanced their experiences of satisfying emotional and relational connections with God and with other persons in non-erotic relationships. These people typically felt themselves more emotionally alive and healthy as a result of experiencing a decrease in homosexual attraction. Second, we must ask who has the authority to deny these individuals the opportunity to make their own choices about what they find satisfying in life? These individuals regard their adjustment to be successful; is Chapman positioned to assert his view of their lives over theirs? Yes, some of the subjects reported experiences discordant with their desires and hopes for complete change. But these individuals (except for the one who retracted his claim to change) did not see these experiences as negating the reality of positive change in their lives.
Chapman’s concluding paragraph deserves careful attention. We quote him, and then comment on each of his challenges:
“Despite explicitly stating that this study cannot demonstrate whether long-lasting change is possible. . .” As stated above, this is NOT what we said. What we said was that our study could not prove change was permanent.
“despite admitting that individuals in ex-gay ministries misreport their condition . . .” This is NOT what we said. Rather, we report in the book how some Exodus ministries urge their clients to reject the notion that their same-sex attractions mean that their identity is that of a homosexual person.
“despite knowing that previous testimonies of change were untrue . . .” Rather, we recognize that some previous testimonies of change have proven to be untrue.
“despite knowing that one of their own ‘Success: Conversion’ participants later recanted his proclaimed ‘conversion’ to heterosexuality. . .” As we say in the book, we report the data as it presents itself, as the experience of one person does not invalidate that of another. The experience of change of Alan Chambers, President of Exodus, does not invalidate Dr. Chapman’s experience that he did not change, and it is for this reason that we insist that the implication of our research is that change appears possible for some, specifically that “change is not impossible” (p. 365), and that our data does not prove “that everyone (or anyone) can change” (p. 372).
“despite the fact that ‘Success: Conversion’ and ‘Success: Chastity’ participants retain a homosexual orientation (using Jones and Yarhouse’s own definition). . . ” Chapman has inadequate basis for this claim. He selectively picks counter-examples to the evidence of significant change, and ignores the direct evidence of change such as the reported changes summarized in the bar graph on page 296.
Given Chapman’s selective engagement with the data of our study—specifically by focusing only on a series of small slices of the results congruent with his skepticism about change—he responds incredulously to the fact that “the authors claim that homosexual orientation is changeable! Clearly their conclusion is not consistent with the evidence.” In contrast, Dr. Chapman; you appear to have reached your conclusion that our evidence proves that change is impossible by selective engagement with only those pieces of evidence that fit your conclusion. We, in contrast, engaged all of the data as a whole.
Response to Dr. Patrick M. Chapman’s Critique of ‘Ex-Gays’ – Part 3
December 17th, 2007 David Roberts
http://www.exgaywatch.com/wp/2007/12/re ... ys-part-3/Response to Part 3 of Dr. Patrick M. Chapman’s Review of “Ex-Gays”, posted on Ex-Gay Watch, November, 2007, by Stanton L. Jones and Mark A. Yarhouse.
Response to “Part 3: A Focus on the Results — Examining if it is Harmful”
In this final response, Chapman raises a number of interesting questions, but again continues 1) applying a pattern of logic and argument that would, if applied broadly in the mental health field, establish self-defeating and unsustainable implications for the entire field and 2) on that basis then highlighting isolated findings and anecdotes as if they refute the broader pattern of empirical findings from the study.
In his first paragraph, Chapman chides us for imprecision and inconsistency both in how we characterize the claims about harm made by the various professional organizations, and in how we characterize our own findings and conclusions. He provides a link to the very same American Psychological Association Public Affairs website that we site in our book that cautions about harm from attempts to change sexual orientation. This is one of the less forceful warnings about harm (we cite others in our book in many places; see for example pp. 330-331). Further, public pronouncements by key professional representatives (for instance, psychiatrist Jack Drescher’s op ed piece, titled “Conversion attempts mostly lead to harm”) have yet further heightened the perceived likelihood and severity of risk of harm. Regarding his listing of how we describe this literature in the book, we do regret using “always” harmful (p. 19) as he points out, but the other quotes are reflective of the diverse array of characterizations of the likelihood of harm.
To address his pattern of logic, let’s begin by some simple clarification of how to think about harm. I (Jones) recently had minor knee surgery, and both the surgery itself and the medication prescribed post-surgery had risks. The fact that the rare person has had serious, even devastating reactions to such surgery and medication did not and can not itself invalidate my choice to pursue this procedure or the doctor’s administration of the treatment. The risks have to be weighed against the potential gains I expected in light of my dissatisfaction with the state of my knee prior to surgery and in light of the likelihood of such risks.
The attempt to change sexual orientation is no doubt much riskier and more challenging than knee surgery. But just how severe are the risks and just how likely are they to obtain? It is to answer this question that we framed our search for answers in this area in terms of harm “on average.” Chapman would seem to want to frame the question in terms of evidence that any harm occurs for anyone, a characterization substantiated by his listing of five anecdotes from our book of some level of unhappy reaction to the change process, followed by his rhetorical question, “One wonders what would have to be the reports of the participants for Jones and Yarhouse to declare the ministry harmful?” If only the matter were that simple. We could ask in return, How many positive results of participants would have to be reported, and how many reports of distress and unhappiness in living in the gay community would have to be reported, to justify the continuing existence of an option for attempting change? The type of standard used by Chapman would be completely unrealistic and paralyzing for the mental health field. Many interventions with complicated or distressing conditions produce some negative outcomes. When starting treatment with a depressed person, one always has some sense that if the attempt to intervene is unsuccessful, the person could plunge into despair about the possibility of change and be worse off than before. But such outcomes are not common.
But our answer was not to make that judgment for ourselves, but rather to report changes in distress level on average for those attempting change and to argue that ultimately it is the individuals themselves seeking change — and not Chapman or us — who should make their choices about whether or not to pursue change based on their own reading of the evidence. Chapman would urge that the professional world together declare such intervention attempts invalid based on the power of the anecdotes of harm; we would argue instead that individuals should be empowered with the best array of information available to make their best choices for themselves (see pp. 377-382).
Armed with a poorly developed rationale for how to handle harm, Chapman utterly disregards the pattern of standardized findings showing no escalating patterns of distress on average across the sample, and instead claims that the five anecdotes of distress and harm we present in our transcripts establish an unacceptable level of harm for participants. He states, “Nonetheless, dismissing this possibility and ignoring the statements of the participants that remained in the program, Jones and Yarhouse confidently declare the change process is not harmful. Once again, their conclusion is not based on the evidence: those who declare they are hurt by the process are evidence of harm.” This is both right and wrong. It is right in that we do indeed handle the few anecdotes of harm under the more general umbrella of empirical findings that distress does not increase on average. It is wrong, in that we do not “declare the change process is not harmful,” but rather declare the change process is not harmful on average. For further evidence that this is so, the reader should read our point 9 in our Conclusion (p. 376), in which we state emphatically that “despite our finding that on average participants experienced no harm from the attempt to change, we cannot conclude that specific individuals are not harmed by an attempt to change.” We point out there that harm may obtain because of the type of intervention or because of the emotional vulnerability of the person seeking change. We also allude to the fact of political realities: “It is also necessary to say that claims of harm may be ideologically based and exaggerated for the sake of foreclosing the option of the attempt to change” (p. 376).
Chapman asks, “How many lives must be broken before the authors realize the actual damage caused by these ministries outweighs any potential good?” This is a good question, but not the only question. A contrasting question might be “How many testimonies of significant and satisfying change, and how infrequent do the empirically documented evidences of harm have to be, before opponents of such change efforts might be willing to cede to these efforts a continuing right to exist as long as they operate with rigorous levels of informed consent?”
Chapman closes with a nod to the bigger picture: Sexual orientation, he asserts, is determined before birth, “no scientific study has successfully identified any postnatal causal factor or factors,” and therefore sexual orientation is immutable. We, in contrast, 1) would acknowledge that there is intriguing evidence of biological factors involved in causation of sexual orientation, but would also argue that the evidence is far from establishing complete biological determination of sexual orientation; 2) suggest in contrast to Chapman that there is intriguing evidence of postnatal factors in causation (see our pp. 122-125 as well as our previous publications); 3) argue that the establishment of partial biological causation does not in itself logically entail that orientation is utterly immutable for everyone, and 4) join with Lisa Diamond (see our response to Part 1) in concluding that “the more we learn, the more we do not understand.”
How would we present the bigger picture in contrast? Chapman’s review adds validity to our study. He asserts bluntly “sexual orientation cannot be changed,” and clearly feels that harm is so likely and likely so devastating (“How many lives must be broken?”) that there is no merit to the attempt to change. It was precisely to address these questions that we performed our study. Chapman ignores the data from our study that does not fit his conclusions. We believe that a fair read of our study produces a more difficult, complex, challenging set of conclusions (see Chapter 10), namely that: 1) change appears possible for some but not for all, and further this change is for some ambiguous, complicated, conflicted, and incomplete; 2) while harm may occur for some, on average the participants did not experience increased distress as a result of the attempt to change; and therefore 3) we would urge that individual consumers be empowered to make the best choices for themselves based on the best evidence and on full disclosure from multiple sources of information.
