Dr. Chapman gives his concluding remarks in a series which began with his three part critique of Ex-gays?: A Longitudinal Study of Religiously Mediated Change in Sexual Orientation by Dr Stanton Jones and Dr Mark Yarhouse. What follows is a response to the author’s comments on the original critique. We thank all three for their participation.
I appreciate Dr Stanton Jones’ willingness to exchange comments on his recent co-authored book and for the spirit of collegiality that is represented in the exchange. Bias is an issue that all researchers are subject to and strive to overcome. It is sometimes difficult to separate one’s emotions from an objective critique of a work that hits close to home. As such, one can sometimes accidentally misrepresent aspects of a study. I appreciate the minor corrections Stanton Jones provided regarding my original critique. However, we remain in discord regarding the substantive points discussed.
In science it is important to be precise, something the Jones and Yarhouse study lacks. The focus of my original critique was the sloppy and biased scholarship, demonstrated partly by imprecision. I highlighted that Jones and Yarhouse:
- reference the entire study as prospective when it is not;
- assert participants are at the start of the change process instead of just being at the start of the Exodus program, although even this is not entirely correct;
- assert the study cannot ascertain if permanent and enduring long-term change occurs, contrasted with their conclusion that change is possible;
- portray specific results as indicating change, specifically Tables 7.4 – 7.6, when those actual results do not support the conclusion;
- employ a moving target regarding what sexual orientation change entails;
- misstate the APAs’ position on the potential harm of reparative therapy;
- and have a moving target regarding if the therapy causes harm.
I would like to further discuss each of these points, addressing the rebuttals provided by Jones in his response. Before I do so, however, I wish to show the sloppy and biased scholarship is not unique to this book; it also exists in Jones and Yarhouse’s previous work, Homosexuality: The Use of Scientific Research in the Church’s Moral Debate.
In that book the authors discuss the relevance of twin studies to causation. They correctly present the concordance rate for female identical twins in the Bailey and Pillard study as 48%. Based on how concordance rates are calculated, in a representative sample of 30 pairs of identical twins one would expect both co-twins to be homosexual in only nine or ten cases. This means that even if the first 20 pairs in a representative sample demonstrate negative concordance, one could still obtain the target concordance rate. However, to counter the 48% concordance rate the authors reference a study with a sample size of four, in which in no cases were both co-twins homosexual. They then state: “The lack of shared homosexual preference for female homosexuals is quite striking when compared to Bailey and Pillard’s remarkably high rate of shared homosexual preference for lesbians” (2000:74; emphasis added). For PhDs in the behavioral sciences to make such a declaration using an incredibly small and non-representative sample is “quite striking” and cannot be attributed to an innocent ignorance of statistical theory. The sloppy scholarship, rooted in what I believe is a predisposed religious bias, is also a significant issue in the current study.
Prospective vs. retrospective and what do the results say?
Jones and Yarhouse claim their study is “prospective.” In my critique I argued: “technically the study is not prospective because 41 individuals were involved in the Exodus program for one to three years prior to the study.” Jones responds that my logic is not compelling and suggests: “If the reader insists on a tighter understanding of “prospective,” then you can narrow the focus to the Phase 1 results. These results were not as positive as those for the population as a whole, but were still statistically significant and meaningful.” I believe even this is not appropriate and once again misrepresents the results.
When I desire to quantify how much my students learn I give them the same test on both the first and final days of class. I do not wait until after one month of lectures before doing the initial assessment, asking the students to think back to the first day and base their answers on their knowledge at that time. The month in class would bias the results. Similarly, although Jones argues the Phase 1 participants are part of a prospective study, I am not convinced this is the case. Phase 1 participants were involved in the Exodus program for up to one full year prior to initial assessment. Thus, statements regarding their starting condition are still retrospective: they had up to one year to internalize the teachings, stereotypes, and views of the Exodus ministry before their starting condition was assessed. For the study to be truly prospective it is necessary to interview participants as they enter the program and not wait up to one year after their participation commenced. Jones and Yarhouse make an admirable attempt to make the study prospective but they fall short of the mark. I recognize a truly prospective study is difficult to achieve, but I do not believe one should claim a study to be prospective unless it is entirely so.
Meanwhile, Jones criticizes me for “selectively” viewing the results when I focused on Tables 7.4 – 7.6, which examine how the participants view their sexual orientation. My use of these tables was to provide an example of how Jones and Yarhouse declare positive progress to have occurred based on specific results, even though the results they reference showed no progress. Jones’ error is repeated when he suggests the Phase 1 results were “statistically significant and meaningful.” This is important because the Phase 1 participants represent the supposedly prospective component when one uses a “tighter understanding” of prospective.
As Timothy Kincaid indicated, when one examines the results for the Phase 1 participants it is clear, contrary to Jones’ declaration, there is little to no statistically significant or meaningful change. Specifically, while Table 7.2 (p. 239) indicates there is a large change in how Phase 1 participants self-identify (“heterosexual” increased from 7 to 14, “homosexual” remained unchanged at 9), the self-reported sexual orientation (Table 7.5, p. 240) shows no significant change (“heterosexual” decreases from 5 to 4, “homosexual” decreases from 25 to 24). The self-identify results are not surprising given the teachings of the Exodus program.
Phase 1 results (Table 7.8, p. 258) for Kinsey 1-item (measuring sexual attraction), 2-item (averaging measures of sexual attraction and sexual behavior), and expanded (averaging measures of sexual attraction, behavior, emotional and romantic attraction, and fantasy), in addition to the Yarhouse thermometer (Table 7.14, p.273) for both “heterosex” and “homosex,” all demonstrate no significant change occurred.
The Shively and DeCecco Phase 1 results (Table 7.11, p. 266) demonstrate no significant change for “heterosex” but a small to medium change for “homosex.” While the Shively and DeCecco measure normally separates out physical preference (sexual attraction) from affectional preference (emotional attraction; pp. 214, 224), Jones and Yarhouse combined those two measures (pp. 224-227) and added measures for intensity of attraction and frequency of attraction. They also explicitly state “these expanded Shively and DeCecco ratings must be regarded with caution because validation work was not done on them beyond pretesting with our pilot subjects” (p. 225; emphasis added). Thus, with the small to medium change reported in the results, it is hard to know what changed, the physical attraction, desire and arousal, or something else, or even if the changes have any scientific meaning.
The Klein 4-item results (Table 7.13, p. 270) also demonstrate small to medium significant change. However, once again it is unclear what actually changed. The Klein composite score is constructed from sexual attraction, sexual fantasies, emotional preference and social preference (p. 228). As with the Shively and DeCecco results it is unclear if the sexual orientation changed, or if the results indicate only the social and emotional preferences of the participants changed.
Throwing caution to the wind, Jones claims the results are “statistically significant and meaningful” evidence of sexual orientation change. However, the results are neither consistent nor compelling, especially if one seeks a change toward heterosexuality. While two measures, one of which has not been scientifically validated, indicate a reduction in homosexual components, it is possibly only the social preference that changed, which would not be surprising given the teachings of Exodus and the self-identify results presented above. Contrary to Jones’ declaration, in their entirety the results do not indicate “statistically significant and meaningful” change in sexual orientation occurred for Phase 1 participants. Thus, the anecdotal evidence provided by the participants, highlighted in my original critique, are important in suggesting sexual orientation, defined by actual attraction, desire and arousal, did not change.
Start of the change process?
Jones and Yarhouse regularly state the participants were “starting the change process.” In reality they were only near the “start of the Exodus program,” given that many of the participants had previously taken “concrete steps” to change through counseling or other religious-based therapies. Jones regards my criticism as somewhat nit-picky, claiming he and Yarhouse were implicit and explicit on the participants starting the change process through Exodus. Unfortunately, they relied on the implicit assumption much more than explicit statements.
Jones essentially argues that previous attempts to change through counseling and other religious based ministries do not impact the results of the current study. However, as someone who actually tried various types of counseling and ministries before entering an Exodus-related program as a final hope for change, I disagree. The methods, explanations, and teachings provided in my Exodus-related program were similar to what I experienced in previous attempts at change. Recognizing this early in my experience in the Exodus-related program, I embraced a false dilemma: either I had been doing something wrong in my previous attempts or God rejected me (as an evangelical I believed that God would help those who attempted to do God’s will, with heterosexuality supposedly God’s desire for all people). Thus, I was more concerned than ever about identifying as heterosexual, more earnest in dating girls and in convincing myself of the heterosexual desires that were supposed to be developing. For the record, I had been celibate before and throughout the entire process, thus it appears I would have been declared “Success: Chastity” by the Jones and Yarhouse study even though no change in desires or behavior ever occurred!
I am not convinced the participants were “starting from scratch,” many of them may have been in a situation similar to mine. Regardless, Jones and Yarhouse should have consistently and explicitly stated the individuals were at the “start of the Exodus program” (although this is not entirely accurate because some of the participants had been in the program for up to three years at the start of the study) and not at the “start of the change process,” for the latter is incorrect and simply not good science.
Long-term change?
Jones defends their calling the study “long-term”: “Our point was not that our study was not a long-term study, nor that our study was inadequate to produce evidence suggesting that change was not impossible. Our point instead was that if you want to show that change is permanent, then logically you have to study subjects throughout their lifespans to death to insure the change was permanent. So our study cannot show that change is permanent, but even so a three to four year span of time is scientifically meaningful and qualifies as ‘long term.’”
Based on anecdotal stories of false reports, I believe a study lasting three or four years is insufficient to determine whether or not meaningful change in sexual orientation, defined by attraction, desire and arousal, has actually occurred or if an individual is deceiving him- or herself. The inadequacy of the short time frame is demonstrated by the individual who recanted his claims when the Jones and Yarhouse manuscript neared completion.
Jones argues the stringent requirements I demand are not normally done in behavioral research. Fair enough. However, the situation with homosexuality is very different from situations he uses for comparison: most people do not consider continued depression or marital strife as important in determining one’s eternal salvation, nor are they told that such conditions do not even exist, as some ex-gay leaders have argued about homosexual orientation. I argue the situation with sexual orientation requires a different set of standards, ones that are much more stringent.
Does the therapy cause harm?
As indicated in my original critique, Jones and Yarhouse use inconsistent and contradictory terminology when discussing if the program causes harm. Thus, it is difficult to understand what the conclusions are because the targets constantly shift: the concluding statement does not reflect the stated goal. It is also difficult to assess if the results refute what the American Psychological and American Psychiatric associations state regarding the potential harm of the therapy. In his response Jones indicates they were examining if the therapy causes harm “on average.” I appreciate the clarification, but restate this is not what the APAs suggest nor what Jones and Yarhouse explicitly present as one of the two goals of the study.
Nonetheless, how can one speak of averages when about one-fourth of the participants dropped out of the program and were not included in the results? This is akin to having soldiers returning from war fill out a personal questionnaire and then declare that, on average, the results indicate soldiers do not experience life-threatening injuries in battle. Of course, the soldiers who died in battle were not included in the study because they did not fill out the questionnaire. A study examining “average” harm must include all of the participants in the study who began the program; otherwise the results are rendered useless by a biased sample.
What does sexual orientation change entail?
Jones criticizes me for dismissing the “conclusions about those who experienced a decrease in the potency of their homosexual desires and were able to embrace chastity, and who themselves considered this a successful outcome to the change process.” This is an important point. Let me be clear, I respect the individuals who embrace chastity to reconcile their dilemma. However, my critique is not concerned with whether the individuals simply change their behavior or consider their outcome to be successful, or even if their homosexual desires rage as strongly as before, but whether or not their sexual orientation actually changed. Change in sexual orientation is a primary goal of the study; determining if participants change their behavior or are satisfied with the program are not presented as such. As discussed earlier, the results for the supposedly “prospective” Phase 1 participants do not demonstrate that sexual orientation changed in any statistically significant or meaningful way.
Jones and Yarhouse define homosexual orientation by “the experience of same-gender sexual desire” and “on the psychological component of attraction, desire or arousal” (p. 209; emphasis in original). Allow me to restate their conclusion (p. 387), replacing “homosexual orientation” with their definition: “we believe we have provided evidence that change of [the experience of same-gender sexual desire, determined by attraction, desire or arousal] may be possible through involvement in Exodus ministries.
[1.] “The change may take the form of … behavioral chastity”
[2.] “The change may take the form of a reduction in homosexual attraction”
[3.] “The change may take the form of the form of … a reduction in homosexual attraction and an increase in heterosexual attraction.”
Regarding point 1, behavioral chastity is not a demonstration that same-gender attraction, desire or arousal has changed. It only indicates the individual’s response to the attraction, desire, or arousal has changed. For Jones and Yarhouse to declare this as evidence of “change” is to shift how they define sexual orientation.
Regarding point 2, a reduction in homosexual attraction may alter the “experience” of same-gender sexual desire in quantity, but not in kind. Let us consider this point using Jones and Yarhouse’s analogy of depression. On a scale of 1-10 where 1 is not depressed, 2 is slightly depressed and 10 is severely depressed, if a person moves from 10 to 8 do we now declare they are “ex-depressed”? Or even if they move from 10 to 2, can we now say they are “ex-depressed”? In each case the answer is no, the individual is still depressed: the intensity diminishes but the depression continues. To argue a reduction in attraction demonstrates sexual orientation change is to shift the definition of sexual orientation: the individuals are still same-sex attracted, they still desire the same-sex, and they are still aroused by the same-sex: they are still homosexual and have not changed their sexual orientation.
Regarding point 3, a reduction in homosexual attraction combined with an increase in heterosexual attraction alters the “experience” of same-gender sexual desire in kind, but not necessarily by eliminating the original condition. The individual is “ex-gay” only in the sense that he or she is now “bisexual” in attraction, desire or arousal. However, there is the legitimate question of whether the heterosexual attraction is now generalized or is directed solely to one specific person, as is reportedly often the case. In other words, are these individuals now “turned on” by the opposite sex in general, or is the heterosexuality represented by their now being aroused solely by a spouse. If the heterosexual attraction is individual-specific while the homosexual attraction is general, then one must question what actual change occurred. Unfortunately, the evidence provided by the authors is inadequate to assess this situation. Thus, the declaration of change in this context works to a certain extent, but not to the extent such declarations are used by ex-gay advocates, especially recognizing this represented a minority situation in the study and one of the supposedly successful individuals later recanted his claim of success.
Ex-gay ministries regularly advertise conferences saying: “you don’t have to be homosexual, change is possible.” The average American interprets this to say that people can change their orientation. Meanwhile, ex-gay advocates primarily or solely mean a person’s identity can change, their behavior can change, or how they feel about their homosexual attraction, desire and arousal can change. Yet, when ex-gay advocates write newspaper editorials, advertise in billboards, etc, they don’t explain what the change entails, even though they presumably know the public understands the concepts differently. But imagine how the public would respond if ex-gay advocates honestly declared, based on the conclusions presented by Jones and Yarhouse:
“You don’t have to be homosexual, you can change how you feel about your homosexual attractions.”
“You don’t have to be homosexual, you can continue to have same-sex attractions.”
“You don’t have to be homosexual, you can be celibate.”
“You don’t have to be homosexual, you can be bisexual.”
I think the response would be derisive laughter. Although such declarations would not be politically expedient, at least they would honestly represent the supposed change. The only way ex-gay ministries and advocates can sell their message is by using imprecise and deceptive language, by obscuring or shifting what change in sexual orientation actually means, or by misrepresenting results. Unfortunately, these are all evident in the Jones and Yarhouse study.
I do not accept Jones and Yarhouse’s conclusion that change occurred because I am using their definition for what constitutes sexual orientation. Jones and Yarhouse’s study reminds me of a two-panel cartoon I once saw. The first panel consists of scientists examining the fossil evidence, declaring: “here we have the facts, how can we incorporate them into a meaningful theory?” Meanwhile, the second panel featured creation science practitioners holding a Bible, declaring: “here we have the facts, how can we make the evidence fit?” This is precisely what ex-gay advocates do, and they achieve it through deceptive language, misrepresenting what change entails, and using misleading data. It is time Jones and Yarhouse put aside their bias and objectively and honestly examine the evidence. The results of the most “prospective” element of their study, further supported by anecdotal comments, provide no evidence of change in sexual orientation, defined by attraction, desire or arousal.
—
Patrick M. Chapman has a PhD in biological anthropology and is author of “Thou Shalt Not Love”: What Evangelicals Really Say to Gays (Haiduk Press: in press).
The portions of the limbic system which produce sexual attraction, desire and arousal have been around since the age of the reptiles, over 300 million years ago. The idea that by taking thought one can alter these primitive responses is just nonsense.
Thank you, Dr Chapman for continuing the dialogue. I hope to see J&Y give another response. I criticized the study for not being peer reviewed, but this is an attempt to compensate.
As an adjunct professor, when I taught Bachelor’s level business students the art of creating surveys, as well as how to statistically interpret the results, I requested that they honestly state their first hypothesis, and then later, note what the evidence seemed to restate as a possible new set of hypotheses.
Thus, business surveys are not just to prove or disprove hypothesis, but to provide insight into new directions. Chapman stated “In science it is important to be precise”, and he is correct. But allow me to apply business modeling to the Jones and Yarhouse study.
I applaud that the Jones and Yarhouse study shows their first hypothesis. And considering the political environment in the USA, as well as the church environment, I was amazed at the honesty of the second series of possible new hypotheses that were expressed by Jones and Yarhouse as “conclusions”. It would have been far safer politically and religiously for them to never have published at all. Their “conclusions” follow:
This set of conclusion yields a number of new questions worthy of further study: what definitions of change should be used? What do we do with “change appears possible for some but not for all”? What do we do with ambiguous, complicated, conflicted, and incomplete change? Why did harm occur for some?
Continuing with applying business models, in Who Moved My Cheese? by Dr. Spencer Johnson, the character Haw writes on the wall of Cheese Station N the following:
– Smell the cheese often so you know when it is getting old
– The quicker you let go of old cheese, the sooner you can enjoy new cheese
– Move with the cheese.
A time ago, the ‘cheese’ of Christian divorce moved. Remember mantras like “If they had been really Christian, they would have never divorced” and/or “Christ ordained only one marriage for one lifetime” ? Remember the slow realization that approximately 50% of Christian marriages ended in divorce; many divorced persons remarried; and many walked in the mercy of God, and lived productive and Christ-like lives? Some churches refused to see that the cheese moved; others began ‘divorce recovery workshops’; still others welcomed remarried couples into church membership; still others allow pastors to continue in ministry following a divorce. At first, the liberal churches took the lead; at this time, I see almost as many liberal as conservative churches welcoming remarried couples.
The ‘cheese’ of the ex-gay movement has, in my opinion, moved. Remember mantras like “If they had only really prayed/fasted/had faith, they would never become (or stayed) homosexual” and/or “Christ ordained only heterosexuality for this lifetime”? The new cheeses may well be chastity, living with conflict, living with self-acceptance, and/or gay marriage.
I do not expect a scholarly work when the ‘cheese’ has moved. I expect a work that struggles to find new terms, new definitions, and new directions. While Chapman notes “sloppy and biased scholarship”, I note authors struggling with the concept that ‘cheese’ is moving, or even searching for terms to describe that the ‘cheese’ has already moved.
I hope and pray for follow-on studies to find the ‘cheese’. We that are Christians have an obligation to provide healing to the wounded over all other differences, per Luke 10, The Parable of the Good Samaritan… if the old mantras do not provide healing, then we need to find new ones.
The beauty of the Jones and Yarhouse study is that they show that the cheese has moved.
Thanks, Caryn, for your cheesy remarks. (Ugh, SAT, I couldn’t help it!)
The pastor who baptized me 65 years ago remained as pastor after his divorce. He was removed as pastor, however, when he remarried several years later. Our ideas about this have changed considerably since then, the cheese has moved, as you say.
On the Charisma online forum, this was part of a post by “Frankie:”
So I guess you might say that “Frankie” and I believe in moving the cheese once again.
Bill
Well, I see the Charisma forum has “lost” the last three months worth of posts, I hope temporarily. So you may have to wait and try the above link again later.
I posted a comment to one of your earlier critiques, but again, I have to thank you for your thoughtful and articulate responses, Dr. Chapman! As I commented last time, though it merits repeating, Jones & Yarhouse frequently misrepresent the standards of psychotherapy research, which are actually quite rigorous in peer-reviewed journals. The norm has been moving toward an “intent-to-treat” analysis – that is, inclusion of outcomes for those who drop out of treatment – over a “survivor analysis” of only those that remain. Survivor analysis creates a “low bar” by which the best responders and those sympathetic to the research are overrepresented. J&Y not only continue to overstate their case, but it has been frustrating to see them continually attempt to align themselves with standard behavioral science researchers. I’m glad to see your contributions to this panel, as the few researchers that contribute often exacerbate confusion over what appropriate behavioral research looks like.