Getting pro-life science right: an example of bad science

[fnr]

One of my fundamental objectives as a way to reduce the incidence of abortion, and someday eliminate it is to convince the pro-life movement to adopt rigorous and objective standards for the conduct, publication, citation, review, and summary of scientific information. At present, there is woeful bounty of cherry-picked research on the web sites and mailings of many pro-life organizations, and the academics whose work the pro-life movement often cite often publish methodologically questionable studies in second or third-rate journals. As a professional health scientist who is also Catholic and pro-life (though differing in strategy from many judicially-oriented pro-life organizations), I view this dearth of sound methodology and objective treatment of evidence as an embarrassment.

My objective with this post is not to simply criticize the pro-life movement. I am not trying to talk down to anyone, I am trying to help (though I understand the paternalism inherent in what I’m writing, so I very much apologize). I would like to make a good-faith effort to demonstrate why someone like me, who understands statistics very well and routinely reviews articles for peer-reviewed scientific journals, views the scientific basis of the pro-life movement as problematic and unable to “cut the mustard” within the broader public health and medical literature.

Before saying anything else, I do want to highlight a number of research articles by authors I view as being methodologically rigorous.

• First, there are a number of recent studies by Elard Koch and colleagues, Chilean researchers who have used rigorous mortality registry data to broadly critique survey-based estimates of abortion and abortion-related mortality developed by researchers at the Guttmacher Institute. In my eyes, they have pretty definitively demonstrated that the Guttmacher methods for estimating international abortion rates are extremely biased, at least in the nations in which Koch and colleagues have looked. I used to cite the Guttmacher papers, and have been pretty soundly convinced by Koch and colleagues.
• Second, there are a series of studies by Jonathan Klick, who has examined how sexually-transmitted infection (STI) rates change as a result of policies such as legalizing abortion and parental notification laws for minors. He’s also looked at the mental health impacts of mandatory waiting periods. His work is of particular importance in challenging the “baked in assumption” of many abortion advocates that sexual behavior does not change. That’s one of the key assumptions included in the contraception chapter of Institute of Medicine Report, Clinical Preventive Services for Women, on which the HHS contraceptive mandate was based.
• A number of other researchers, such as Elizabeth Oltmans Ananat (the “Power of the Pill for the Next Generation” paper) and Kearney (the MTV 16 and Pregnant study, among others), have published evaluations of how media can influence abortion rates and the effects of “the pill” on children’s family economic status.

Below, I link to one free, online article authored by David Reardon (author of Aborted Women, Silent No More, a book I see as being so methodologically flawed as to be unusable for objective assessments of abortion’s impacts on women) and Priscilla Coleman. I try to explain why I think the authors over-interpret their own results and show that their presentation makes their conclusions impossible to accept.

 

[fnr]

Here’s a link to the paper by Reardon and Coleman. Here’s the reference.
Author: David C. Reardon and Priscilla K. Coleman.
Title: Short and long term mortality rates associated with first pregnancy outcome: Population register based study for Denmark 1980–2004
Reference: Medical Science Monitor 2012; 18(9): PH71-PH76.

Here is the abstract of the study:
Background: There is a growing interest in examining death rates associated with different pregnancy outcomes for time periods beyond one year. Previous population studies, however, have failed to control for complete reproductive histories. In this study we seek to eliminate the potential confounding effect of unknown prior pregnancy history by examining mortality rates associated specifically with first pregnancy outcome alone. We also examine differences in mortality rates associated with early abortion and late abortions (after 12 weeks).
Material/Method: Medical records for the entire population of women born in Denmark between 1962 and 1991 and were alive in 1980, were linked to death certificates. Mortality rates associated with first pregnancy outcomes (delivery, miscarriage, abortion, and late abortion) were calculated. Odds ratios examining death rates based on reproductive outcomes, adjusted for age at first pregnancy and year of women’s births, were also calculated.
Results: A total of 463,473 women had their first pregnancy between 1980 and 2004, of whom 2,238 died. In nearly all time periods examined, mortality rates associated with miscarriage or abortion of a first pregnancy were higher than those associated with birth. Compared to women who delivered, the age and birth year adjusted cumulative risk of death for women who had a first trimester abortion was significantly higher in all periods examined, from 180 days (OR=1.84; 1.11 <95% CI <3.71) through 10 years (1.39; 1.22 <95% CI <1.61), as was the risk for women who had abortions after 12 weeks from one year (OR=4.31; 2.18 <95% CI <8.54) through 10 years (OR=2.41; 1.56 <95% CI <2.41). For women who miscarried, the risk was significantly higher for cumulative deaths through 4 years (OR=1.75; 1.34 <95% CI <2.27) and at10 years (OR=1.48; 1.18 <95% CI <1.85).
Conclusions: Compared to women who delivered, women who had an early or late abortion had significantly higher mortality rates within 1 through 10 years. A lesser effect may also be present relative to miscarriage. Recommendations for additional research are offered.

So first, what’s right with the study? Well, first, the authors published it in a peer-reviewed journal. That in itself is a big deal. Second, the study is very large! It is based on Danish national registries for abortion, births, stillbirths, and deaths. As a result, the study was able to include 463,473 women who had at least one pregnancy between 1980 and 2004, 2,238 whom had died during the period of follow up. With such a large population, the study seems to have had sufficient statistical power (the ability to detect significant differences) for evaluating the hypotheses posed by the investigators.

 

[fnr]

What are the problems with the study? The biggest is one that the investigators acknowledge as a limitation:
“…our analysis does not control for socioeconomic factors, marital status, psychological history, or other factors prior to first pregnancy which may affect the subsequent risk of death.”

The importance of this limitation cannot be overstated. There is enormous potential for confounding by socioeconomic status, in particular. “Confounding” takes place when there is a reported association between two variables, say A and B, but the unreported variable C is correlated with both A and B. Imagine that in Poland, between 1900 and 1950, A represents the number of people who died of starvation and B represents the annual consumption of vitamin B across all Poland. It would not be surprising to say that A and B seem associated. When the country consumed a lot of Vitamin B, very few people die of starvation. When the country consumed very little Vitamin B, a lot of people died of starvation. From these two pieces of information, one might be tempted to conclude that a lack of Vitamin B causes starvation. However, this would not be a correct conclusion, because C, the total amount of food produced or imported into Poland minus the food exported from the country, is correlated with both A and B. When Poland has a lot of food, people don’t starve. When Poland has a lot food, people consume a lot of Vitamin B. So here, we have an example of how C, the total food available in Poland, is a confounder of the relationship between Vitamin B consumption and the number of deaths by starvation. Looking closer, one might find that the correlations were driven by years around World War 2, when Nazi Germany instituted the “Hunger Plan,” intended to feed the German Army with the agricultural produce of the Slavic lands it occupied. Without accounting for total food available, which the Nazis curtailed, one might foolishly think the solution to Poland’s starvation problem was to drop vitamin capsules all over the country.

Women who have abortions are also disproportionately lower in income than average, with the about 42% of U.S. abortions occurring among women making less than 200% of the Federal poverty line in the, and 27% among women with incomes between 100-199% of the Federal poverty line. I got these data from the notably abortion-friendly Guttmacher Institute [Characteristics of U.S. Abortion Patients, 2008 | Guttmacher Institute]](http://www.guttmacher.org/pubs/US-Abortion-Patients.pdf]), which nevertheless publishes data of generally high quality.

Women who have low incomes also have higher mortality rates, and counties with lower median household incomes have higher mortality rates. Here’s a graph from a CDC report


So right there is an enormous confounder for which the Reardon and Coleman did not account. To me, as someone who reviews conducts and reviews studies using similar statistical methods, this lack of control of confounding for income alone makes the study almost entirely worthless.

In that the authors were using national registry data from a highly industrialized European nation, it would not have been that difficult to control for socioeconomic status, at least by using information about the location of each medical event. In that there municipal average income levels published for Denmark (which I easily found online), it would not have been difficult to assign each death, birth, or abortion event with an associated municipality’s average income. This approach would be a form of “semi-ecological” control for confounding. I’ll grant that working with large health databases can be difficult and time consuming, particularly if published in a foreign language. But without such control of confounders, the statistical results are almost worthless. I am left unconvinced that abortion increases the mortality rate in women who receive one.

I’m hoping that this example provides an illustration of how poor science seriously undermines the objectives of the pro-life movement.

The Guttmacher Institute knows and understands the importance of science, and their research has had an enormous influence over the health intelligentsia in the U.S. (and Justice Ginsburg, in her dissent in the Hobby Lobby case). Their statistics dominated the section on contraception in the IOM report on clinical preventative services for women, while a lack of attention to quality science by critics of contraception gave the report a “free pass,” which left only the dissent in Appendix D to stand against the major recommendations. The IOM’s selective citation of literature favoring contraception, and omission of NFP studies published during years similar to those that were cited in the report, is a travesty of science, one that only the resolution Hobby Lobby case was able to (partly) counter.

It is time that the pro-life movement stopped relying exclusively on lawyers and elections to spread the Gospel of Life. It is time for rigorous, objective, and peer-reviewed science to take its place as one of the central tools in the pro-life arsenal. This will be a difficult change, I admit, but to me it’s the only path to reducing abortion to a significant degree.

 

[AussieTrish]

Thank you.
I’ve bookmarked your valuable information for future reference

 

[TarkanAttila]

Someone needed to say it. Thank you for saying it.

 

[Petaro]

Just from a cursory reading, it becomes apparent that we pro-lifers must be more scientific in any report purporting to use statistics (the greatest liar since the devil).
I sincerely doubt that these attempts at scientific analysis of the issue has any pulling power amongst those intellectually committed to abortion as a right, nor, unfortunately, to the poor women whose circumstances lead them to this terrible sin.
It is like climate science, a tool for the fanatic not a proof for the discerning intellect.

 

[Bobby_Jim]

Thank you, this is something that has bugged me for years (use of cherry-picked science). It’s of course not just the pro-life movement that does it. Ultimately I think it is a moral issue and the most compelling arguments are moral arguments about the dignity of all human life. But if statistics and scientific arguments are used to bolster the moral argument, it has to be done carefully, lest we sound foolish.

It’s very easy to fall into this, of course. Confirmation bias, I think they call it. I know or believe (A) to be true, so I will incorporate or interpret new data to support (A) but much more easily ignore or explain away data that doesn’t support (A). It happens in all fields. The problem is that someone who doesn’t already believe (A) can easily see through the misrepresentations, which undermines your credibility even if you are, in fact, right (which is the case with pro-life).