T
Teflon93
Guest
As a stats guy, I have to caution people using terms “direct correlation” regarding such public policy issues, much less deriving causation from it.
The first challenge is in measuring the amount of sex education in schools. These kind of numbers are hard to come by. Does one measure hours of instruction, pages of instruction, amount retained from instruction, or some other metric?
Abortion numbers also tend to be obscure. In America in particular, there is resistance to publishing actual numbers of abortions. What numbers we have are necessarily speculative.
Then you’ve got to measure these two metrics over the same period of time and calculate the correlation coefficient between them.
Even when you get a strong correlation, causation still has not been proven (when teaching this, we use the example of the number of butterflies and number of hurricanes to note the absurdity of an interpretation that butterflies cause hurricanes).
Causation is proven (to the extent it can be in public policy) through observation and experimentation, both of which are quite difficult.
All of the above are reasons why one should be very, very skeptical of public policy studies which claim to “prove” something. It is easy to claim such a thing, but invariably one finds glaring problems in methodology which severely limit if not overturn such claims.
I had a law professor friend whose colleague undertook a study claiming to prove that racism drove death penalty convictions in Texas. Once I actually saw their data, it became very clear that he had saturated his model with 27 variables (“everything but the kitchen sink”) and failed to assess the proper fit (how much of the data the model explained) by eliminating insignificant variables first. When corrected, his model explained a tiny minority of the data, refuting his premise and indicating he’d missed some variables (such as prior convictions).
The point of which is that we need to be careful in the claims we make and the evidence we muster to support them.
The first challenge is in measuring the amount of sex education in schools. These kind of numbers are hard to come by. Does one measure hours of instruction, pages of instruction, amount retained from instruction, or some other metric?
Abortion numbers also tend to be obscure. In America in particular, there is resistance to publishing actual numbers of abortions. What numbers we have are necessarily speculative.
Then you’ve got to measure these two metrics over the same period of time and calculate the correlation coefficient between them.
Even when you get a strong correlation, causation still has not been proven (when teaching this, we use the example of the number of butterflies and number of hurricanes to note the absurdity of an interpretation that butterflies cause hurricanes).
Causation is proven (to the extent it can be in public policy) through observation and experimentation, both of which are quite difficult.
All of the above are reasons why one should be very, very skeptical of public policy studies which claim to “prove” something. It is easy to claim such a thing, but invariably one finds glaring problems in methodology which severely limit if not overturn such claims.
I had a law professor friend whose colleague undertook a study claiming to prove that racism drove death penalty convictions in Texas. Once I actually saw their data, it became very clear that he had saturated his model with 27 variables (“everything but the kitchen sink”) and failed to assess the proper fit (how much of the data the model explained) by eliminating insignificant variables first. When corrected, his model explained a tiny minority of the data, refuting his premise and indicating he’d missed some variables (such as prior convictions).
The point of which is that we need to be careful in the claims we make and the evidence we muster to support them.