If you click on the link in the story, it takes you to the report where they describe the methodology:
tigger.uic.edu/~pdoran/012009_Doran_final.pdf
Target Population:
An invitation to participate in the survey was sent to 10,257 Earth scientists. The database was built from Keane and Martinez [2007], which lists all geosciences faculty at reporting academic institutions, along with researchers at state geologic surveys associated with local universities, and researchers at U.S. federal research facilities (e.g., U.S. Geological Survey, NASA, and NOAA (U.S. National Oceanic and Atmospheric Administration) facilities; U.S. Department of Energy national laboratories; and so forth). To maximize the response rate, the survey was designed to take less than 2 minutes to complete, and it was administered by a professional online survey site (
questionpro.com) that allowed one-time participation by those who received the invitation.
Response Rate:
With 3146 individuals completing the survey, the participant response rate for the survey was 30.7%. This is a typical response rate for Web-based surveys [Cook et al., 2000; Kaplowitz et al., 2004]. Of our survey participants, 90% were from U.S. institutions and 6% were from Canadian institutions; the remaining 4% were from institutions in 21 other nations. More than 90% of participants had Ph.D.s, and 7% had master’s degrees. With survey participants asked to select a single category, the most common areas of expertise reported were geochemistry (15.5%), geophysics (12%), and oceanography (10.5%). General geology, hydrology/hydrogeology, and paleontology each accounted for 5–7% of the total respondents. Approximately 5% of the respondents were climate scientists, and 8.5% of the respondents indicated that more than 50% of their peer-reviewed publications in the past 5 years have been on the subject of climate change. While respondents’ names are kept private, the authors noted that the survey included participants with well-documented dissenting opinions on global warming theory.
There are two weaknesses in the survey:
This is a population survey, meaning that the “universe” of people they were targeting is defined by the database of professions they used to get the names. Therefore, the authors should, but do not provide statistics of the population proportions for each group in the database. So we can’t tell if 5% is greater, lesser, or about the same proportion of climate scientists that exists in the population. If it is about the same, then 5% (N=77) is fine.
The overall response rate is not high (30.7%), which means the possibility exists that some bias may be skewing the results (i.e., some unknown factor that motivates one person to respond while another to ignore the survey). The question then becomes whether climate change skeptics are less likely than climate change believers to respond to the survey. My guess is that they are not.
On the plus side, the effect size is
huge. Very seldom do you get such a lop-sided result. Even if response bias accounts for 20% of the responses, you still have more than three quarters of climatologists (77%) agree that human activity is causing global warming.
It may not be the best piece of survey research, but it is certainly better than the totally unscientific claims offered by the climate change deniers.