A recent article in Educational Researcher shows that hardly any research experiments in the field of education are replicated by other researchers, casting heavy doubt on the veracity of their findings.

Charlie Tyson at Inside Higher Ed reports:

Almost no education research is replicated, new article shows

The word “replication” has, of late, set many a psychologist’s teeth on edge. Experimental psychology is weathering acredibility crisis, with a flurry of fraud allegations and retracted papers. Marc Hauser, an evolutionary psychologist at Harvard University, left academe amid charges of scientific misconduct. Daniel Kahneman, a Nobel-Prize-winning psychologist at Princeton University, entered the fray in 2012 with a sharply worded email to his colleagues studying social priming. He warned of a “train wreck looming” that researchers would avoid only if they focused more diligently on replicating findings. And the journal Social Psychology devoted its most recent issue to replication – and failed to replicate a number of high-profile findings in social psychology.

Yet psychologists are not the worst offenders when it comes to replication, it turns out. That distinction might belong to education researchers, according to an article published today in the journal Educational Researcher.

Only 0.13 percent of education articles published in the field’s top 100 journals are replications, write Matthew Makel, a gifted-education research specialist at Duke University, and Jonathan Plucker, a professor of educational psychology and cognitive science at Indiana University. In psychology, by contrast, 1.07 percent of studies in the field’s top 100 journals are replications, a 2012 study found.

Makel and Plucker searched the entire publication history of the top 100 education journals – ranked according to five-year impact factors — for the term replicat*. They found that 221 of 164,589 total articles replicated a previous study. Just 28.5 percent were direct replications rather than conceptual replications. (Only direct replications, which repeat an experiment’s procedure, can disconfirm or bolster a previous study. Conceptual replications, on the other hand, use different methods to test the same hypothesis.)

What’s more, 48.2 percent of the replications were performed by the same research team that had produced the original study. Attempts to replicate an experiment failed more often if there was no author overlap. When the same authors who published the original study published a replication in the same journal, 88.7 percent of replications succeeded. (The figure dropped to 70.6 percent when the same authors published in a different journal.) By contrast, replications conducted by new authors succeeded 54 percent of the time.