Data Availability Policy: American Economic Review

As announced in my previous blogpost, I ‘m starting the presentation of some data availability policies and replication policies with the American Economic Review (AER). The AER is a flagship of the economic profession and one of the top ranked journals in this scientific discipline.
The AER was published in 1911 for the first time. Only 7 – 10 percent of the submissions are accepted and later on published.

The AER adopted a so called replication policy in 1986 – despite the fact that studies (for example by Dewald, Thursby and Anderson) already claimed, that a replication policy is not enough to promote replicable results.
In their policy, the Review pledged authors to provide datasets and code for processing the data to other scientists that are interested in replicating the results on request.

Replication policies have often failed, even if the corresponding author is willing to support other researchers…and I imagine that this szenario is not very common …After publishing an article, authors mostly don’t have incentives to prepare the data and code for other researchers. It costs time and the rewards the scientific system pays for sharing data often are marginal.

Read the rest of this entry »

Nature Magazine Special Issue: Data Replication & Repoducibility

The nature magazine has just published a special issue about data replication and reproducibility.

In their introduction the authors are claiming that replication is considered the scientific gold standard. To give a broader view about replication, the journal explores some of the issues associated with the replication of results in different scientific disciplines as for example primate cognition and behaviour research, computer sciences, biology and climate change studies.

Worth reading is the Editorial by J. Crocker and L. Cooper that is dealing with the fraud of Diederik Stapel and raises the question “what could be done to protect science and the public from fraud in the future?”. The answer of the authors, both psychologists, is:

“Greater transparency with data, including depositing data in repositories where they can be accessed by other scientists […], might have sped up detection of this fraud, and it would certainly make researchers more careful about the analyses that they publish.”

Read the rest of this entry »