New paper highlights replication studies in economics

Four researchers associated with the former project EDaWaX have recently published a new research article entitled “Replication studies in economics – How many and which papers are chosen for replication, and why?” The article, written by Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff and Gert G. Wagner, sheds light on replication practice in empirical economics. The article can be found here (published in Research Policy under a Creative Commons license). Here, they provide a brief overview of the paper:

Read the rest of this entry »

KNAW recommends making replication studies a normal and essential part of science

Over the past few years several systematic series of replication studies have been unable to reproduce many important scientific findings in a whole slew of disciplines. This has led to a debate within the scientific community about the way science is currently being conducted and the role of replication studies.  Our blog also published many posts which deal with this question.

Now, KNAW (the Royal Netherlands Academy of Arts and Sciences) has published a report, which analyses the causes of non-reproducibility, assesses the desirability of replication studies and also offers recommendations for improving reproducibility and for conducting replication studies. Read the rest of this entry »

The replication crisis in economics – and how we might answer it

On September 8, 2017 the ZBW Leibniz Information Center for Economics hosted the workshop “Replications in Empirical Economics – Ways out of the Crisis” at the Annual Conference of the Verein für Socialpolitik in Vienna, Austria. Thirty participants and four speakers engaged in lively and stimulating discussions about replications and the publication of replications in Economics.

This is a cross-post of the weblog of the replication network. Read the rest of this entry »

Against the replication crisis: New international journal encourages replication studies

Read the rest of this entry »

New Working Paper published: ‘The Economics of Replication’

Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff and Gert G. Wagner have published a new working paper on the ‘economics of replication’.

In the abstract of the paper, the researchers state:

“Replication studies are considered a hallmark of good scientific practice. Yet they are treated among researchers as an ideal to be professed but not practiced. To provide incentives and favorable boundary conditions for replication practice, the main stakeholders need to be aware of what drives replication. Here we investigate how often replication studies are published in empirical economics and what types of journal articles are replicated. We find that from 1974 to 2014 less than 0.1% of publications in the top-50 economics journals were replications. We do not find empirical support that mandatory data disclosure policies or the availability of data or code have a significant effect on the incidence of replication. The mere provision of data repositories may be ineffective, unless accompanied by appropriate incentives. However, we find that higher-impact articles and articles by authors from leading institutions are more likely to be subject of published replication studies whereas the replication probability is lower for articles published in higher-ranked journals.”

The paper is available here.

Read the rest of this entry »

Dutch research funder grants 3 million Euros for replication studies

The Netherlands Organisation for Scientific Research (NWO), one of the biggest science-funding bodies in the Netherlands, is making 3 million euros available for a Replication Studies pilot programme. In this programme, scientists will be able to repeat research that has been carried out by others. The pilot focuses on replicating studies that have a large impact on science, government policy or the public debate – namely on the area of the social sciences and health research and healthcare innovation.

The pilot programme Replication Studies focuses on two types of research: On the one hand it focusses on replications with existing data: the datasets from the original study are reanalysed. On the other hand, it focusses on reproductions (with new data): a data collection is put together, which is subjected to the same research protocol as in the original study (read more about the different definitions of reproduction and replication). Read the rest of this entry »

New Working Paper: “Perceptions and Practices of Replication by Social and Behavioral Scientists”

One of our project partners has just released a publication that deals with the replication crises in economics and the social sciences.

In the abstract the three autors state:

We live in a time of increasing publication rates and specialization of scientific disciplines. More and more, the research community is facing the challenge of assuring the quality of research and maintaining trust in the scientific enterprise. Replication studies are necessary to detect erroneous research. Thus, the replicability of research is considered a hallmark of good scientific practice and it has lately become a key concern for research communities and science policy makers alike.

In their discussion paper Fecher, Fräßdorf and Wagner analyze perceptions and practices regarding replication studies in the social and behavioral sciences. Their analyses are based on a survey of almost 300 researchers.

Read the rest of this entry »

Note on “research parasites” published in SCIENCE

Benedikt Fecher and Gert G. Wagner, both also affiliated with the EDaWaX-project, have just published a note in a recent issue of sience under the title “A research symbiont”.

The background for their note is the editorial of D. Longo and J. Drazen – two editors of the New England Journal of Medicine – published in January 2016. In the editorial, Longo and Drazen critically assessed the concept of data sharing in medicine. Their main concern is that a “new class of research person will emerge” that uses data, which were gathered by other researchers, for their own original research questions. The authors, although indirectly, later referred to this class of researcher as “research parasites”.

Fecher and Wagner argue contrarily: For them, the two editors of the New England Journal of Medicine miss the core of the scientific paradigm when writing that researchers may “even use the [open] data to try to disprove what the original investigators had posited.” Instead, Fecher and Wagner argue that data users are research symbionts and benefit the research ecosystem: Using research data to try to disprove a result is good scientific practice, especially in light of the replication crisis (not only) in the social sciences.

A longer version of the Fecher’s and Wagner’s note is available open access (working paper).

Picture: “share-computer-key-260″ by Emilio Quintana on flickr.com. License: CC BY-NC-SA 2.0

Contemporary, useful and subject-based: The replication network

Today I would like to introduce the replication network (TRN) to our readers, a network whose purpose is “to encourage economists and their journals to publish replications.” This is all along in line with the purpose of our own project.

The website of the replication network serves as a channel of communication to both update scholars about the state of replications in economics and to establish a network for the sharing  of information and ideas among economists. It offers important information on the possibility to publish replication studies in economics journals and provides lists of publications dealing with the topic of replications in economic research. Also a list of published replication studies is available. Read the rest of this entry »

Goettingen University: Project fosters Replication in Economics

There are many good reasons why we should replicate scientific findings. In his article “Open Access Economics Journals and the Market for Reproducible Economic Research“, the economist B.D. McCullough (2009) lists some of the reasons why replicable research is crucial for science:

„[…]replication ensures that the method used to produce the results is known. Whether the results are correct or not is another matter, but unless everyone knows how the results were produced, their correctness cannot be assessed. Replicable research is subject to the scientific principle of verification; non-replicable research cannot be verified. Second, and more importantly, replicable research speeds scientific progress. We are all familiar with Newton’s quote, ‘If I have seen a little further it is by standing on the shoulders of giants.’ […] Third, researchers will have an incentive to avoid sloppiness. […] Fourth, the incidence of fraud will decrease.“ (p.118)

More recently, the case of the US-economists Rogoff and Reinhart clearly illustrated the need for replications to be much more common in science and scientific education. But… Read the rest of this entry »