New paper highlights replication studies in economics

Four researchers associated with the former project EDaWaX have recently published a new research article entitled “Replication studies in economics – How many and which papers are chosen for replication, and why?” The article, written by Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff and Gert G. Wagner, sheds light on replication practice in empirical economics. The article can be found here (published in Research Policy under a Creative Commons license). Here, they provide a brief overview of the paper:

Read the rest of this entry »

Replications in the social sciences: New study confirms ongoing challenges

Much has been said on the importance of replications. Recently, nature has published another comment that deals with this question. Paul Gertler, Sebastian Galiani and Mauricio Romero (GGR) conducted a survey, in which they focussed on the fields of economics, political science, sociology and psychology. They conclude that ‘the current system makes original authors and replicators antagonists.’

They found, that in the top-tier economics journals only few articles are replications – and all of those refute the original results. That said, GGR also asked 35 editors and co-editors of these economics journals about their perceptions towards publishing replications. While all editors who responded would publish a study that refutes the original findings, only a fourth can image to publish a study which confirms the original results. Read the rest of this entry »

KNAW recommends making replication studies a normal and essential part of science

Over the past few years several systematic series of replication studies have been unable to reproduce many important scientific findings in a whole slew of disciplines. This has led to a debate within the scientific community about the way science is currently being conducted and the role of replication studies.  Our blog also published many posts which deal with this question.

Now, KNAW (the Royal Netherlands Academy of Arts and Sciences) has published a report, which analyses the causes of non-reproducibility, assesses the desirability of replication studies and also offers recommendations for improving reproducibility and for conducting replication studies. Read the rest of this entry »

Against the replication crisis: New international journal encourages replication studies

Read the rest of this entry »

New paper published: “Open access to research data: Strategic delay and the ambiguous welfare effects of mandatory data disclosure”

Frank Mueller-Langer and Patrick Andreoli-Versbach have published a new research paper on ‘Strategic delay and the ambiguous welfare effects of mandatory data disclosure’.

In the abstract of the paper, the researchers state:

Mandatory disclosure of research data is an essential feature for credible empirical work but comes at a cost: First, authors might invest less in data generation if they are not the full residual claimants of their data after the first journal publication. Second, authors might “strategically delay” the time of submission of papers in order to fully exploit their data in subsequent research. We analyze a three-stage model of publication and data disclosure. We find that the welfare effects of universal mandatory data disclosure are ambiguous. The mere implementation of mandatory data disclosure policies may be welfare-reducing, unless accompanied by appropriate incentives which deter strategic delay.

Read the rest of this entry »

UK Data Service launches Syntax Upload Facility

The UK Data Service has just released a new service called Syntax Upload Facility. The Syntax Upload Facility enables researchers to upload and share syntax they create using UK Data Service data, building a library of syntax for other users to utilise and cite. The new Syntax Upload Facility aims to enhance researcher’s work and provide an opportunity to increase the impact of research through having syntax used and cited by others. UK Data Service also suggests how to cite these syntax-files. In addition, the syntax itself has a DOI, what promotes uniform citation standards.

The syntax created using our datasets can be uploaded by clicking Contribute your syntax/code at the bottom of the respective catalogue records. Once logged in, the syntax files can be uploaded in various formats including SPSS, Stata, SAS and R.

The Syntax Upload Facility then automatically creates a citation and a file which can be downloaded by other users. The new facility is particularly useful for researchers, students and those teaching with data. An example is available here.

Read the rest of this entry »

New Working Paper published: ‘The Economics of Replication’

Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff and Gert G. Wagner have published a new working paper on the ‘economics of replication’.

In the abstract of the paper, the researchers state:

“Replication studies are considered a hallmark of good scientific practice. Yet they are treated among researchers as an ideal to be professed but not practiced. To provide incentives and favorable boundary conditions for replication practice, the main stakeholders need to be aware of what drives replication. Here we investigate how often replication studies are published in empirical economics and what types of journal articles are replicated. We find that from 1974 to 2014 less than 0.1% of publications in the top-50 economics journals were replications. We do not find empirical support that mandatory data disclosure policies or the availability of data or code have a significant effect on the incidence of replication. The mere provision of data repositories may be ineffective, unless accompanied by appropriate incentives. However, we find that higher-impact articles and articles by authors from leading institutions are more likely to be subject of published replication studies whereas the replication probability is lower for articles published in higher-ranked journals.”

The paper is available here.

Read the rest of this entry »

Dutch research funder grants 3 million Euros for replication studies

The Netherlands Organisation for Scientific Research (NWO), one of the biggest science-funding bodies in the Netherlands, is making 3 million euros available for a Replication Studies pilot programme. In this programme, scientists will be able to repeat research that has been carried out by others. The pilot focuses on replicating studies that have a large impact on science, government policy or the public debate – namely on the area of the social sciences and health research and healthcare innovation.

The pilot programme Replication Studies focuses on two types of research: On the one hand it focusses on replications with existing data: the datasets from the original study are reanalysed. On the other hand, it focusses on reproductions (with new data): a data collection is put together, which is subjected to the same research protocol as in the original study (read more about the different definitions of reproduction and replication). Read the rest of this entry »

European Commission invests 8.5 billion EUR in research and innovation – and data sharing becomes default

Yesterday, the European Commission (EC) announced an investment of 8.5 billion Euros into research and innovation in 2017.

This investment follows an update to the work programme of the 8th research framework program of the European Union (better known as “Horizon 2020”).

From the perspective of the research data management, this update incorporates some important changes: The current Open Research Data Pilot will be extended so that research data sharing by default will apply to all Horizon 2020 calls covered by the 2017 Work Programme. This means that grant beneficiaries must take measures to enable third parties to access, mine, exploit, reproduce and disseminate research data underlying their scientific peer reviewed publications free of charge. Horizon 2020 beneficiaries are encouraged to also share datasets beyond these publications. Projects can “opt-out” of these provisions in case of conflicts with IPR, personal data protection, national security or other concerns. Read the rest of this entry »

New Working Paper: “Perceptions and Practices of Replication by Social and Behavioral Scientists”

One of our project partners has just released a publication that deals with the replication crises in economics and the social sciences.

In the abstract the three autors state:

We live in a time of increasing publication rates and specialization of scientific disciplines. More and more, the research community is facing the challenge of assuring the quality of research and maintaining trust in the scientific enterprise. Replication studies are necessary to detect erroneous research. Thus, the replicability of research is considered a hallmark of good scientific practice and it has lately become a key concern for research communities and science policy makers alike.

In their discussion paper Fecher, Fräßdorf and Wagner analyze perceptions and practices regarding replication studies in the social and behavioral sciences. Their analyses are based on a survey of almost 300 researchers.

Read the rest of this entry »

Note on “research parasites” published in SCIENCE

Benedikt Fecher and Gert G. Wagner, both also affiliated with the EDaWaX-project, have just published a note in a recent issue of sience under the title “A research symbiont”.

The background for their note is the editorial of D. Longo and J. Drazen – two editors of the New England Journal of Medicine – published in January 2016. In the editorial, Longo and Drazen critically assessed the concept of data sharing in medicine. Their main concern is that a “new class of research person will emerge” that uses data, which were gathered by other researchers, for their own original research questions. The authors, although indirectly, later referred to this class of researcher as “research parasites”.

Fecher and Wagner argue contrarily: For them, the two editors of the New England Journal of Medicine miss the core of the scientific paradigm when writing that researchers may “even use the [open] data to try to disprove what the original investigators had posited.” Instead, Fecher and Wagner argue that data users are research symbionts and benefit the research ecosystem: Using research data to try to disprove a result is good scientific practice, especially in light of the replication crisis (not only) in the social sciences.

A longer version of the Fecher’s and Wagner’s note is available open access (working paper).

Picture: “share-computer-key-260″ by Emilio Quintana on flickr.com. License: CC BY-NC-SA 2.0

Action Poll- tools for working with research data

You are a researcher and you are working with data? Great!

This short, anonymous poll- targeted at researchers- aims to characterise the range of research data management tools that are used across domains and regions. It is being conducted by Research Data Alliance (RDA) Long Tail of Research Data Interest Group.

The results will be reported at the next RDA meeting in Paris in September 2015 and also made available through the Long Tail Interest Group webpage.

The poll will be open from June 15 to July 31, 2015 and is available on https://www.surveymonkey.com/r/TVQ7RY6.

 It is only one-page long and will take less that 5 minutes to complete.

For more information, please contact: Kathleen Shearer, Executive Director, COAR Read the rest of this entry »

Save the date: Workshop “Open data, restricted data and the role of libraries”

NEREUS – a network of European libraries in Economics and Social Sciences –  is happy to announce a pre-conference workshop (2015-6-23 in London) at this year’s LIBER-Conference.

Open Science and open data have become hot topics in recent years. Effective research data management is more and more postulated by research funders. Research infrastructure providers worldwide are busy building up various services and tools for researchers to support them within their research and the management of research data. But how successful are these approaches and their impact in supporting research? How open could or should data be and which role(s) libraries can play to support researchers effectively? Read the rest of this entry »

Paper on the Incentives for Academic Data Sharing published: „What Drives Academic Sharing?“

Benedikt Fecher, Sascha Friesike, and Marcel Hebing have published another paper presenting further results of their study concerning academic data sharing. Since data sharing enables researchers to verify results and to pursuit new research questions with “old” data, it is of particular importance for scientific progress.

Fecher, Friesike, and Hebing conducted a systematic review of 98 scholarly papers as well as an empirical survey among 603 secondary data users. In order to explain the data sharing process from the primary researcher’s point of view, the authors introduce a conceptual framework based on the analyses. They divide the data sharing process into six descriptive categories: data donor, research organization, research community, norms, data infrastructure, and data recipients. Read the rest of this entry »

New Working Paper on Data Sharing Practises published: “A Reputation Economy”

Academic data sharing is a way for researchers to collaborate and thereby meet the needs of an increasingly complex research landscape. It enables researchers to verify results and to pursuit new research questions with “old” data.

.
It is therefore not surprising that data sharing is advocated by funding agencies, journals and researchers alike. Read the rest of this entry »