New paper highlights replication studies in economics

Four researchers associated with the former project EDaWaX have recently published a new research article entitled “Replication studies in economics – How many and which papers are chosen for replication, and why?” The article, written by Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff and Gert G. Wagner, sheds light on replication practice in empirical economics. The article can be found here (published in Research Policy under a Creative Commons license). Here, they provide a brief overview of the paper:

Read the rest of this entry »

New Paper asks: “Do journals journals enforce their data policies?”

I am happy to announce that our paper “Journals in Economic Sciences: Paying Lip Service to Reproducible Research?” finally has been released.

The article, published in the latest issue of the IASSIST Quarterly, regards the practice of 37 journals in economics and business administration when it comes to the enforcement of their data policies.

For this purpose, we checked 599 articles which have been published in two issues of these journals. We chose the issues 1/2013 and 1/2014.

First, we estimated the share of articles that fall under a data policy, because replication data is needed to verify the published results. Afterwards, we checked the journal data archives and supplemental information section of each of these articles for the availability of replication files.

For a reduced sub-sample of 245 data-based articles, we checked in depth whether the replication files we found are compliant with the requirements of the journal’s respective data policy. Thereby, we are able to determine how much journals in economic sciences enforce their data policies and to calculate the ‘compliance rate’ for each journal in our sample. Read the rest of this entry »

Wikidata as authority linking hub: The examples of RePEc and GND

From time to time, I post a few more technical oriented articles on this blog. This one is about the opportunities to use Wikidata as an authority linking hub – e.g. for the purpose of correctly identifying authors of scientific publications or data. For developing research infrastructures, it is often is a complex task to offer suitable solutions for a correct identification of an author. Most often it is up to the researchers to correctly indicate their names and to provide a personal identifier (PI), like an ORCID-ID, or a RePEc short-ID.

On ZBW labs, my colleague Joachim Neubert a very interesting blog post about connecting such personal identifiers. In particular, he discusses the possibilities to connect researchers’ personal identifiers from the RePEc (Research Papers in Economics) Author Service (RAS) to those of the GND (Integrated Authority File). Read the rest of this entry »

New paper published: “Open access to research data: Strategic delay and the ambiguous welfare effects of mandatory data disclosure”

Frank Mueller-Langer and Patrick Andreoli-Versbach have published a new research paper on ‘Strategic delay and the ambiguous welfare effects of mandatory data disclosure’.

In the abstract of the paper, the researchers state:

Mandatory disclosure of research data is an essential feature for credible empirical work but comes at a cost: First, authors might invest less in data generation if they are not the full residual claimants of their data after the first journal publication. Second, authors might “strategically delay” the time of submission of papers in order to fully exploit their data in subsequent research. We analyze a three-stage model of publication and data disclosure. We find that the welfare effects of universal mandatory data disclosure are ambiguous. The mere implementation of mandatory data disclosure policies may be welfare-reducing, unless accompanied by appropriate incentives which deter strategic delay.

Read the rest of this entry »

Managing research data using the ZBW Journal Data Archive

Yesterday, a new blog post has been published on ZBW Mediatalk in which I describe the history and the background of the EDaWaX-project that led to the development of the ZBW Journal Data Archive. The Journal Data Archive (JDA) is a service for editorial offices of journals in economics and partially also in the social sciences. Currently this service is free of charge and already two renowned journals are using our services productively.

The  JDA offers direct linking between published research papers and the underlying data, that has been used to generate the findings of an article. The workflow is time-saving and easy to handle: Authors of a journal’s article submit their replication files to the system and describe these files with additional metadata. For us it was quite important to find a useful balance between the efforts a user has to invest to generate these additional metadata and the needs of our information systems. For the development of our metadata schema, we followed the motto ‘as much as necessary, as little as possible.’ Read the rest of this entry »

New Working Paper published: ‘The Economics of Replication’

Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff and Gert G. Wagner have published a new working paper on the ‘economics of replication’.

In the abstract of the paper, the researchers state:

“Replication studies are considered a hallmark of good scientific practice. Yet they are treated among researchers as an ideal to be professed but not practiced. To provide incentives and favorable boundary conditions for replication practice, the main stakeholders need to be aware of what drives replication. Here we investigate how often replication studies are published in empirical economics and what types of journal articles are replicated. We find that from 1974 to 2014 less than 0.1% of publications in the top-50 economics journals were replications. We do not find empirical support that mandatory data disclosure policies or the availability of data or code have a significant effect on the incidence of replication. The mere provision of data repositories may be ineffective, unless accompanied by appropriate incentives. However, we find that higher-impact articles and articles by authors from leading institutions are more likely to be subject of published replication studies whereas the replication probability is lower for articles published in higher-ranked journals.”

The paper is available here.

Read the rest of this entry »

New Guest Blog for TRN

In a Guest Blog for the Replication Network, I wrote a little bit about the history of the EDaWaX project and listed some of the project’s outputs. In addition -and possibly even more important- I also outlined the current and future developments: In particular I dwelled on the question what remains after the project completed its second funding term in autumn.

One answer is the ZBW Journal Data Archive. The technological core of the project has become part of ZBW’s infrastructure. Currently, we are still in the process of enhancing technical functionalities and layout of the application. In the next weeks I will report on detail on the application. Read the rest of this entry »

EDaWaX: Final spurt

Slowly but steady, the EDaWaX project draws to a close. For me a good point in time to outline some of the final steps of our project.

First of all, we are very much engaged in transforming EDaWaX from a project to a regular service for scholarly journals in economic sciences. Therefore we still have a lot of work to do: There are still some issues to solve on the level of software engineering. Currently we are also in the process of improving some of the application’s features and to update the documentation. Nevertheless the software for journals in economics and business research already is available. Starting in the next months we will offer our service (a publication-related data archive) free of charge for editorial offices.

Second, we will present some rescent results of the project at national and international conferences, in roadshows and on annual meetings of learned societies in economics and business studies.

This program starts in two weeks in Munich, where we will disucss the topic of managing research data with the editors of journals in business research and present our software solution at the annual meeting of the German Association for Business Research (VHB), the German learned society for business studies. Read the rest of this entry »

Note on “research parasites” published in SCIENCE

Benedikt Fecher and Gert G. Wagner, both also affiliated with the EDaWaX-project, have just published a note in a recent issue of sience under the title “A research symbiont”.

The background for their note is the editorial of D. Longo and J. Drazen – two editors of the New England Journal of Medicine – published in January 2016. In the editorial, Longo and Drazen critically assessed the concept of data sharing in medicine. Their main concern is that a “new class of research person will emerge” that uses data, which were gathered by other researchers, for their own original research questions. The authors, although indirectly, later referred to this class of researcher as “research parasites”.

Fecher and Wagner argue contrarily: For them, the two editors of the New England Journal of Medicine miss the core of the scientific paradigm when writing that researchers may “even use the [open] data to try to disprove what the original investigators had posited.” Instead, Fecher and Wagner argue that data users are research symbionts and benefit the research ecosystem: Using research data to try to disprove a result is good scientific practice, especially in light of the replication crisis (not only) in the social sciences.

A longer version of the Fecher’s and Wagner’s note is available open access (working paper).

Picture: “share-computer-key-260″ by Emilio Quintana on flickr.com. License: CC BY-NC-SA 2.0

Documentation of EDaWaX’s CKAN-Application: First draft available

One of the primary objectives of the EDaWaX-project is the development of a software application which intends to facilitate the management of research data for journals in the social sciences (and especially for those in economics and business studies).

The software application is approaching its public release at the end of the current funding term in late April. But we are already very far with the software’s developement. For those of you who can’t await the public release, we have just published a very first draft version (Version 0.1) of the application’s documentation. Even though there are still some parts of the functionalities that are not yet covered by the documentation, interested readers and researchers can get a picture of the application’s functionalities and features.

Read the rest of this entry »

Merry Christmas – and a happy new year!

The year draws to a close – therefore the EDaWaX-project team wants to thank our partners, funders and readers for a successful year!

Again, 2015 has been a year full of thrilling events and developments within our project. This year’s focal point primarily was on the advancement of our CKAN-based application. In the first quarter of 2016 we will present more information on the future of the EDaWaX-application and in which direction the project will move after the end of the funding period in May 2016.

Since the start of the second funding period, we implemented a lot of new functionalities. For instance, it is possible to register DataCite-DOIs (digital object identifiers) for submitted research data directly from the application’s frontend. We also slightly improved CKAN’s user and right management to make it more suitable for an utilisation in the context of journal’s data archives. One result is that authors and editorial offices now receive e-mail notifications on recent activities within the data archive.

But we also changed some things based on feedback from the community: For example, we completely overhauled the metadata fields and the web frontend for uploading data. Now it is much catchier for researchers to submit their data to our application.

Read the rest of this entry »

EDaWaX: New research paper published

We are happpy to announce that our new peer-reviewed research paper “Data Policies and Data Archives: A New Paradigm for Academic Publishing in Economic Sciences?” has just been published in the Conference Proceedings of the 19th International Conference on Electronic Publishing (ELPUB15). The paper summarises some results of a broader analyses on data policies of journals in economics and business studies.

In our paper we summarise the findings of an empirical study in which a sample of 346 journals in economics and business studies were examined. We regard both the extent and the quality of journals’ data policies, which should facilitate replications of published empirical research. The paper presents some characteristics of journals equipped with data policies and gives some recommendations for suitable data policies in economics and business sciences journals. In addition, we also evaluate the journals’ data archives to roughly estimate whether these journals really enforce data availability. Our key finding is that we are currently not able to determine a new publishing paradigm for journals in economic sciences.

I would like to thank the reviewers and the organisers of the conference for their useful comments! The full paper is available here.

Read the rest of this entry »

Busy months before summer holidays

It is vacation season and like many others, also some of the people working for the EDaWaX-project enjoy their well-earned summer holidays. Before I also leave the office, I would like to jump on the chance to report on the progress of our project for the last 6 months or so.

It has been a quite busy time for most of us and we achieved progress in many of our work packages. The main focus has been on advancing the technical work packages, but we were also able to finalise two work packages of the project’s analyses phase.

Let me go into detail: Read the rest of this entry »

Open Access to Research Data: Strategic Delay and the Ambiguous Welfare Effects of Mandatory Data Disclosure

Patrick Andreoli-Versbach and Frank Mueller-Langer (two economists from the Max Planck Institute for Innovation and Competition in Munich) have published a paper on strategic delay and the ambiguous welfare effects of mandatory data disclosure (2014).

By setting up a model describing the incentives of a researcher to share self-generated data with the research community in the context of a published article, they found that these incentives can be distorted by a policy they call “First Paper Policy” (requires authors to share their data immediately after the first publication): Read the rest of this entry »

EDaWaX-Study on Data Policies of Journals in Economics and Business Studies (Part I)

One work package (WP2) of EDaWaX’s second funding phase deals with a broader analysis and comparison of journals’ data policies in economics and business studies. In the project’s first funding phase we already have conducted a similar survey, but it primarily focused on journals in economics.

Because research data and methodology in business studies are not necessarily identical to those employed in economics, we found it to be important to compare journals’ data policies of both branches of economic research. Read the rest of this entry »