New Paper asks: “Do journals journals enforce their data policies?”

I am happy to announce that our paper “Journals in Economic Sciences: Paying Lip Service to Reproducible Research?” finally has been released.

The article, published in the latest issue of the IASSIST Quarterly, regards the practice of 37 journals in economics and business administration when it comes to the enforcement of their data policies.

For this purpose, we checked 599 articles which have been published in two issues of these journals. We chose the issues 1/2013 and 1/2014.

First, we estimated the share of articles that fall under a data policy, because replication data is needed to verify the published results. Afterwards, we checked the journal data archives and supplemental information section of each of these articles for the availability of replication files.

For a reduced sub-sample of 245 data-based articles, we checked in depth whether the replication files we found are compliant with the requirements of the journal’s respective data policy. Thereby, we are able to determine how much journals in economic sciences enforce their data policies and to calculate the ‘compliance rate’ for each journal in our sample. Read the rest of this entry »

Wiley partners with Figshare to enable data sharing

At the end of June, Figshare (an online digital repository where researchers can preserve and share their research outputs, including figures, datasets, images, and videos) announced a partnership with the publishing house Wiley to support authors who wish to openly share their data.

Read the rest of this entry »

Open Access to Research Data: Strategic Delay and the Ambiguous Welfare Effects of Mandatory Data Disclosure

Patrick Andreoli-Versbach and Frank Mueller-Langer (two economists from the Max Planck Institute for Innovation and Competition in Munich) have published a paper on strategic delay and the ambiguous welfare effects of mandatory data disclosure (2014).

By setting up a model describing the incentives of a researcher to share self-generated data with the research community in the context of a published article, they found that these incentives can be distorted by a policy they call “First Paper Policy” (requires authors to share their data immediately after the first publication): Read the rest of this entry »

EDaWaX: First funding period terminates with evaluation workshop

A week ago our project held its final evaluation workshop. We presented the main results of some of our work packages and also introduced a beta version of our pilot application for the management of publication-related research data in journals.

In preparation of the workshop we invited more than 30 editors of scholarly journals and almost a dozen scientists from 15 journals accepted our invitation. Read the rest of this entry »

Introducing the OJS-Dataverse Integration Project

Today I want to introduce the PKP- Open Journal System / Dataverse Integration Project to our readers, an approach that is in some parts quite similar to our own approach in the field of economics.

Funded by a $1 million Alfred P. Sloan Foundation grant, the OJS-DVN project intends to develop a plugin for journals that are using the Open Journal System (OJS), a journal management and publishing system that has been developed by the Public Knowledge Project (PKP), to expand and improve access to research. Read the rest of this entry »

EDaWaX: New Article published

Last year our project won the LIBER-Award for Library Innovation 2012. Now our paper was published in LIBER-Quarterly, the peer-reviewed journal of the Association of European Research Libaries.

Photo: “Tippen” by Jorma Bork / pixelio.de. License:>click<

Research Data Management in Economic Journals (Part I)

This article is cross-posted from the blog of the Open Economics Working Group

Background

In Economics, as in many other research disciplines, there is a continuous increase in the number of papers where authors have collected their own research data or used external datasets. However, so far there have been few effective means of replicating the results of economic research within the framework of the corresponding article, of verifying them and making them available for repurposing or using in the support of the scholarly debate.

In the light of these findings B.D. McCullough pointed out: “Results published in economic journals are accepted at face value and rarely subjected to the independent verification that is the cornerstone of the scientific method. Most results published in economics journals cannot be subjected to verification, even in principle, because authors typically are not required to make their data and code available for verification.” (McCullough/McGeary/Harrison: “Lessons from the JMCB Archive”, 2006)

Harvard Professor Gary King also asked: “[I]f the empirical basis for an article or book cannot be reproduced, of what use to the discipline are its conclusions? What purpose does an article like this serve?” (King: “Replication, Replication” 1995). Therefore, the management of research data should be considered an important aspect of the economic profession.

Read the rest of this entry »

EDaWaX: First paper published! #Update

I am happy to announce that some of the results we obtained in our analyses have been published in the proceedings of the WissKom2012 conference (pp.29 ff).

In the research paper for Wisskom2012 Olaf Siegert and I summarized some of the findings of the EDaWaX project.  Unfortunately the paper is available in German only, but I promise to publish some more of our major findings in English – eighter on this blog or in a seperate paper. Some results of our analyses are already published here: ( |1 | 2 | 3 | 4 | )

The presentation I gave in Jülich is available in the download section of the blog.

I also wrote a paper in English that has already been accepted by a peer-reviewed journal. I hope to find time to rework the article and to integrate the very helpful notes and comments of the reviewers anytime soon.

I’ll come back to you, when I’ve got news.

# Update 2012-11-14 #

A slightly updated version of the paper is also available now. It has been published in Working Paper Series of the German Data Forum (RatSWD), No. 210. The RatSWD Working Papers series was launched at the end of 2007. Since 2009, the series has been publishing exclusively conceptual and historical works dealing with the organization of the German statistical infrastructure and research infrastructure in the social, behavioral, and economic sciences.

Data Policies in Open Access Journals: Not all that glitters is “gold”

At the end of September, I was in beautifull Vienna for some days and presented a few results of our project at the 6th Open Access Days in a session about “Making Research Data publicly available: opportunities and challenges“.
It’s been a very nice conference – and some blogposts [Uli Herb -> telepolis (in German) | Astrid Recker -> admtic (in English)] report on the conference and some of the presentations. My colleques Olaf Siegert, Ralf Flohr and I also wrote a summary of the conference for ZfBB (in German) that will be published in the next issue of the journal.

Well, to come back to my talk:  In this post I want to point out some single aspects of my presentation: the availability of data policies in economic scholarly journals that are published open access.
Our project compared the number and quality of data policies we found in a sample of 43 open access Journals to a sample of 141 traditional subscription journals in regard to the implementation of data availability policies.

Generally speaking I wondered how open access journals acquit themselves in this context. Prior to starting our analysis I was quite sure that the percentage of journals equipped with a data availability policy would be higher in open access journals than the proportion of subscription journals equipped with such a policy. I even thought that open access journals could have a comparative advantage to traditional subscription journals: Read the rest of this entry »

Introducing the Journal Research Data Policy Bank (JoRD)-Project

Journal Research Data Policy Bank (JoRD) is a JISC funded initiative conducting a feasibility study into the scope and shape of a sustainable service that will collate and summarise journal policies on Research Data to provide researchers, managers of research data and other stakeholders with an easy source of reference to understand and comply with these policies. Read the rest of this entry »

Data Policies in Economic Journals: What Data Authors have to provide beside the Manuscript?

As mentioned in some of my previous blogposts we analyzed more than 140 economic scholarly journals regarding their data availability policies. It has been an interesting work checking the quality and the extend of these data policies.

In our sample (that was evaluated in 2011 by the German Economics Bräuninger, Haucap and Muck regarding the reputation and relevance of these journals) we were able to find 29 journals equipped with a data availability policy and 11 journals that own a so called “replication policy”.

Read the rest of this entry »

Second Phase of our Survey just started

The EDaWaX Project today has send a reminder to 350 editors of economic journals to fill out the online questionnaire dealing with data availability policies in scholarly journals. One of the questionnaire was designed for journals without data policies; the second questionnaire was created for journals that already own a data policy.

Read the rest of this entry »

EDaWaX-Project launches Online-Surveys for Editors of Economic Scholarly Journals

The EDaWaX Project today has launched two online questionnaire for editors of scholarly journals. One of the questionnaire was designed for journals without data policies, whereas the second questionnaire was created for journals that already own a data policy.

Read the rest of this entry »

EDaWaX WP2: Some Information about Journals and Selection of our Analysis

Today I want to share some parts of our current research with you. While writing our report on the work package 2, I thought it would be a good idea to publish some of my thoughts – it helps to structure all the information we have.
I start with a brief introduction about the selection of scholarly journals for our research work. Surely this will be only a small excerpt of the whole information presented in the report. So watch out for this publication!

Read the rest of this entry »

Data Availability Policy: American Economic Review

As announced in my previous blogpost, I ‘m starting the presentation of some data availability policies and replication policies with the American Economic Review (AER). The AER is a flagship of the economic profession and one of the top ranked journals in this scientific discipline.
The AER was published in 1911 for the first time. Only 7 – 10 percent of the submissions are accepted and later on published.

The AER adopted a so called replication policy in 1986 – despite the fact that studies (for example by Dewald, Thursby and Anderson) already claimed, that a replication policy is not enough to promote replicable results.
In their policy, the Review pledged authors to provide datasets and code for processing the data to other scientists that are interested in replicating the results on request.

Replication policies have often failed, even if the corresponding author is willing to support other researchers…and I imagine that this szenario is not very common …After publishing an article, authors mostly don’t have incentives to prepare the data and code for other researchers. It costs time and the rewards the scientific system pays for sharing data often are marginal.

Read the rest of this entry »