New Paper asks: “Do journals journals enforce their data policies?”

I am happy to announce that our paper “Journals in Economic Sciences: Paying Lip Service to Reproducible Research?” finally has been released.

The article, published in the latest issue of the IASSIST Quarterly, regards the practice of 37 journals in economics and business administration when it comes to the enforcement of their data policies.

For this purpose, we checked 599 articles which have been published in two issues of these journals. We chose the issues 1/2013 and 1/2014.

First, we estimated the share of articles that fall under a data policy, because replication data is needed to verify the published results. Afterwards, we checked the journal data archives and supplemental information section of each of these articles for the availability of replication files.

For a reduced sub-sample of 245 data-based articles, we checked in depth whether the replication files we found are compliant with the requirements of the journal’s respective data policy. Thereby, we are able to determine how much journals in economic sciences enforce their data policies and to calculate the ‘compliance rate’ for each journal in our sample. Read the rest of this entry »

Against the replication crisis: New international journal encourages replication studies

Read the rest of this entry »

New paper published: “Open access to research data: Strategic delay and the ambiguous welfare effects of mandatory data disclosure”

Frank Mueller-Langer and Patrick Andreoli-Versbach have published a new research paper on ‘Strategic delay and the ambiguous welfare effects of mandatory data disclosure’.

In the abstract of the paper, the researchers state:

Mandatory disclosure of research data is an essential feature for credible empirical work but comes at a cost: First, authors might invest less in data generation if they are not the full residual claimants of their data after the first journal publication. Second, authors might “strategically delay” the time of submission of papers in order to fully exploit their data in subsequent research. We analyze a three-stage model of publication and data disclosure. We find that the welfare effects of universal mandatory data disclosure are ambiguous. The mere implementation of mandatory data disclosure policies may be welfare-reducing, unless accompanied by appropriate incentives which deter strategic delay.

Read the rest of this entry »

Managing research data using the ZBW Journal Data Archive

Yesterday, a new blog post has been published on ZBW Mediatalk in which I describe the history and the background of the EDaWaX-project that led to the development of the ZBW Journal Data Archive. The Journal Data Archive (JDA) is a service for editorial offices of journals in economics and partially also in the social sciences. Currently this service is free of charge and already two renowned journals are using our services productively.

The  JDA offers direct linking between published research papers and the underlying data, that has been used to generate the findings of an article. The workflow is time-saving and easy to handle: Authors of a journal’s article submit their replication files to the system and describe these files with additional metadata. For us it was quite important to find a useful balance between the efforts a user has to invest to generate these additional metadata and the needs of our information systems. For the development of our metadata schema, we followed the motto ‘as much as necessary, as little as possible.’ Read the rest of this entry »

New Working Paper published: ‘The Economics of Replication’

Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff and Gert G. Wagner have published a new working paper on the ‘economics of replication’.

In the abstract of the paper, the researchers state:

“Replication studies are considered a hallmark of good scientific practice. Yet they are treated among researchers as an ideal to be professed but not practiced. To provide incentives and favorable boundary conditions for replication practice, the main stakeholders need to be aware of what drives replication. Here we investigate how often replication studies are published in empirical economics and what types of journal articles are replicated. We find that from 1974 to 2014 less than 0.1% of publications in the top-50 economics journals were replications. We do not find empirical support that mandatory data disclosure policies or the availability of data or code have a significant effect on the incidence of replication. The mere provision of data repositories may be ineffective, unless accompanied by appropriate incentives. However, we find that higher-impact articles and articles by authors from leading institutions are more likely to be subject of published replication studies whereas the replication probability is lower for articles published in higher-ranked journals.”

The paper is available here.

Read the rest of this entry »

ICPSR and Elsevier announce data linking

As announced a few days ago, over 400 Elsevier journals will offer direct linking options with datasets in ICPSR. ICPSR, the Inter-university Consortium for Political and Social Research maintains and provides access to a vast archive of social science data – in total more than 500,000 files. It hosts 16 specialized collections of data in education, aging, criminal justice, substance abuse, terrorism and other fields.

These datasets become discoverable through corresponding research articles on ScienceDirect, a website operated by the Anglo-Dutch publisher Elsevier. This allows researchers to access underlying research data immediately while reading the research paper, and potentially reuse the data for their own research. Vice versa, research articles on ScienceDirect can also be accessed directly from within the ICPSR datasets providing additional information to place the data into context. An example is available here. Read the rest of this entry »

Wiley partners with Figshare to enable data sharing

At the end of June, Figshare (an online digital repository where researchers can preserve and share their research outputs, including figures, datasets, images, and videos) announced a partnership with the publishing house Wiley to support authors who wish to openly share their data.

Read the rest of this entry »

German Council of Science and Humanities publishes Recommendations on Scientific Integrity

Yesterday, the German council of Science and Humanities (Wissenschaftsrat) published a 54-page position paper on scientific integrity (unfortunately in German only). In a press release the scientific body stated:

“Prosperity and development of modern societies today depend to a large extent on the quality and progress of scientific knowledge. Therefore it is important that the public can trust in the ethical and quality standards of the scientific community. Cases of fraud and misconduct – particularly major fraud and plagiarism scandals – undermine that trust. In response to such cases of fraud and loss of confidence many scientific organisations have published rules and guidelines dedicated to a Good Scientific Practice in the last few years. The German council of Science and Humanities now has adopted a position paper and recommendations on scientific integrity.” Read the rest of this entry »

EDaWaX-Study on Data Policies of Journals in Economics and Business Studies (Part I)

One work package (WP2) of EDaWaX’s second funding phase deals with a broader analysis and comparison of journals’ data policies in economics and business studies. In the project’s first funding phase we already have conducted a similar survey, but it primarily focused on journals in economics.

Because research data and methodology in business studies are not necessarily identical to those employed in economics, we found it to be important to compare journals’ data policies of both branches of economic research. Read the rest of this entry »

Science established “Statistical Board of Reviewing Editors” to investigate data

In order to provide better oversight of the interpretation of observational data handed in to the journal along with manuscripts, Science established the Statistical Board of Reviewing Editors (SBoRE), which started working on 1^st July, 2014.

The board consists of experts in various aspects of statistics and data analysis. Their task is to investigate manuscripts, in order to identify issues which need screening, and to suggest experts from the statistics community to examine that data. Editors, BoRE members (members of the BoRE – Board of Reviewing Editors – conduct a rapid quality check of the manuscripts and propose which of them should be reviewed by technical specialists), and reviewers can decide which manuscripts need additional examination of the data analysis or statistical treatment, which will then be handed in to the SBoRE. Read the rest of this entry »

Do open access articles in economics have a citation advantage?

The two economists Klaus Wohlrabe and Daniel Birkmaier (both from the Ifo Institute for Economic Research in Munich) have published a new working paper in which they analyse the impact of open access publishing in economics on citations.

Their sample consists of articles from 2005 from 13 economic journals (including the top five journals). In addition to standard mean comparisons Wohlrabe and Birkmaier also use a negative-binomial regression model with several covariates to control for potential selection effects and quality bias. For their analysis they used citation data from three different databases, namely the Web of Science, RePEc and Google Scholar.

The results they retrieved are very interisting and might light the debate on open access publishing in academia.

Read the rest of this entry »

OJS-Dataverse Integration Plugin released!

In one of my previous blog posts I introduced the PKP/IQSS OJS-Dataverse integration project. After a really short developmental period the project now is happy to announce that the first version has been released! Congrats!

The plugin has been developed by PKP (Public Knowledge Project) in collaboration with Harvard’s Institute for Quantitative Social Science (IQSS).  Funded by a $1 million Alfred P. Sloan Foundation grant, the OJS-DVN project developed the plugin for journals that are using the Open Journal System (OJS). a journal management and publishing system .

Read the rest of this entry »

NISO and NFAIS publish Recommended Practice on Online Supplemental Journal Article Materials

The National Information Standards Organization (NISO) and the National Federation for Advanced Information Services (NFAIS) have published a new Recommended Practice on Online Supplemental Journal Article Materials (NISO RP-15-2013).

Supplemental materials are increasingly being added to journal articles, but until now there has been no recognized set of practices to guide in the selection, delivery, discovery, and preservation of these materials.

To address this gap, NISO and NFAIS jointly sponsored an initiative to establish best practices that would provide guidance to publishers and authors for management of supplemental materials and would address related problems for librarians, abstracting and indexing services, and repository administrators.

The Supplemental Materials project involved two teams working in tandem: one to address business practices and one to focus on technical issues. This new publication is the combined outcome of the two groups’ work.

Read the rest of this entry »

New Open Access Data Journal in Geosciences

The Royal Meteorological Society has partnered with Wiley and Sons to launch a the new Open Access Geoscience Data Journal.

Geoscience Data Journal is online-only and will publish short data papers (articles describing a dataset, giving details including collection, processing, software and file formats) covering topics ranging from weather and climate, to oceanography, atmospheric chemistry and geology. All published data papers will be linked to datasets deposited in approved data centres, which provide details of the collection, processing and file formatting of data and awarded DOIs (Digital Object Identifiers)

Read the rest of this entry »

OSF- Reproducibility Project tries to replicate the results published in three psychological journals

“If you’re a psychologist, the news has to make you a little nervous…”. With this statement Tom Bartlett introduced his article  “Is Psychology About to Come Undone?” in the Chronicle of Higher Education.

The source of his fears is the Reproducibility Project  – a group of researchers that aim to replicate every study within the three journals Psychological Science, the Journal of Personality and Social Psychology and the Journal of Experimental Psychology: Learning, Memory, and Cognition published in the year 2008.

The project is part of Open Science Framework (OSF), a group that is interested in increasing the alignment between scientific values and scientific practices. Despite developing some tools and infrastructure projects its stated mission is to “estimate the reproducibility of published psychological science.”

Read the rest of this entry »