KNAW recommends making replication studies a normal and essential part of science

Over the past few years several systematic series of replication studies have been unable to reproduce many important scientific findings in a whole slew of disciplines. This has led to a debate within the scientific community about the way science is currently being conducted and the role of replication studies.  Our blog also published many posts which deal with this question.

Now, KNAW (the Royal Netherlands Academy of Arts and Sciences) has published a report, which analyses the causes of non-reproducibility, assesses the desirability of replication studies and also offers recommendations for improving reproducibility and for conducting replication studies. Read the rest of this entry »

Wikidata as authority linking hub: The examples of RePEc and GND

From time to time, I post a few more technical oriented articles on this blog. This one is about the opportunities to use Wikidata as an authority linking hub – e.g. for the purpose of correctly identifying authors of scientific publications or data. For developing research infrastructures, it is often is a complex task to offer suitable solutions for a correct identification of an author. Most often it is up to the researchers to correctly indicate their names and to provide a personal identifier (PI), like an ORCID-ID, or a RePEc short-ID.

On ZBW labs, my colleague Joachim Neubert a very interesting blog post about connecting such personal identifiers. In particular, he discusses the possibilities to connect researchers’ personal identifiers from the RePEc (Research Papers in Economics) Author Service (RAS) to those of the GND (Integrated Authority File). Read the rest of this entry »

OECD publishes report on business models for sustainable data repositories

In 2007, the OECD Principles and Guidelines for Access to Research Data from Public Funding were published and in the intervening period there has been an increasing emphasis on open science. At the same time, the quantity and breadth of research data has massively expanded. The promise of open (research) data is that they will not only accelerate scientific discovery and improve reproducibility, but they will also
speed up innovation and improve citizen engagement with research.

However, for the benefits of open science and open research data to be realised, these data need to be carefully and sustainably managed so that they can be understood and used by both present and future generations of researchers. Data repositories are where the long-term stewardship of research data takes place and hence they are the foundation of open science. So, the development of sustainable business models for research data repositories needs to be a high priority in all countries. Read the rest of this entry »

CESSDA becomes an ERIC

CESSDA, the Consortium of European Social Science Data Archives, has been established as an ERIC (European Research Infrastructure Consortium) by the European Commission a few days ago. It is the first time in the history of the European Union that a non-member state (Norway) hosts an ERIC.

An ERIC is a specific legal form to facilitate the establishment and operation of research infrastructures with European interest. The principal task of ERIC is to establish and operate new or existing research infrastructures on a non-economic basis. The ERIC becomes a legal entity from the date the Commission decision setting up the ERIC takes effect. Read the rest of this entry »

German Research Foundation (DFG) publishes Statement on Replicability

The German Research Foundation (DFG) has currently released a statement on the replicability of research results.

Interestingly (at least for me), the five-pager first starts with a broader definition of what replicable research is NOT.

Of course, replication is a very important method for testing empirical knowledge claims based on experimental and quantitative research in medicine, the natural, life, engineering, social and behavioural sciences, as well as the humanities.

But, according to DFG, there are limitations:

• Replicability is not a universal criterion for scientific knowledge.
• Ascertaining the replicability or non-replicability of a scientific result is itself a scientific result. As such, it is not final but subject to methodological scepticism and further investigation.
• Non-replicability is not a universal proof by falsification.
• Non-replicability is not a universal indicator of poor science.

‘Well, an unorthodox starting point for a paper on reproducible research‘ – so, at least, were my thoughts when I read the first page of the statement. Wouldn’t it be more common to first depict the important aspects of reproducible research and to suggest measures to support it, instead of rowing back at the beginning of such a statement? Read the rest of this entry »

UK Data Service launches Syntax Upload Facility

The UK Data Service has just released a new service called Syntax Upload Facility. The Syntax Upload Facility enables researchers to upload and share syntax they create using UK Data Service data, building a library of syntax for other users to utilise and cite. The new Syntax Upload Facility aims to enhance researcher’s work and provide an opportunity to increase the impact of research through having syntax used and cited by others. UK Data Service also suggests how to cite these syntax-files. In addition, the syntax itself has a DOI, what promotes uniform citation standards.

The syntax created using our datasets can be uploaded by clicking Contribute your syntax/code at the bottom of the respective catalogue records. Once logged in, the syntax files can be uploaded in various formats including SPSS, Stata, SAS and R.

The Syntax Upload Facility then automatically creates a citation and a file which can be downloaded by other users. The new facility is particularly useful for researchers, students and those teaching with data. An example is available here.

Read the rest of this entry »

Dutch research funder grants 3 million Euros for replication studies

The Netherlands Organisation for Scientific Research (NWO), one of the biggest science-funding bodies in the Netherlands, is making 3 million euros available for a Replication Studies pilot programme. In this programme, scientists will be able to repeat research that has been carried out by others. The pilot focuses on replicating studies that have a large impact on science, government policy or the public debate – namely on the area of the social sciences and health research and healthcare innovation.

The pilot programme Replication Studies focuses on two types of research: On the one hand it focusses on replications with existing data: the datasets from the original study are reanalysed. On the other hand, it focusses on reproductions (with new data): a data collection is put together, which is subjected to the same research protocol as in the original study (read more about the different definitions of reproduction and replication). Read the rest of this entry »

European Commission invests 8.5 billion EUR in research and innovation – and data sharing becomes default

Yesterday, the European Commission (EC) announced an investment of 8.5 billion Euros into research and innovation in 2017.

This investment follows an update to the work programme of the 8th research framework program of the European Union (better known as “Horizon 2020”).

From the perspective of the research data management, this update incorporates some important changes: The current Open Research Data Pilot will be extended so that research data sharing by default will apply to all Horizon 2020 calls covered by the 2017 Work Programme. This means that grant beneficiaries must take measures to enable third parties to access, mine, exploit, reproduce and disseminate research data underlying their scientific peer reviewed publications free of charge. Horizon 2020 beneficiaries are encouraged to also share datasets beyond these publications. Projects can “opt-out” of these provisions in case of conflicts with IPR, personal data protection, national security or other concerns. Read the rest of this entry »

Knowledge Exchange: New Report on Research Software Sustainability published

Knowledge Exchange, a consortium of five national organisations within Europe tasked with developing infrastructure and services to enable the use
of digital technologies to improve higher education and research, has just a released another report on research software sustainability. The report is motivated by the importance of research software for the whole scientific process in every research discipline. The report argues:

“Without software, modern research would not be possible. Understandably, people tend to marvel at results rather than the tools used in their discovery, which means the fundamental role of software in research has been largely overlooked. But whether it is widely recognised or not, research is inexorably connected to the software that is used to generate results, and if we continue to overlook software we put at risk the reliability and reproducibility of the research itself.”

Read the rest of this entry »

ICPSR and Elsevier announce data linking

As announced a few days ago, over 400 Elsevier journals will offer direct linking options with datasets in ICPSR. ICPSR, the Inter-university Consortium for Political and Social Research maintains and provides access to a vast archive of social science data – in total more than 500,000 files. It hosts 16 specialized collections of data in education, aging, criminal justice, substance abuse, terrorism and other fields.

These datasets become discoverable through corresponding research articles on ScienceDirect, a website operated by the Anglo-Dutch publisher Elsevier. This allows researchers to access underlying research data immediately while reading the research paper, and potentially reuse the data for their own research. Vice versa, research articles on ScienceDirect can also be accessed directly from within the ICPSR datasets providing additional information to place the data into context. An example is available here. Read the rest of this entry »

Contemporary, useful and subject-based: The replication network

Today I would like to introduce the replication network (TRN) to our readers, a network whose purpose is “to encourage economists and their journals to publish replications.” This is all along in line with the purpose of our own project.

The website of the replication network serves as a channel of communication to both update scholars about the state of replications in economics and to establish a network for the sharing  of information and ideas among economists. It offers important information on the possibility to publish replication studies in economics journals and provides lists of publications dealing with the topic of replications in economic research. Also a list of published replication studies is available. Read the rest of this entry »

New working paper: “Is Economics Research Replicable?”

Andrew Chang and Phillip Li, two researchers working at the Board of Governors of the Federal Reserve System and at the Office of the Comptroller of the Currency/U.S. Department of the Treasury, attempt to replicate 67 papers published in 13 well-regarded economics journals (American Economic Journal: Economic Policy, American Economic Journal: Macroeconomics, American Economic Review, American Economic Review: Papers and Proceedings (P&P), Canadian Journal of Economics, Econometrica, Economic Journal, Journal of Applied Econometrics, Journal of Political Economy, Review of Economic Dynamics, Review of Economic Studies, Review of Economics and Statistics, and Quarterly Journal of Economics), using author-provided replication files that include both data and code.

Some journals in the sample of Chang and Li require data and code replication files, and other journals do not require such files. Aside from 6 papers that use confidential data, they obtain data and code replication files for at least 29 of 35 papers (83%) that are required to provide such files as a condition of publication, compared to 11 of 26 papers (42%) that are not required to provide data and code replication files. Read the rest of this entry »

New KE-Report: Researcher identifiers – National approaches to ORCID and ISNI implementation

Knowledge Exchange (KE) – a collaboration between five national organisations, DFG – the German Research Foundation, JISC , DEFF – Denmark’s Electronic Research Library, SURF – the ICT organisation for Dutch higher education and research and CSC – the IT Centre for Science in Finland- has just released a new report on research identifiers.

In June 2015, KE brought together representatives from its five member countries for a Knowledge Exchange Workshop on National approaches to ORCID (Open Researcher and Contributor ID) and ISNI (International Standard Name Identifier) implementation. The aim of the workshop was to share national perspectives on ORCID and ISNI, including the challenges, solutions and lessons learned with regards to implementation of ORCID and ISNI on a national scale. Read the rest of this entry »

Wiley partners with Figshare to enable data sharing

At the end of June, Figshare (an online digital repository where researchers can preserve and share their research outputs, including figures, datasets, images, and videos) announced a partnership with the publishing house Wiley to support authors who wish to openly share their data.

Read the rest of this entry »

Article: The Economics of Reproducibility in Preclinical Research

In 2012, an estimated 114.8 billion $ in the US were spent on life sciences research. Roughly half of it is spent on preclinical research, with government sources providing the majority of funding – approximatly 38 billion US$.

Now, three researchers calculate the costs of irreproducible research in preclinical research near 28 billion $ – only for the United States alone. That is the conclusion of a study published in PLoS Biology a few days ago.

In the opinion of the study’s authors, the giant amount of 28 billion $ accumulates, because low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development.

Read the rest of this entry »