Research Transparency in the Social Sciences

Citation:
Research Transparency in the Social Sciences. Eds. Michael G. Findley, Nathan M. Jensen, Edmund J. Malesky, and Thomas B. Pepinsky. Comparative Political Studies Special Issue.

Original Request for Proposals:

We invite proposals for a special issue of Comparative Political Studies (CPS) on research transparency in the social sciences. Proposals for original research papers using quantitative or qualitative approaches, and collecting quantitative or qualitative data are all encouraged. The deadline for submitted proposals is October 15, 2014 for a Special CPS issue scheduled to appear in 2015-16 academic year.

There is growing momentum in the natural and social sciences for greater transparency in research. For examples, see the Experiments in Governance and Politics Network (EGAP; www.e-gap.org) and the Berkeley Institute for Transparency in the Social Sciences (BITSS; www.bitss.org). While there are varied objectives driving the shift towards greater transparency, one of the key motivations is to avoid publication bias, the result of peer review processes that privilege the significance of results over the theoretical contribution or integrity of the research design. On the other hand, critics of pre-registration argue that it can handcuff authors, leading to journals filled with projects that are less theoretically innovative and path breaking than would otherwise be possible.

This Special Issue of Comparative Political Studies will help to assess the potential benefits and costs associated with new models of the publication process by studying how new models can work in practice. Transparency should obviously be a central objective in contemporary social science, but what are the costs? Do strict pre-registration protocols commit scholars to carry out projects that are unfeasible, or dissuade creative dialogue between theory and data? Is it possible for a full recording of all steps of a research project, from conceptualization to empirical testing? How will manuscript referees respond to manuscripts without results or conclusions? These questions cannot be settled in the abstract.

The Special Issue aims to study the role of full transparency in research in two ways: 1) accepting work based on prospective research designs; and 2) opening up field notes and last-minute alterations in the research design through online archiving (as with replication data). Articles in the Special Issue will be bookended by two articles by the editors, which introduce the goals of the project and critically evaluate the pros and cons of pre-registration and research transparency in political science.

To this end, we invite one of two types of submissions:

1) full research designs for prospective research projects that have not yet been conducted

2) full research designs for projects that have already been conducted, and for which any discussion of results has been stripped out of the manuscript.

If the first type of submission, the design needs to be a thorough project prospectus, sometimes referred to as a pre-analysis plan. While there are multiple ways to construct a pre-analysis plan, submissions for this special issue should provide designs that enable a reviewer to assess as fully as possible the theory, main hypotheses, design, feasibility, and potential contributions of the results. This information should be sufficient to allow reviewers to reach a firm conclusion on the project, and ultimately accept or reject the project for publication in the special issue.

If the second type of submission, the author(s) need to provide a similar level of detail on the theory, design, and credible documentation that the results of the study are not posted or circulated in any way such that a peer reviewer could view the results and make a judgment on the paper with conclusions in mind. Preference will be given to submissions that have not been previously reviewed at another journal.

Once the designs have been submitted, they will be sent out for full peer-review. Designs will be accepted, rejected, or invited to make revisions with resubmission. Once a determination has been made on the design, that decision will be the near-final decision on the manuscript, subject only to the constraint that the research is executed. Deviations from the accepted research designs are acceptable, but need to be documented rigorously and discussed thoroughly. In fact, it is expected that authors of projects that have already been conducted will be asked by reviewers to perform analyses outside of their initial research protocol. This is a normal part of the peer review process: we ask that authors delineate the alterations made as a result of reviewer suggestions in the final article in order to clearly and publicly differentiate them from analyses that were pre-registered. This will provide the editors with unique insight into how the peer review process shapes scientific knowledge and accumulation.

Authors of research papers that are invited to move forward with publication will need to make available all background documents including coding notes, full replication files, etc. To facilitate this process, authors will be eligible for a $5,000 grant provided through the University of Texas at Austin to offset the costs of gathering and making available the required documents, notes, etc.

As with regular submissions, the CPS permanent editors will make a definitive acceptance or rejection based on how authors address the reviewers’ comments, but will not make an independent evaluation of the paper based on the final results.

Proposals should follow the standard CPS submission requirements for normal articles, but should be submitted to directly to the special issue editors at: [email protected]. Please indicate “CPS Special Issue Submission” in the subject line. We encourage you to contact the special issue editors if you have any questions at the above email.

Replication Data: TBA
Registration: TBA