Vinícius Medina KernDepartamento de Ciência da Informação, Universidade Federal de Santa Catarina (UFSC), Brasilvmkern@pq.cnpq.br, www.kern.prof.ufsc.br

Elea Ruth Giménez-ToledoGrupo de Investigación sobre el Libro Académico (ILIA), Consejo Superior de Investigaciones Cientificas (CSIC), Madrid, España, elea.gimenez@cchs.csic.es

The editorial peer review systemThe editorial peer review systemTowards a comprehensive description with the CESM system metamodelTowards a comprehensive description with the CESM system metamodel

Acknowledgments

Funding by CAPES (Brasil, grant no 5934-14-1) and CNPq (Brasil, grant no 487120/2013-2) both to the first author.Anonymous referees provided criticism and recommendations regarding the method, its application in this study, and conclusions.

1 Bunge, M. (2003). Emergence and convergence: qualitative novelty and the unity of knowledge. Toronto, ON: University of Toronto Press..2 Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2-17.3 Bunge, M. (1979). Treatise on basic philosophy: ontology II: a world of systems (vol. 4). Dordrecht, Holland: Reidel.4 Ford, E. (2015). Open peer review at four STEM journals: an observational overview. F1000Research, 4, 6. doi: 10.12688/f1000research.6005.2

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

What is this?A model of the peer review system (graphical and excerpt of textual model).

Or: a CES (Composition-Environment-Structure) system model of editorial peer review.

It is based on the CESM (M for Mechanism) system metamodel1.

It shows the descriptive part of a CESM model, dealing with detectable aspects of a system. Mechanisms are processes that explain emergence. They are usually hidden and need to be conjectured.

CESM correspond to Mario Bunge's reduction to the system1, which is different from analytical science (reductionism, reduction to the atom) and from holism (“reduction to the whole” according to Bunge).

What is the research problem?Complex systems (such as the editorial peer review system) are frequently ill understood and, consequently, poorly approached. There are several biases2 and varied, even conflicting views are frequent (about blinding, for instance). So what is peer review?

This paper's objective is to describe peer review as a system, accounting for its context and culture.

Why is this a research problem?You can’t manage a complex system effectively without some understanding of its functioning, starting with what it is.

A CES model is a systemic description but, as Bunge3 acknowledges, it may be extremely difficult to identify and model a concrete system, 'particularly if it is strongly coupled to other systems', however this is a scientific problem, not an ontological one.

How is it done? (What is the methodological approach?)It is an abstraction (modelling) of a system’s components, environmental items, and bonds found in textual sources (but this usually yields an incomplete model). We then complement the model with experts’ input.

The methodology is under development and goes like this:

(a) Marking: Candidate terms for components, environmental items and bonds are marked in selected textual sources. In this modelling, 3 texts from the Nature Peer Review Debate were used.

(b) Single-text synthesis: Terms are collected and synthesized as system elements, resulting in a single-text model of the system.

(c) Literature-based synthesis: All single-text models are synthesized as one literature-based model of editorial peer review.

(d) Expert panel modelling and review: Gaps (lacking components, environmental items, and most likely bonds) are filled through an expert panel. The final model, supposedly comprehensive, includes definitions for all system elements.

What's new in it? What does it mean to library and information science? (What's the contribution?)It is a description of the editorial peer review system. We still don’t know about the practical consequences for peer review. In the paper, we give a hint of what’s possible with an example from institutional repositories, in which the choice of the best-used software (DSpace) with its default metadata specification (Dublin Core) can kill it – in a systemic effect.

For information science, it is the first study to our knowledge based on Mario Bunge’s CESM system metamodel).

What are the next steps?Consolidate the methodology. We’ve already dropped systematic literature review (too costly) as abstraction method and interviews (too superficial) for filling in the gaps from literature-based abstraction. Those were switched to few-sources literature review and modelling review by a panel of experts.

Formulate peer review research questions as mechanisms (or mechanismic hypotheses) and run experiments. For instance, what changes in the system when we include review reports and research data as part of the publication, as in some open review initiatives4?