Maria Teresa Biagetti, Aldis Gedutis, Towards Ethical Principles of Research Evaluation in SSH.

Towards Ethical Principles of Research Evaluation in SSH This paper is an interdisciplinary attempt to map rather unexamined ethical subfield in the broader scientific field of Research Evaluation. Some needs in the area of Research Evaluation are based on qualitative criteria. For this reason it is important to employ assessment criteria based on ethical principles and to have available shared guidelines to research evaluation ethics. Evaluative Bibliometrics uses quantitative criteria (the count of publications and citation analysis) to assess the works of scholars to have rewards, and it produces rankings of institutions for distributing resources. It is, thus, worthwhile to consider that the Evaluative Bibliometrics also requires to employ ethical principles (Furner, 2014). According to Furner, needs of ethical principles in Evaluative Bibliometrics may concern the following: (1) identification of the values held by the members of subgroups that are responsible for actions taken in the course of bibliometric evaluations; (2) identification of the principles for which the members of each subgroup advocate; (3) transparency about the statistical methodologies used and clear description of the results. Moreover, the evaluation process should be based on verified evidence and be unbiased, therefore, statisticians should present results based only on observed phenomena. In the field of Sociology of science, Richard Whitley emphasized that the systems of research evaluation (RES) affect the organization and governance of knowledge production. Strong research evaluation systems ̶ with high standardization, rules and procedures formally established for evaluation and publication of results ̶ influence the research strategies of universities and research institutes, with differences between various scientific fields (Whitley, 2007). Among consequences of strong retrospective assessment systems there is the restriction on universities’ independence in pursuing unorthodox methodologies, in developing innovative theories, and the dissuasion to establish new fields of research in disagreement with dominant disciplinary ideals. The impact of RES is more evident on sciences that present a high level of research objectives coordination, a high level of mutual dependence between scientists to maintain scientific reliability and a high cohesion of scientific elites. In contrast, it is lower in Social Sciences and Humanities (SSH), which present a significantly lower level of scientific production organization, a higher level of disciplinary fragmentation, a higher grade of uncertainty about scientific objectives and a lower level of mutual dependence on disciplinary elites (Whitley, 1984: 87-95; 159-160). Holding in high regard Whitley’s deep analysis, we assume that Research Evaluation plays a fundamental role both in the development of disciplines and in the career advancements of researchers. It is expected to impact on the development of scientific fields, as it may limit novelty and inventiveness of emerging researchers, which must conform to the dominant elites to achieve academic consensus (Whitley, Gläser and Laudel, 2018). This is the reason why ethical principles to support the assessing procedures and shared guidelines for ethical behaviour are highly required. In the field of Evaluation Ethics, the most part of researches are devoted to the evaluation of social projects, to highlight ethical involvements from the point of view of evaluators (Morris, 2008; Schwandt, 2015) and to investigate ethical dilemmas in professional behaviour and in program evaluation, stressing on the problems that arise from the relationships between evaluators and stakeholders and clients (Newman and Brown, 1996). The purpose of this work is not to address issues of Evaluation Ethics from the point of view of evaluators of projects and the ethical challenges that arise in different professions, but to examine the core of Evaluation ethics in connection to Research Ethics, and to assume from the available guidelines for the Research Ethics suggestions and indications for providing guidelines for Research Evaluation in SSH. There are different research evaluation situations in SSH: ex ante research evaluation: attribution of competitive research funding (national or international); ex post research evaluation: reviews after call for papers, articles to be published in scientific journals or in proceedings of scientific conferences; institutional evaluations by Ministries of Education, or national habilitation procedures, but also attribution of funding based on the evaluation of careers and scientific production. Both ex ante and ex post evaluations are involved in funding allocation. In both cases ethical issues are relevant: in ex ante evaluation it is required to verify the feasibility of the research project, involving the stakeholders that should collaborate; in ex post evaluation are involved judgements on careers of researchers. Thus, analysing different contexts, which might be applicable to the Research evaluation ethics, we faced a series of difficulties. First, the field is underdeveloped: majority of researchers do participate in various research evaluation procedures on different levels, but nobody bothers to provide the clear and intelligible set of ethical rules and/or recommendations. Second, there are loads of the issue related material, which is mostly irrelevant as it is scattered between the not-inter-related fields and as a rule is too general and abstract for the ethical research evaluation guidelines. Third, this to-be-established field of the Research evaluation ethics neighbours with two disconnected albeit important fields: (1) Research ethics, which covers mostly natural sciences and psychology and do not think of another SSH, especially, humanities; (2) Evaluation ethics, which mainly aims at evaluation of different social programmes, not concentrating on research. Fourth, the above-mentioned ethical attempts lack sufficient theoretical background in ethical theories as it is by no means clear what is expected from an evaluator as a moral agent. In order to tackle these shortcomings and to provide tentative principles in Research evaluation ethics, available materials and data in the fields of Research ethics and Evaluation ethics should be analysed, compared and combined with those of ethical theories. The most relevant Research ethics sources: ALLEA The European Code of Conduct for Research Integrity (2017), ESRC Framework for Research Ethics (2015), DFID Review of Ethics Principles and Guidance in Evaluation and Research (2015), ESF Peer Review Guide (2011), The Norwegian National Committees for Research Ethics etc., as well as works by researchers such as Robert Merton (1973), David R. Resnik (1998), Michael Morris (2008), Thomas Schwandt (2015), Henrikka Mustajoki and Arto Mustajoki (2017) etc. Data for Evaluation ethics is provided by UNEG Ethical Guidelines for Evaluation (2008), as well as by guidelines found in the documents of American Evaluation Association, Australasian Evaluation Society, Canadian Evaluation Society, Czech Evaluation Society, (German) Evaluation Society, Japan Evaluation Society, Swiss Evaluation Society etc. Both Research ethics and Evaluation ethics provide certain moral principles to deal with proper conduct in their ethics-related situations. In Research ethics the most frequent principles and/or values are the following: rigour, reliability, respect, responsibility, honesty, value-free etc. Accordingly, Evaluation ethics is meant to be grounded in autonomy, nonmaleficence, beneficence, responsibility, justice, fidelity etc. These principles are classified and reinterpreted with the help of toolboxes provided by relevant ethical theories. For our purposes three types of ethical theories are relevant and should be taken into consideration: Deontological ethics: What are moral agent‘s duties to perform? Who or what justifies moral duties? What are rules of research evaluation? Utilitarian ethics: What consequences can be achieved by the action of moral agent? Will they increase common good? What evaluation strategies provide best moral consequences (for society in general, for evaluators and the evaluated)? Virtue ethics: What is moral phronesis? What are the virtues and moral character of a moral person? What are moral characteristics of evaluators? We consider the five ethical principles for evaluation (suggested by Karen S. Kitchener in 1984 for psychological field, and again presented by Newman and Brown 1996, also represented by Resnik 1998, Mustajoki and Mustajoki 2017): Respect for autonomy; Non maleficence (do not harm, do not cause injury); Beneficence (to do good); Justice: procedural (decisions that impact on scholars) and distributive (resources allocation); Fidelity (honesty, integrity). Non maleficence and procedural and distributive Justice are the topics most relevant for our purposes. With the aim of developing and constructing a tentative set of minimal moral requirements and guidelines applicable to the Ethics of research evaluation in the contexts of peer review, ex ante and ex post research evaluations, we try to suggest the following lines to adopt in SSH. Following suggestions from the most relevant Research ethics aforementioned sources, we identified the concept of objectivity (Daston and Galison, 2007), applied to the critical evaluation, as the most relevant for our purposes. Professional ethics of evaluation involves the absence of bias: political, personal, cultural, disciplinary, etc. and an evidence-based evaluation. A general guideline could be to distinguish three aspects: 1) the identification and analysis of the object of a work; 2) the subjective judgement; 3) the possible stakeholders. The aim of our work is to suggest guidelines for Research Evaluation in SSH grounded on the specific characteristics of the sciences that must be evaluated. A characteristic of History, for instance, is the use of historical sources. A historical monograph based on unknown, or never used, important sources, such as archival documentation, must be, of course, recognized as a work that brings about an important contribution to the field. In this case, an ethical principle should be the identification of the objective relevance of the work. On the other hand, the subjective judgement could highlight methods and procedures in using the sources (archival documentation), how well the author has analysed the documents, or what are the borders of the subjective opinion etc. Finally, what benefits of the research at hand give to possible stakeholders (professional community or wider public)? References Daston, Lorraine and Galison, Peter. 2007. Objectivity. New York: Zone Books. Furner, Jonathan 2014. The Ethics of Evaluative Bibliometrics, in: Cronin, B. and Sugimoto, C. R. (eds.) Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact. Cambridge, Mass. and London: The MIT Press, pp. 85-107. Merton, Robert 1973. The Sociology of Science: Theoretical and Empirical Investigations. Chicago: The University of Chicago Press. Morris, Michael (ed.) 2008. Evaluation Ethics for Best Practice. Cases and Commentaries. New York: The Guilford Press. Morris, Stephan G. 2015. Science and the End of Ethics. Palgrave Macmillan. Mustajoki, Henrikka and Mustajoki, Arto. 2017. New Approach to Research Ethics: Using Guided Dialogue to Strengthen Research Communities. London and New York: Routledge. Newman, Dianna L. and Brown, Robert D. 1996. Applied Ethics for Program Evaluation. Thousand Oaks: SAGE Publications. Resnik, David B. 1998. The Ethics of Science: An Introduction. London: Routledge. Schwandt, Thomas A. 2015. Evaluation Foundations Revisited: Cultivating a Life of the Mind for Practice. Stanford: Stanford University Press. Whitley, Richard 1984. The Intellectual and Social Organization of the Sciences, Oxford: Clarendon Press. Whitley, Richard 2007. Changing Governance of the Public Sciences. The Consequences of Establishing Research Evaluation Systems for Knowledge Production in Different Countries and Scientific Fields, in: Whitley, R. and Gläser, J. (eds.) The Changing Governance of the Sciences. The Advent of Research Evaluation Systems. Dordrecht, the Netherlands: Springer, pp. 3-27. Whitley, Richard, Gläser, Jochen and Laudel, Grit 2018. The Impact of Changing Funding and Authority Relationships on Scientific Innovations, “Minerva”, 56:109-134.