Interest in bibliometrics increased in the 1970s due to the need to share knowledge and streamline documentation services (Leksikografski zavod Miroslav Krleža, n.d). The 1970s are described (Heymann, 2017) as a ‘turning point in time’, a period of profound economic, social and cultural changes that led to accelerated production and dissemination of scientific information. An important contribution to the identification of relevant articles in the vast volume of the research literature was made by Eugene Garfield – the inventor of the Impact Factor and the Scientific Citation Index (SCI), which enabled a quantitative analysis of the scientific literature (Garfield, 1979). The advent of SCI strongly influenced the development of bibliometrics, and in the 1980s and 1990s bibliometrics became focused on research evaluation (Franssen & Wouters, 2019).

In recent years, the widespread use of citations in the evaluation of scientific productivity has generated controversy and concern among researchers regarding qualitative evaluation. This issue has been addressed by Aksnes et al. (2019), who conclude that citations reflect, with significant limitations, aspects related to scientific impact and relevance, but they have not been proven to reflect other important dimensions related to research quality. Ma (2021) goes a step further by addressing issues of publishing market pressures under which metrics take on a market value, which can lead to citations and metrics becoming a non-linguistic medium for management rather than evidence of research quality and impact.

Following the recent aspiration of the scientific community and information experts to change the way scientific productivity is evaluated, in early 2022 more than 350 organisations from over 40 countries signed the Research Assessment Reform Agreement, which provides a common direction for change in assessment practices for research basing assessment primarily on qualitative judgement (COARA, n.d.). Croatian University of Rijeka is one of the signatories of the Agreement.

The advent of electronic journals in the late 1980s revolutionised access to knowledge and the latest achievements in the world of science. Increased collaboration between scientists on an international level was encouraged, leading to the globalisation of science. This change led to an even greater acceleration of scientific productivity and an increase in the volume of published material. As the dissemination of knowledge changed significantly with the advent and widespread use of the Internet in the 1990s, the idea of Open Science emerged, advocating scientific transparency and dissemination. As a result, the Open Access (OA) movement was launched, whose primary goal is to make scientific works immediately and widely available to the public and to propagate free access to peer-reviewed scientific papers. Considering the benefits of Open Science, OA journals began to attract attention as it soon became apparent that such journals are more visible and consequently have a greater number of citations. As a result, many information and library specialists have become involved in the bibliometric analysis of OA journals.

Bibliometric indicators, scientific indexing and citations are important elements for assessing the scientific productivity and evaluation of research outputs of the academic population. Therefore, research and academic libraries are taking up the role of research evaluation by providing publication analysis services to University staff. Bibliometric services at academic and research libraries improve support for research, and the services strengthen the partnership between librarians and research institutions. In Croatia, according to the Ordinance on the conditions for selection into scientific professions (Official Gazette 28/2017 (Pravilnik o uvjetima za izbor u znanstvena zvanja, 2017)), bibliometric services are provided by public academic and science libraries in accordance with the available databases and electronic sources. The bibliometric services include:

However, there are three factors that are crucial for a well-established bibliometric service and scientific support: the training of bibliometrics professionals, the legal framework and the operational management of the service provided. Based on these three prerequisites and research findings, this article provides a case study of bibliometric library support practises at the University of Zagreb, Croatia, with the aim of defining the role of academic libraries in the assessment of scientific production.

The legislative framework in the field of science and higher education is based on the work of the Agency for Science and Higher Education, which cooperates with higher-level committees for specific scientific fields and areas. The higher-level committees report to the National Council for Science, Higher Education and Technological Development. The Agency for Science and Higher Education appoints the members of the parent committees from among eminent scientists and members of the academic community. Apart from national regulations, the policies of each faculty of the University are based on a set of statutory regulations that are designed to govern the activities of the faculty. For example, each academic discipline has an Academic Promotion Committee that manages the process of academic promotion at the University. In order to investigate whether there is a legal basis for including libraries in the process of evaluating scientific achievements, the main laws and regulations in the field of librarianship, higher education and science in Croatia were analysed, as mentioned in the Methods section.

Scientists, their scientific titles, academic promotion and the selection procedure for scientific titles are regulated by the Act on Scientific Activity and Higher Education (hereinafter ZZDVO), Art. 31–40, issued by the Ministry of Science and Education as the highest authority in the Croatian science and education system (Zakon o Znanstvenoj Djelatnosti i Visokom Obrazovanju, 2017). The Act in Art. 32, para. 5 defines the set of rules on the conditions for selection into scientific professions, which “… will prescribe in detail the conditions for selection into scientific professions (type and number of scientific works, evaluation, etc.) … on the basis of which the commissions will evaluate the scientific activity of scientists.” Art. 53. para. 2. and Art. 54. para. 2. also describe the establishment of the University and define its structure, highlighting each faculty library as part of the University. In addition, it also refers to the need to develop a unique library system (Zakon o Znanstvenoj Djelatnosti i Visokom Obrazovanju, 2017). It should be noted that the fundamental law of the Croatian library system is the Law on Libraries and Library Activities, which Zakon o Knjičnoj Djelatnosti 2020, Art. 1 refers to the legal obligation for each constituent unit of the University (each faculty) to establish an academic library. Furthermore, Art. 12 sets the Standard for academic, university and scientific libraries and defines their role and activities. It is interesting to note that the Standard does not mention the role of libraries in the assessment of scientific production. In summary, these are the basic legal acts defining the role and position of university libraries in Croatia. The role of academic libraries in the assessment of academic productivity is presented in more detail in the Rulebook on the Conditions for the Selection of Scientific Professions, adopted by the National Council for Science, Higher Education and Technological Development. The Rulebook prescribes the elections for scientific positions by scientific fields in Art. 2. paragraph 4: “The representation (indexation) of journals in the databases prescribed by this Regulation, the impact factors (IF, Impact Factor, i.e. SJR, SCImago Journal Rank Indicator) of scientific journals, the citations of works and other bibliometric data prescribed by the provisions of this Regulation shall be determined by public university libraries and scientific institutions in accordance with the available databases and corresponding certificates shall be issued by a library to all candidates”. As the Regulation does not define criteria and citation indexes by which citations should be displayed, the citations of scientists from all fields of science are displayed in such a way that citations of works represented in the Web of Science database and the Scopus citation database as well as the Google Scholar search engine are displayed, which is in line with the recommendations of the Rectors’ Council from 2015 (Pravilnik o Uvjetima Za Izbor u Znanstvena Zvanja, 2017).

The above-mentioned Standard for higher education, university and scientific libraries defines (Standard Za Visokožnice, 2022) defines the activity of academic, university and scientific libraries whose main task is to provide services to students and scientific staff, and one of the services listed in Art. 11. paragraph 2 are bibliometric services. Thus, from the analysis of the Croatian legal acts, it appears that library services for the assessment of scientific productivity are regulated by law and that bibliometric, metric and similar analyses of scientific productivity are carried out by libraries and professional library staff. Considering that we are talking specifically about the University of Zagreb here, it is important to point out that Art. 48 of the Statute of the University of Zagreb defines the University Library System and the University Library (Statut Sveušta u Zagrebu, 2017). Also, in the Strategy for the Quality Assurance System of the University of Zagreb can be found the development of the internal system and quality control, which includes procedures for external evaluation of systems and institutions of national and international character. The implementation of quality control and academic promotion activities is reflected in the adoption of the national legal framework, the establishment of commissions and the adoption of various regulations. The activities are listed in the Strategy, but for this topic it is important to highlight the activities that control the rules and procedures to ensure and improve the quality of higher education. All the above activities should be carried out in all areas of the university, through the adoption of legal acts, the development of criteria and the orientation of the university towards scientific, research and artistic excellence (Sveušte u Zagrebu, 2014). The University of Zagreb, as the largest university in the Republic of Croatia and a member of the Board of Rectors of the Republic of Croatia, makes clear recommendations for the assessment of scientific productivity, stating that it is important that scientific journals are visible (by indexing in relevant databases and by publishing in open access), while the assessment of scientific impact takes into account all available metric tools used in the field of bibliometrics, scientometrics and webometrics, with the number of citations being one of the metric elements used. In addition, the need for free access to scientific publications and research results is also emphasised (Rektorski zbor Republike Hrvatske, 2015). Finally, it is important to mention that the individual faculties of the University of Zagreb have clearly defined library missions at the institutional level. These regulations regulate in detail the process of evaluating the academic performance of the staff of the parent institution. Such procedures are prescribed by internal acts of individual institutions, such as the Statute of an institution, library regulations, etc.

There are both public and private higher education institutions in Croatia. There are currently 117 higher education institutions (Agency for science and higher education, n.d.). However, most Croatian universities are public institutions. All higher education programmes in Croatia were adapted to the Bologna Process in 2005, which means that potential students can choose between two types of higher education studies: university and vocational studies, with vocational studies being offered mainly at polytechnics and colleges at undergraduate and graduate level, while university studies are offered at undergraduate, graduate and postgraduate level. In addition, the European Credit Transfer System (ECTS) is used to record students’ workload. The University of Zagreb is the oldest Croatian university and the oldest university in Southeast Europe. The university was founded in 1669 by Emperor and King Leopold I. Habsburg. As a broad-based public Central European university, it provides education and research in all scientific fields (arts, biomedicine, biotechnology, engineering, humanities, natural and social sciences) and offers a wide range of study programmes at all levels, from undergraduate to postgraduate. The University consists of 31 faculties, 3 art academies and the University Centre for Croatian Studies. The University is a place where students acquire knowledge and skills. However, in addition to teaching, it also excels in research, contributing over 50 per cent of the annual research output in Croatia and 80 per cent of the scientific productivity of all Croatian universities.

It is clear from the above documents that bibliometric and other metric analyses fall within the remit of academic libraries. However, the question arises as to how well professional librarians are trained for such tasks and to what extent they provide such services as mentioned above, all three study programmes for the training of professional librarians in the Republic of Croatia were analysed.

According to the old study programme for undergraduate and graduate studies at the Department of Information Science in Zadar, which dates back to 2005 and was revised in 2013, bibliometrics is an elective course at the graduate level (4th and 5th year of studies). In the new programme found on the same website (Aparac–Jelušić, 2005), the course covering bibliometrics and evaluation of scientific production could not be discovered at the undergraduate level (year 1–3), but at the graduate level of the new programme (Elaborat o Studijskom Programu, 2013), the elective course called Scientific Communication and Evaluation of Scientific Work could be found.

At the undergraduate level of the Department of Information and Communication Sciences, Faculty of Humanities and Social Sciences, University of Zagreb, there was a course called Bibliometrics in the academic year 2020/2021 as part of the single and double degree programme in Librarianship, while an elective course Bibliometrics, Journals and Scholarly Communication was offered in the academic year 2021/2022. From the information on the department’s website (Sveušte u Zagrebu, n.d.), it was not possible to determine when the course became available to students, so Professor Ana Barbarić, the Head of the study programme, was contacted by email. Prof. Barbarić confirmed that course Bibliometrics has been offered as an elective since the 1986/87 academic year. However, the course was not included in the implementation plan in every academic year because it had the status of an elective. The programme of study for all information science students includes two compulsory courses called Marking and Search Systems I and II, in which students deal with various aspects of evaluating scholarly communication, among other things. The aforementioned course is a compulsory course in librarianship. The study programme for Information Science at the Faculty of Philosophy in Osijek, which is published on the Faculty’s website, does not contain any information about a course on bibliometrics. Therefore, the head of the Department of Information Science, Professor Kornelija Petr Balog, was contacted by email. According to correspondence with Professor Balog, there has been an elective course called Bibliometrics since 2010. The study programme was revised in 2014 and the course Bibliometrics is still available as an elective.

Continuing professional development through part-time study at the Department of Information Sciences at the University of Zadar also offers a part-time programme called Programme for the Acquisition of Library Skills (Sveušta u Zagrebu, 2019), but at that time a course in bibliometrics was not included in the programme.

The Centre for Continuous Professional Development of Librarians (CSSU) in the Republic of Croatia has been operating since 2002 and runs an accredited continuing education programme for librarians and information specialists in the Republic of Croatia. The centre has developed a programme for lifelong learning and continuous professional development of librarians and information specialists. The courses are aimed at librarians of all types of libraries, library assistants, students of library and information science and information specialists. The programme offers a wide range of training and workshops in the field of information and communication sciences and librarianship. An analysis of the CSSU programme was conducted to determine whether the programme includes courses on bibliometric training and assessment of academic productivity. The analysis shows that the CSSU course programme offers a course Bibliometric Analysis and Assessment of Scientific Productivity of Scientists in the Croatian Scientific Community in Module V. Evaluation, Research and Project Management (Centar za stalno stručara, n.d.). Furthermore, in 2019, CSSU published a report on the state of lifelong learning for academic librarians in 2018 (Filipeti, 2019). The report shows that the course on bibliometrics was attended by 112 users, 16 of whom were from university libraries. In addition, a webinar entitled Bibliometric Indicators in the evaluation of Academic Productivity was held in 2018, which was attended by 7.4% of academic librarians. Since the topic of this work was to find out when CSSU started offering training in bibliometrics, the CSSU Centre Manager was contacted by email. The email correspondence revealed that the courses started in 2013: Bibliometric Library Services 1 – Indexing and Citation of Scholarly Works; and Bibliometric Library Services 2 – Metric Indicators of Journals. The courses were held in 2013, 2014, 2015, 2016 and 2018. In addition, the programme included the course Scientific representation and citation: bibliographic and citation databases, held from 2013 to 2015. In 2016, the course was renamed to Evaluation of Scientific Productivity: index and citation analysis of scientific papers and journals and was held in 2016, 2017 and 2019. From 2018 to 2021, the webinar called Bibliometric Indicators in the Evaluation of Scientific Productivity was also held. The webinar was updated in 2022 and renamed Bibliometric Analysis and Assessment of Scientific Productivity of Scientists in the Croatian Scientific Community. The webinar has been available since 2022.

In order to achieve the research objectives, the following methodological approaches were used:

Questionnaire (UNIZG Libraries),

In order to investigate the aforementioned objectives, a qualitative methodology (case study method) was chosen as the research method, as Pickard (2013) argues that the purpose of the case study is to provide a holistic account of the case and in-depth knowledge of the specific through rich descriptions situated in context.

Survey and focus group were used to find out how many UNIZG libraries offer bibliometric services, how the service is organised and what competences librarians should acquire for this type of service.

For the purpose of this research, a 12-question questionnaire was designed and sent to all academic library heads at the University, i.e. a total of 39 e-mail addresses. There were two groups of questions; the first group related to the specification of the bibliometric service and its users, while the second group of questions related to the professional competences of the librarians. The first group of questions is based on the legislative facts from the rules of the library performing these services for the scientific advancement of scientists which is explained/specified in chapter 1.2. The second group of questions relates to librarians’ competences acquired during the study or lifelong learning, as listed in chapter 1.3. To obtain more comprehensive results, the focus group method was used to clarify individual questions and answers (7 library managers were included in such a way that all academic areas were covered). In this way it was possible to examine in more detail the specifics in terms of experience, staff structure and competences/training as well as the form of the bibliometric certificates,¹ the impact on the position of the library within the home institution and the legal framework.

The results obtained from the survey and the focus group interviews should illustrate the service, its organisation at the University, as well as the competences of the librarians. Furthermore, based on the results of the survey, possible irregularities and shortcomings will be easily identifiable. Finally, concluding observations and recommendations will be made based on the overall results.

To find out what practical experiences the libraries of the University of Zagreb have had with the provision of bibliometric services, an online questionnaire was created and sent to the e-mail addresses of the UNIZG library managers. The questionnaire was sent to 39 managers and consisted of two sets of questions. The first set of questions relates to the organisation and scope of the service and determines the categories of the service and its users, while the second set of questions relates to the professional competences of the bibliometric experts. Out of 39 libraries, 24 managers responded. The questionnaire itself was anonymous and the questions related to the size of the library by number of staff and the ratio between the number of staff and bibliometrics professionals at each library. The academic libraries at the University of Zagreb differ considerably in terms of the size of the collection, the library space and the number of staff. For this research it was important to find out the number of staff in each library to see how the bibliometric tasks are distributed. The questionnaire shows that more than half of the libraries at the University of Zagreb has up to 5 staff members (54.2%) and one third of the libraries (33.3%) have only one staff member. Of the other responses received, there are only 3 libraries with more than 20 librarians employed.

Most libraries provide bibliometric services (95.8%), while a smaller number of libraries rely on other libraries (especially the National and University Library, NSK) for this type of service. The types of bibliometric services offered by UNIZG libraries are shown in Figure 1.

Of the bibliometric services offered, the research results (Figure 1) show that the most popular and widely used service is the issuing of bibliometric reports for academic promotion, with 75% of certificates issued for this purpose. Certificates for selection to higher positions are requested most frequently (75%), followed by issuing reports for faculty evaluation procedures (faculty accreditation, accreditation of doctoral programmes) (8.3%) and for the purposes of planning or decision-making at the institution level (8.3%). Other bibliometric services are requested less frequently and relate to reports for University or Ministry of Science and Education needs and bibliometric reports for project applications and reports for faculty or university awards.

The bibliometric library services are mainly used by researchers and professors from the parent institutions (95.8%), while some users come from other institutions (4.2%). In 83.3% of the libraries, the service is free of charge but only available to the staff of the respective institution, while in some libraries (16.7%) the bibliometric services are chargeable but only for external users.

Bibliometric services in the libraries of the University of Zagreb are relatively newly introduced, as can be seen from the data on the duration of bibliometric services presented in Figure 2.

The results regarding the question of how many library staff work as bibliometrics professionals are shown in Figure 3.

As for the distribution of work, according to the data, there are no librarians employed specifically as bibliometricians. Academic librarians do the bibliometric tasks in addition to all other library work. In more than half of the libraries, only one staff member (62.5%) works on these tasks, while in one third of the libraries (33.3%) several librarians are assigned to academic production evaluation.

As far as the knowledge required is concerned, the library managers (the respondents) mention various knowledge and skills required to carry out bibliometric tasks, which can be grouped into the following categories

Figure 4 illustrates the way in which librarians have acquired the expertise required for bibliometric work.

Regarding the required knowledge and skills, a considerable number of librarians (45.8%) working on bibliometric services have completed a special course at the Centre for Continuous Professional Development of Librarians, while as many as 37.5% of them are self-taught. The remaining 16.7% gained their knowledge through a combination of CSSU courses, self-study and consultation with colleagues from other libraries.

The survey also aimed to find out the views of library managers on the need for bibliometric services within the work of each academic library. When asked if bibliometric services should be a regular library service of every library at the University, 83.3% of the library managers answered that they believe that bibliometric services should be an integral part of every library’s workflow, while 16.7% of them believe that these services can be provided by another library.

A focus group interview was conducted to further explore the views of library managers on the issues covered in the questionnaire. The focus group method was organised and conducted as a focus group with a total of 6 participants, in a hybrid format (live and online) at the Library of the Faculty of Humanities and Social Sciences, University of Zagreb on 20 June 2022 from 11:00 to 12:30. The participants of the focus group were selected in such a way that all academic fields were covered, also taking into account the size of the library and the competences of the participants to discuss the given topic. All participants were library managers. The participants were first presented with the results of the questionnaire that had been completed from 26 May to 7 June 2022. The key findings of the focus group interview are represented through the following categories: experiences, form of certificates and personnel structure, and competences/education.