Staying Relevant in the Google Age:
Implementing Vertical Search at the University of Manchester
— a Technological and Cultural perspective

Lorraine Beard
The John Rylands University Library, The University of Manchester, United Kingdom
lorraine.beard@manchester.ac.uk

Andy Land
The John Rylands University Library, The University of Manchester, United Kingdom
andy.land@manchester.ac.uk.

Abstract

John Rylands University Library at the University of Manchester makes a clear commitment to improving resource discovery in its New Directions strategy. In addition an external review of the Library’s systems portfolio identified a clear need to improve the user experience in finding and accessing information. There are also drivers to ensure better utilisation of the Library’s investment in e-resources, as well as the need for the Library to respond to the question of how libraries continue to stay relevant in the Google age. Taking all these factors into account the Library took a strategic decision in 2010 to implement a vertical search solution and it established a formal project to do so in Summer 2010. The project has involved working in close partnership with a number of key stakeholders from the Library, IT Services, multiple vendors and the academic community and is a good example of the changing way in which libraries need to plan and manage their projects in order to improve and deliver services which address the needs of their users. Effective governance, communication, change and project management have been key to the success of this project and highlights the growing importance of these skills in libraries today.

The paper will consider the complex technological issues around integrating data from different sources and using different metadata standards. The project has involved vendors working co-operatively together to enable data to move between systems and different metadata formats, which also presents challenges. The cultural issues involved in implementing vertical search will also be discussed, including a shift in the Library’s internal philosophy towards search, the possible impact on the traditional Library catalogue and associated workflows and the changing user perception of the Library in response to vertical search.

The project is an excellent case study in new ways of working for libraries, tackling complex technological issues and remaining focussed on improving user experience. The paper will be of interest to other libraries considering or in the process of improving their library’s resource discovery experience

Key Words:
Vertical search; research libraries; resource discovery

Introduction

In the John Rylands University Library New Directions strategy there is a clear commitment to improving the search experience. In 2009 the Library conducted an external review of its systems portfolio, to ensure its systems where aligned to its strategic ambitions. One of the main recommendations of the review was that due to the diverse range of interfaces provided by the Library for its users, that we should simplify the user experience by implementing a vertical search solution. This would also provide greater value for money by increasing discovery of the resources that the institution has made a significant financial investment in. A project was therefore commissioned to implement a vertical search solution.

Project Implementation

A project board was established with senior Library leadership and representation from across the Library as well as academic representation. The entire project covered the period from February 2010 to February 2011 and involved procuring and implementing a search solution, building a portal interface to the solution and embedding it within the Library’s web presence, as well as decommissioning any systems no longer needed. To start the procurement process the requirements were defined, a market survey was conducted and we invited a number of vendors to give on-site presentations focussed on our requirements. From this process a product was selected in August 2010, Primo from Ex Libris, and a contract signed by October 2011. An implementation team was established with membership consisting of the different system owners and a Project Manager assigned from ExLibris working with a local Project Manager. The team had an initial kick-off meeting and then had a weekly conference call throughout the project. The team focussed on the data migration from each of the datasources being integrated into Primo, agreeing the interface and ensuring integration with existing University systems. The data sources that we focussed on importing included catalogue data, e-resources data, the institutional repository Manchester eScholar, LUNA (image database) and Elgar (archive database).

We opted for a cloud-based solution so any infrastructure issues were managed by ExLibris. This solution fits in with the wider University IT strategy of using cloud and virtualisation to reduce carbon footprint and also means the total cost of ownership was reduced by requiring less staff support. The main concerns with this were related to reliability of a hosted solution, but so far this has not been an issue. Also access to the back-end is limited as we are sharing server space with other institutions. However, the Primo APIs are good, so, thus far, this has not been a problem. One issue is that we have not been able to get a Manchester specific URL (http://man-fe.hosted.exlibrisgroup.com) — we have a bridging page (http://www.library.manchester.ac.uk/searchresources/librarysearch/).

We met our implementation timescale of going live by February 2011, with our implementation marketed as Library Search. Our experience of working with ExLibris and the project manager assigned was extremely positive.

Systems Integration Issues

Talis

ExLibris had not integrated Primo with the Talis products before this project and so to integrate with Talis Alto Ex Libris built a new pipe based on MARC21. We have not yet achieved real-time availability information in Primo, although this is expected shortly with Talis and Ex Libris working together to achieve this. Talis Alto tables are complex and the existing Talis export routines did not grab the appropriate data and supply it in the required way. Ex Libris worked with Talis to resolve this and a new routine is now in place. For updates Manchester have created an sftp area where we place a nightly update file which is then grabbed by Ex Libris.

We are also working to achieve integration with Primo and Talis Aspire, our reading list solution, so that content bookmarked in Talis Aspire from Talis Prism imports key bibliographic data into Aspire.

Manchester eScholar

As eScholar is built using Fedora Commons it is able to supply data in multiple formats. It was agreed that data would be supplied in MARC21 as a pipe already existed. However, a lot of the metadata in eScholar does not easily map to MARC21 and therefore we agreed that only the key bibliographic data be exported to Primo. Once a record is found the user can then view the full eScholar record as a reformatted eScholar page within the search results record iFrame.

Primo works on the premise that a record will either relate to a physical item held by the Library or to an electronic item that it has access to. In the case of eScholar this is not always the case. The majority of records on eScholar record the fact that a member of Manchester staff has produced a particular piece of work but does not provide access to this. We are still working with Ex Libris to provide a helpful way of indicating this.

LUNA

LUNA is not yet currently fully OAI-PMH-compliant and therefore Ex Libris are not currently able to harvest our LUNA data into Primo. This is recognised by LUNA Inc. as a problem with their OAI-PMH implementation and they are working on a fix which will be applied in the next release (July 2011). There is, therefore, no LUNA content on Primo at present. LUNA metadata is based on extended DC and a significant amount of work had to be done to map the LUNA metadata to the fields used in Primo (both for filtering and search purposes).

User Interface Issues

Primo comes with an out-of-the-box interface which, if you use a hosted service, can be revised in a limited number of ways (addition of logo, amended field labels, changes to the positioning of navigation). Ex Libris were able to quickly provide an initial interface with some content only three weeks into the project. The search window has been embedded into the Library’s website homepage, and into the University’s staff and student portals.

We have a mobile-friendly version of Primo developed which is waiting to be launched alongside the University’s mobile service.

Most interface issues have been minor and easily resolved. There remains a problem displaying non-roman characters in some Primo Central records caused by metadata supplied by some publishers — this is being addressed on a case by case basis by Ex Libris. There also remain issues with the date-newest sort Which, again, has been diagnosed as a problem with some Primo Central content and is being dealt with by Ex Libris.

The grouping of editions using FRBRisation works well on the whole. However, the display of the latest edition of the book, whilst making sense for standard course materials is not always appropriate for Special Collections materials where latest is not always best. Another issue is that filter options in Primo are driven by the metadata in the retrieved records. For FRBRised records only metadata from the record been displayed will be included in the filter options. One could, therefore, have an older edition of a book available at a different location but this location would not appear in the location filter option.

The Primo results page uses iFrames to display content from other sources such as the Library catalogue. We have, though, found that LUNA images do not display well (the ability to zoom in and out does not work well in an iFrame), so we have LUNA records opening in a new window.

Authentication Issues

Integration with Shibboleth authentication was straightforward to achieve. The outstanding issues are that if a user comes to Primo via the University portal having already authenticated, they are not immediately recognised by Primo. Also, if users authenticate in Primo and then go to the Library catalogue, their credentials are not passed through. ExLibris and Talis are working on this together.

Search

Primo uses the Solr search platform.The decision was taken during implementation to provide very few pre-search filter options (you can only choose to exclude journal articles) and to focus instead on post-search filtering. This has been an issue for some users (mainly Library staff and postgraduate researchers who want to pre-select the databases they are searching). The speed of searching and the returning of results is well within what could be considered reasonable. Feedback from Special Collections staff suggests that we need to retain the option of browse searches by title and author and also to be able to turn off the grouping by edition and version (see above).

Management Information

Primo does not make use of Google Analytics but there are management information tools within Prism. Usage of Primo has been high: in the first month of launching there were 122,427 searches and this increased in March 2011 to 205,445 searches.

Cultural Issues

After the implementation, we have surveyed students and staff to gain an insight into their initial user perceptions and their perceptions of the solution after one semester of usage.

To do this, we conducted an email survey of our student ambassadors asking the following:

We received a 9.5% response rate to the survey, with the following results:

Have you used Library Search? The results are shown in Figure 1.


Figure 1: Student usage of Library search at the University of Manchester, from February to June 2011: 83% said they had used Library search while 17% said they had not used it.

How simple was it to use? The results are shown in Figure 2.


Figure 2: Feedback from students showed 33% said it was very simple to use, 56% said it was simple, 11% were undecided. Three people skipped the question entirely.

Did you find what you were looking for? The results are shown in Figure 3.


Figure 3: Student feedback on whether they had found what they were loooking for using Library search: 89% of respondents found what they were looking for; 11% did not.

A summary of the free-text student feedback is given in Table 1.

Table 1: Summary of student feedback on Library Search.

Likes Dislikes
Fast Too many results
Easy to use Unrelated materials
Journal access Difficult to locate a particular paper
Very clear Not all books have images

In terms of improvements to Library Search, the main improvements requested are: more images and extra search options. In comparison to Google and the catalogue students felt that Library Search is faster and simpler and that Library search is better for in-depth search, with Google better for quick background search. Some students commented that they could ‘Find journals I can’t find in Google in Library Search’.

Library Staff Feedback

To gather feedback from Library staff we asked the following questions via email:

Expectations prior to implementation were mixed, from pessimistic to excited to have a good way to compete with Google and that it would produce more relevant results and look and work like Amazon. The key feedback from Library staff regarding Library Search is shown in Table 2:

Table 2: Summary of Library staff feedback on vertical search.

Positive Feedback Negative feedback
• Faster • Students need to understand it has limitations
• Easier for students • Too much content
• Promotes visibility of repository content • Facets have variable value
• Range of resources • Different results to catalogue
• Alerts valuable • Range of resources
• Preferable to Google • Not suitable for researchers
• Less intimidating • Not suitable for Special Collections material

In terms of improvements that staff hoped to see, these were in the areas of improving content coverage, particularly in the field of business and law, improving search including Boolean searching, improved search filtering and search history. The next area for improvement was integration with more resources including LUNA, Elgar and Aspire. There were also comments about better communication and the need to manage user expectations by giving guidance on how Library Search can be used.

Lessons Learnt — Technical/Implementation

The main lessons learnt were around the implementation schedule. We were a little ambitious with the timeframe for the implementation and this limited the amount of time that was available for both usability testing and the metadata mapping which is crucial to the effectiveness of the search results. In order to work to a tight implementation timescale we chose to exclude some resources from the original project and the contract with the supplier and instead would integrate them later outside the project window. This presents difficulties for users in selling a solution as providing full coverage. In retrospect it would have been better if we could have extended the project timeline in order to implement as many data sources as possible to be able to launch a fuller solution. Launching at the start of the academic year would have perhaps also had a more favourable response for users.

In terms of the cultural implications with Library Search there have been a number of key lessons that have emerged. It is clear that Library search is unlikely to be the one stop search shop that we were hoping for. In some subject areas key resources are not available within vertical search and for researchers the subject-specific search functionality that cannot be achieved through vertical search is essential. We therefore need to provide clear routes to other sources of information and guidance on which works best according to the user’s requirements.

Above all we need to engage closely with users as vertical search continues to develop, in order to map search habits and desired outcomes to the interface and functionality we present. We cannot make any assumptions about how users prefer to search without extensive and iterative user testing. One very clear outcome was the need to invest time with Library staff throughout the implementation process and beyond to manage their expectations of vertical search and to gain their valuable professional insights. We had a small steering group which did provide this input, but ideally we would like to have invested more time in the project to get the buy-in from a wider range of staff before the product was launched. A strong marketing and communications plan would also have helped to deliver a clear message about what vertical search is, how it fits in with the search landscape generally and what its limitations are. We opted instead to have a soft launch of vertical search which has left a number of unanswered questions for some users.

The Future of Library Search at the University of Manchester

Firstly we would like to integrate more databases into Library search and complete the integration of LUNA and Elgar. We would like to improve the search functionality by including more prefiltering options. We would like to run some detailed user focus groups to establish what our priorities are for future development and to review the usage patterns of Library search compared to Google, the catalogue and subject databases, in order to inform how we market Library Search most effectively. Key groups of Library staff that we need to obtain more buy-in to the vertical search are the subject librarian team and librarians in Special Collections. New developments that we would like to consider for the future include a recommendations tab and the use of user created content.

Summary

In response to the question originally posed of whether vertical search stays relevant in the Google age, the answer is it is a positive development for us in improving the user experience, but there is still a long way to go before vertical search will be embedded in search habits as deeply as Google is. It does however demonstrate the Library’s willingness to innovate and improve services in line with customer expectations. For Manchester we now intend to engage closely with a range of customers, including Library staff, to improve the current implementation to deliver better results and a better customer experience.

Reference
John Rylands University Library New Directions strategy, http://www.library.manchester.ac.uk/aboutus/strategy/_files/new-directions-2009-2012.pdf (2009).