Tag Archives: Expert Search

Expert Search Early Adopters pilot – training evaluation

In mid-September 2020, HEE began a pilot to understand how best to help LKS in England move from using HDAS to using provider interfaces (EBSCOhost, Proquest and Ovid – see here for further detail: https://kfh.libraryservices.nhs.uk/resourcediscovery/frequently-asked-questions-2/)

We’re now heading towards the end of the project and are currently analysing all of the data we’ve collected so we can pull together a report with recommendations for moving forward. Today we’re sharing some of the results from our training evaluation survey – the valuable feedback we’ve collected from pilot participants will shape the way training is rolled out to the rest of LKS in preparation for migration from HDAS to provider interfaces.

Training was offered by all three interface providers (EBSCO, Proquest and Ovid). The team at University of Cambridge Medical Library also very kindly provided online training that was open to all pilot participants, regardless of geographical region. All of the training sessions were recorded and made available to those who couldn’t attend the live sessions. Of the 68 participants who completed the training evaluation survey, 93% had attended some training as part of the pilot.

When asked if the training provided meant that participants felt ready to switch from using HDAS to provider interfaces for literature searching, the responses were split pretty much equally, with 51% answering yes and 49% answering no. We asked for further detail to help clarify the issues around preparedness, as this will be key to helping staff feel confident about moving from HDAS. Those who had answered ‘yes’ said that a) the training sessions were a good starting point, and that they were ready to dive in and start practicing (38%), or b) they already had some familiarity with provider interfaces and so the training acted as a refresher (34%). The responses for those who answered ‘no’ were more difficult to categorise. There was a feeling that the training on its own was not enough, that people needed time to consolidate their learning and practice. There were also comments about the need for follow up sessions so that there was an opportunity to ask questions and share experiences after the initial sessions.

When asked what they would like to change about the training offered, 68% of participants said that the timing of sessions needed to be earlier. There were definitely slippages in the timing of the whole project, and we appreciate that the training schedule didn’t fit with the expectations of the pilot participants. We know from other comments that people needed time to process what they had learnt, to practice searching so they could increase their familiarity with interfaces and also to have a chance to come back to training if they need to. All of the sessions were demonstrations with time for questions, and 10% of participants would have liked hands on training, where there’s a chance to try a live search for yourself. However, there was a recognition that could be difficult in an online format.

There was a marked difference in the feedback for the training delivered by provider representatives and the sessions held by the University of Cambridge Medical Library. Although satisfaction levels were not drastically different, participants commented positively on the content and structure of the Cambridge sessions. Their training focussed on taking a search from start to finish in an interface, whereas participants felt that the interface providers were demonstrating functionality and features that weren’t necessarily relevant to the search process.

The next blog post for the pilot will be sharing some of the results from the data we’ve collected around ‘post-switch’ searches – those carried out in provider interfaces. Participants captured information about 296 searches, which is a fantastic resource for us to draw from.

For any questions about the project, please email Emily.hurt@lthtr.nhs.uk.

Emily Hurt, Lancashire Teaching Hospitals NHS Foundation Trust
Vicky Price, Vicky Price Consulting

Expert Search Early Adopters Pilot Update

In mid-September HEE began a pilot to understand how best to help LKS in England move from using HDAS to using provider interfaces (EBSCOhost, Proquest and Ovid – see here for further detail). The Expert Search Early Adopters pilot is now nearly two months in, and the following is an update of how the project is progressing, along with some detail around what will happen over the next few months.

There are four early adopter groups, which are roughly aligned to regional geographies: Greater Manchester, East Midlands, East of England, and Kent, Surrey and Sussex. 33 LKS are taking part with approximately 120 staff involved in total in the pilot. There’s a wide mix of experience across the groups, with some services frequently using provider interfaces to search and others rarely setting foot outside of HDAS.

Training sessions have been delivered to pilot participants by Proquest, EBSCOhost and Ovid. Cambridge University Medical Library have also run sessions for staff as they have extensive experience of using provider interfaces for searching and wanted to share their knowledge with peers.

Those services without reference management software in place were offered licences for Endnote, and a subscription to Refworks is also available to all participants. The two products use different distribution models (Endnote requires individual licences and software needs downloading onto PCs, whereas Refworks is web based) as the pilot needs to evaluate ease of access and stability as well as the functionality of reference management software. Participants are also being encouraged to use freely available reference management tools such as Zotero and Mendeley if they wish.

To date there have been three different stages of data collection: an initial baseline participant survey, which collected some demographic info and asked people to think about the potential challenges and benefits of switching to provider interfaces, a phase one baseline search data collection survey (pre-switch) and a phase two search data collection survey (post-switch). It was hoped that participants in the pilot would be able to switch from using HDAS in October, but for a variety of reasons this has been moved to early November.

84 participants completed the initial baseline participant survey. 69% of respondents had 5 or more years searching experience, with 52% carrying out between 1 and 3 searches a week, and 10% carrying out between 4 and 6 searches a week. Participants were asked: What do you expect to be the greatest challenges of moving away from HDAS to provider interfaces? Responses were analysed for themes and six were identified, in order of frequency:

• Challenges relating to search results
• Challenges relating to using new products
• Challenges relating to helping library users
• Challenges relating to time
• Challenges relating to provider interfaces
• Challenges relating to using multiple interfaces

Participants were also asked: What do you foresee as the biggest potential benefits of moving from HDAS to provider interfaces? For this question there were nine themes identified, listed below in order of frequency:

• Benefits relating to stability of the search interface
• No benefits
• Benefits relating to search results
• Currently unsure of benefits
• Benefits relating to the skills of library staff
• Benefits relating to library users
• Benefits relating to the development and improvement of interfaces
• Benefits relating to reference management software
• Benefits relating to the functionality of interfaces

During the evaluation stage of the project, which will be towards the end of this year and the start of the next, participants will be asked about the training provided, reference management experiences and revisiting what they thought would be the potential challenges and benefits of switching. The information we collect about searches carried out pre and post switch will be examined to ascertain the impact on the time it takes to carry out a search, and identify any specific issues with the search process. All of the resulting data will be used to help shape the rollout of training and support for the rest of the LKS in England.

For any questions about the project, please email Emily.hurt@lthtr.nhs.uk.

Emily Hurt
Lancashire Teaching Hospitals NHS Foundation Trust

Vicky Price
Vicky Price Consulting