Expert Search Early Adopters pilot – searches in provider interfaces

In mid-September 2020, HEE began a pilot to understand how best to help LKS in England move from using HDAS to using provider interfaces (EBSCOhost, Proquest and Ovid – see here for further detail).

The pilot period has now ended and we’re in the process of finalising the project report. Today we’re sharing our findings from the post-switch searches that our participants carried out in provider interfaces between November 2020 and January 2021. We asked people to record data for one day of each week, and they filled in an online form for any searches they did that day. We had 296 searches captured, from quick 20 minute scoping searches to 16 hours’ worth of searching to support a large evidence review and everything in between. We think it’s a fascinating set of data and we could write many, many blog posts about it, but we’re showcasing some of the highlights here. If you have any burning questions about our findings that aren’t addressed here, please post them in the comments below and we’ll try and answer.

Pilot participants were faced with a choice when searching, as Medline is available in all three provider interfaces – Ovid, EBSCO and Proquest. As reflected in pre-switch search data it continued to be the most frequently used database, with 92% of searches using Medline from one provider or another. Ovid Medline was the most frequently used resource overall (70% of searches), with Ovid EMBASE second (61%) and EBSCO CINAHL third (40%).

More post switch searches were carried out over multiple sessions (57%) than in a single session (43%). The time taken to complete a search varied between 20 minutes and 16 hours. As with our pre-switch search data, there were two searches captured that fitted into the systematic review category, and as their times vastly skewed the average search time they were removed from calculations for this blog post. With the remaining 294 searches the average time to complete was 3 hours 13 minutes, with most taking between 2-3 hours or 3-4 hours (23% and 21%).

Participants used many different combinations of interfaces and databases to search. The use of multiple interfaces was spread pretty evenly, with 41% using one provider interface, 39% using two and 20% using three. Where searches did use just one interface, 62% used Ovid.

We asked participants to tell us what had gone well with their search, what didn’t go so well, and what changes could be made to improve their search experience. Things that worked well included the use of reference management software for amalgamating results, the ease of use of subject headings within the provider interfaces, and using one interface – where the search is carried out in multiple databases but in a single provider interface. Things that didn’t work so well included difficulty producing outputs for end user, a perception that the whole process took a lot longer, and problems with reference management software, whether it was difficulty importing results or not being able to format references in a desired style. The suggestions for improvements to the search experience covered a multitude of elements, some related to the interfaces themselves, others around issues with reference management software, the time burden of adapting to new functionality and the extra login steps needed to access databases.

Finally we asked people to rate their search experience a star rating, where 1 is poor and 5 is excellent. The average rating was 3, with 10% rating 1 star, 18% rating 2 stars, 42% rating 3 stars, 21% rating 4 stars and 9% rating 5 stars.

The qualitative data collected about the search experience has been key in shaping the final report for the project, including recommendations for moving forward, and it will be published towards the end of April. For any questions about the project, please email Emily.hurt@lthtr.nhs.uk.

Emily Hurt
Lancashire Teaching Hospitals NHS Foundation Trust

Leave a Reply

Your email address will not be published. Required fields are marked *