Tag Archives: Quality

Building on good metrics

The Metrics Task and Finish Group is happy to able to share the results of their work looking at principles for good metrics.  You can read our report now on the TaF Reports page.

The goal of this report was to create a shared understanding of what makes a good metric.  Through examining practice within NHS libraries and elsewhere we agreed a set of four core requirements.

* Meaningful

* Actionable

* Reproducible

* Comparable

There is more detail about what each of these means in practice in the report so do give it a read.

To build on the report we now want to encourage the recognition, creation and sharing of good metrics.  To this end we have also prepared a Quality Metrics Template. This is a brief document that can help you structure your thinking as you consider a metric. It also provides the kind of information others would need to be able to see if your metric will also work in their setting.  Please do share your completed templates with us – you can drop them in an email to alan.fricker@kcl.ac.uk as we work out how best to present and share them.  Hopefully we should have some to share with you shortly.

 

Clinical librarians really make a difference!

Alison Brettle, Reader in Evidence Based Practice, University of Salford a.brettle@salford.ac.uk
@Brettleali

One of the proudest moments for me professionally this month was the publication of a study which demonstrates the impact of clinical librarians in the North West.  The paper has been a long time coming so it was very exciting to see it finally in print (or should that be online!).  The paper (Brettle, Maden and Payne, 2016) was the result of a number of years work and collaboration between clinical librarians working in the North West and myself.

The project really began in 2009, when I returned from the EBLIP4 conference in North Carolina. As LIHNN had kindly sponsored my conference fees I wrote an article in LIHNNK Up about the conference, expressing some frustration about the lack of evidence within our profession. My practical way forward was to suggest librarians conducted systematic reviews so we would know what evidence there was – and where the gaps were.  I also strongly believed that getting involved in systematic reviews was a good introduction to research. The clinical librarians group in the North West were interested in publishing more about the work they were doing and they got in touch, and to cut a long story short, this was the beginning of a partnership which resulted in the group undertaking a systematic review on evaluating clinical librarian services (Brettle et al, 2011).  The systematic review updated the evidence on effectiveness as well as highlighting what was needed to provide rigorous evidence to demonstrate the impact of clinical librarian services.  The next logical step was to put these findings into practice and conduct an evaluation across the North West.  This was to be the largest clinical librarian study in the UK to date, and all clinical librarians across the region were invited to participate.  Both these studies benefited from small grants from HCLU, which were key to providing a small amount of resources to get the projects off the ground.

Building on the recommendations from our systematic review, our aim was to understand the impacts of CL services within National Health Service (NHS) organisations, by

  • Using a framework that ensured consistent and robust data collection across all participants
  • Testing the Making Alignment a Priority (MAP) Toolkit (https://maptoolkit.wordpress.com) in measuring the CL contribution to organisational objectives
  • Developing research skills amongst the group of librarians involved.

The paper describes the results and the tools used. Using both questionnaires and interviews, we found that the interventions or services provided by CL’s are complex and each contributes to multiple outcomes of importance to their organisation.  So for example each literature search or participation in a journal club or current awareness bulletin could impact on multiple areas and decisions, and will be unique to each encounter.  We found that the questionnaires were useful in providing data about the outcomes to which the librarians contributed, whilst the interviews really brought this data to life, explaining how one piece of information could really contribute in a wide range of ways that are important within the NHS context.

In brief we found that clinical librarians contribute to a wide range of outcomes in the short and longer term and really do make a difference within the NHS. These include direct contributions to choice of intervention (36%) diagnosis (26%) quality of life (25%), increased patient involvement in decision making (26%) and cost savings and risk management including avoiding tests, referrals, readmissions and reducing length of stay (28%).  As well as looking at contributions to patient care, we looked at other outcomes that are important within the NHS (this is where the MAP toolkit came in), so the study is relevant across all types of NHS organisations not just acute patient care.  We were able to show that clinical librarians improve quality and help save money as well as affecting patient care directly – all key outcomes in the current NHS climate.

The third objective of the study was to help improve research skills, and this isn’t really covered in the paper.  The approach we used built on that used in the systematic review project (Brettle et al, 2011) and has since been described as a “hive approach” (Buckley-Woods and Booth, 2013).  Another way of describing it is “doing with” rather than “doing for”. As an experienced researcher I directed and guided the research but it was very much a partnership and mentoring relationship where the clinical librarians really contributed to the research (and it wouldn’t have taken place if they hadn’t done so).  For this project the clinical librarians were invited to take part in the research at a level that worked for them.  For example, some participated in the survey part whereas others took part in the interviews, interview analysis, and writing up the results.  At the questionnaire design stage, a small group drafted the questionnaire, as a group this was discussed and modified to suit everyone’s needs, and then piloted on each service.  Standard documents were developed and provided to all, on how to conduct the pilot and how to obtain ethical and governance approval.  Meetings were used for agreeing procedures and training.  A wiki was used to share and update resources.  At the interview stage, meetings were held to develop the interview schedule and provide training to those taking part in this stage.  Librarians were “buddied” and conducted interviews in each others organisations (to enhance rigour) but with the advantage that the buddies could practice on each other as well as bounce ideas (and fears!).

The tools developed in the project have informed the development of a simpler generic tool for use across all health libraries and have been incorporated in a revised impact toolkit, for those who want to conduct more in-depth, rigorous impact studies.  In terms of further research what we need to do next is find out whether this approach of building research capacity has made a difference in the longer term.  If this is the case we can use this approach more widely to develop the evidence base of health libraries and librarians for the future.

Acknowledgements
This project wouldn’t have been possible without the librarians involved.  Thanks to: Michelle Maden-Jenkins, Clare Payne, Helen Medley, Tracey Pratchett, Michael Reid, Debra Thornton, Rosalind McNally, Pippa Orr, Morag Platt, Denise Thomas, Anne Webb, Riz Zafar

References
Brettle, A., Maden, M., Payne, C. (2016) The impact of clinical librarian services on patients and health care organisations, Health Information and Libraries Journal. Available from: http://dx.doi.org/10.1111/hir.12136

Brettle, A., Maden-Jenkins, M., Anderson, L., McNally, R., Pratchett, T., Tancock, J., Thornton, D. and Webb,A. A 2011, ‘Evaluating clinical librarian services: a systematic review’ , Health Information & Libraries Journal, 28 (1) , pp. 3-22.

Buckley-Woods, H. and Booth, A. (2013) What is the current state of practitioner research: the 2013 LIRG scan.  Library and Information Research, 37(116).  Available from: http://www.lirgjournal.org.uk/lir/ojs/index.php/lir/article/view/598

Value and Impact Task and Finish Group Update

Since our last blog in early November the group met to compile core questions for an Impact questionnaire. It was agreed that there was value in identifying a small set of core questions which could be recommended for all impact surveys. These could then be supplemented, if desired by individual services, with more targeted questions relevant to their own sectors or services. The advantage of this approach is that it offers the potential for feedback on these core questions to be collated nationally if desired.

The task and finish group used feedback from our reference group members, and the earlier SurveyMonkey responses, to compile a long list of themes which could then be summarised into a few core questions. There was also an awareness of the questions used in earlier surveys and the advantages of providing an element of continuity in order to allow comparison of future results with those already available.

A resulting draft list of four core questions is currently with the reference group for comment before being presented to the Quality and Impact Working Group in January 2016

Membership of the Value and Impact Task and Finish Group

  • Jenny Turner – East Sussex Healthcare NHS Trust (Chair)
  • Stephen Ayre – George Eliot Hospital NHS Trust
  • Douglas Knock – King’s College Hospital NHS Foundation Trust
  • Rebecca Mitchelmore – Isle of Wight NHS Trust
  • Susan Smith – Mid Cheshire Hospitals NHS Foundation Trust
  • Sophie Pattison – University College London
  • Dominic Gilroy – Leeds and York Partnership NHS Foundation Trust