Category Archives: Metrics TaF

Making sense of metrics …. by Tracey Pratchett

Hello from the Metrics Task and Finish Group.

In our last blog post, we provided an update on the results of our survey that you all kindly filled out for us. In this new post, we want to tell you about our progress for defining what a good metric looks like and what it should include.

On Tuesday 15th December the Task and Finish Group met to outline our definition for developing good metrics for NHS library, knowledge and information services. Thanks to Dorothy we had access to wide range of literature, which really helped to inform our discussions – who’d have thought there was so much to consider! It provided a really great basis for our group to discuss metrics and will be summarised in our final report.

To inform our discussion we focussed on two key resources which provided a basic framework for us to adapt for the project:

Fuelled by tea and biscuits and after some interesting debates about what metrics meant to each of our group we adapted Markgraf’s (2015) succinct definitions on the following characteristics for good metrics which should be, in summary:

  • Meaningful
  • Actionable
  • Reproducible
  • Comparable

In order to ensure that these criteria would be applicable for NHS library, knowledge and information service metrics, we tested them on some generic metrics from Knowledge for Healthcare and some Key Performance Indicators (KPIs) from HEE LKSL.

The testing highlighted some key challenges in terms of creating metrics which were meaningful to everyone, something which we discussed at length. It also highlighted issues around the distinction between a metric and a target.

What is your experience of using metrics, do you agree with Markgraf’s criteria, or do you have another take on them?

Please share your comments or feedback via this blog or contact the Task and Finish Group lead Alan, Fricker alan.fricker@kcl.ac.uk

Metrics in use – what did you tell us?

In our previous blog post we asked you to give us details of the Metrics you are currently using and why they are effective.  Last week I presented a poster based on the survey results at the London and South East NHS / HE Conference (the presentations from the day are available with lots about KfH).

The poster provides a building block as we continue to work on our report examining the use of Metrics and principles for future developments. Have a look for yourself.

If you have strong views about metrics we are looking for people to help on the group or you can drop me a line with your ideas.

Alan Fricker – Metrics Chair Alan.Fricker@kcl.ac.uk

Hello from the Metrics Task and Finish Group

Metrics are the second task and finish group in the Quality and Impact work stream of the Knowledge for Healthcare programme.  Our remit initially is to consider how we should define metrics for the programme and in Library and Knowledge Services for the NHS more generally.  We want to ensure that we measure our services in ways that are valid, unambiguous and effective.  Building on this we will look at the use of balanced scorecards for gathering and reporting performance indicators.  The group will then consider the NHS LKS annual statistics return.

Each piece of work should inform the others building our understanding and options.  One potential scenario is to end up with widely acceptable and replicable performance indicators, presented consistently and reporting simply at year end or when required.  Clearly this is an ambitious vision and a certain amount of work away!

The group has been meeting virtually since the summer following a change of Chair.  We have been busy on preparatory research, literature searches and assembling the framework for our metrics report.  We are looking at past work in this area in the NHS and at ongoing developments elsewhere.

At the heart of the report will be an exploration of principles for good metrics.

To help us establish those principles we need to have a clear picture of what current practice looks like.  To that end we are asking you to complete a brief survey tool.  The tool invites you to present and explain the metrics you are currently finding most useful.  This could be in terms of understanding your service or in making the case with your stakeholders.  There is space for three but if you have fewer that you really like then just give us those. Equally you can offer more by just completing it again.  There are very few other questions.  If you can provide an email address for us to check any details that would be appreciated but it is not mandatory.

As good metrics will underpin all aspects of Knowledge for Healthcare we need to move on with this work quickly.  Accordingly the closing date for the survey has been set for the 31st of October.  We will analyse the data in November and then push on with the report to ensure it is complete before the end of the year.

https://www.surveymonkey.com/r/HEELKSMetrics2015

Members of the Task and Finish Group

Angela Clifford, Nottinghamshire Healthcare FT
Dorothy Curtis, Gloucestershire Hospitals FT
Alan Fricker, King’s College London (Chair)
Richard Parker, Heart of England
Lorna Wilson, Norfolk & Norwich UHT

Previous members –

Andrew Brown, Frimley Health NHS Foundation Trust
Steve Glover, Central Manchester University Hospitals NHS Foundation Trust