Tag Archives: metrics

Metrics Update

The Metrics Task and Finish group is largely resting after the publication of their Report and the accompanying Quality Metrics Template.

I spoke about the work we had completed at HLG2016 Scarborough and it brought home to me the need to make the materials we had produced more accessible.  The strong attendance and active debate impressed on me that people were definitely interested but struggling to get to grips with how to proceed.

To address this I took advantage of the annual poster competition at #NHSHE2016 and submitted the below poster on Metrics.  I went with trying to hammer home the message about the four principles and what they mean in practice. Using MARC as an acrostic had the bonus of chucking in a feeble nerdy library pun.

The poster was well received (though only placed 6th in the competition) and I was particularly pleased to see a tweet afterwards sharing the poster with a group of other libraries after it had been raised at a network meeting.

I am hoping that people will share with me examples of how they have used the metrics work. Development is underway to provide a submission tool for sharing metrics similar to that to be used around impact case studies. In the meantime – do get in touch with thoughts, questions and comments.

Alan Fricker
Metrics TaF Chair

Building on good metrics

The Metrics Task and Finish Group is happy to able to share the results of their work looking at principles for good metrics.  You can read our report now on the TaF Reports page.

The goal of this report was to create a shared understanding of what makes a good metric.  Through examining practice within NHS libraries and elsewhere we agreed a set of four core requirements.

* Meaningful

* Actionable

* Reproducible

* Comparable

There is more detail about what each of these means in practice in the report so do give it a read.

To build on the report we now want to encourage the recognition, creation and sharing of good metrics.  To this end we have also prepared a Quality Metrics Template. This is a brief document that can help you structure your thinking as you consider a metric. It also provides the kind of information others would need to be able to see if your metric will also work in their setting.  Please do share your completed templates with us – you can drop them in an email to alan.fricker@kcl.ac.uk as we work out how best to present and share them.  Hopefully we should have some to share with you shortly.

 

Making sense of metrics …. by Tracey Pratchett

Hello from the Metrics Task and Finish Group.

In our last blog post, we provided an update on the results of our survey that you all kindly filled out for us. In this new post, we want to tell you about our progress for defining what a good metric looks like and what it should include.

On Tuesday 15th December the Task and Finish Group met to outline our definition for developing good metrics for NHS library, knowledge and information services. Thanks to Dorothy we had access to wide range of literature, which really helped to inform our discussions – who’d have thought there was so much to consider! It provided a really great basis for our group to discuss metrics and will be summarised in our final report.

To inform our discussion we focussed on two key resources which provided a basic framework for us to adapt for the project:

Fuelled by tea and biscuits and after some interesting debates about what metrics meant to each of our group we adapted Markgraf’s (2015) succinct definitions on the following characteristics for good metrics which should be, in summary:

  • Meaningful
  • Actionable
  • Reproducible
  • Comparable

In order to ensure that these criteria would be applicable for NHS library, knowledge and information service metrics, we tested them on some generic metrics from Knowledge for Healthcare and some Key Performance Indicators (KPIs) from HEE LKSL.

The testing highlighted some key challenges in terms of creating metrics which were meaningful to everyone, something which we discussed at length. It also highlighted issues around the distinction between a metric and a target.

What is your experience of using metrics, do you agree with Markgraf’s criteria, or do you have another take on them?

Please share your comments or feedback via this blog or contact the Task and Finish Group lead Alan, Fricker alan.fricker@kcl.ac.uk