Quality and Improvement Outcomes Framework
What is the timeline for the baseline self-evaluation and validation process?
|Self-evaluation submission deadline||26th June 2020|
|Validation of submissions by HEE||July – October 2020|
|Validated reports returned to organisations||November – December 2020|
How do the levels in each outcome relate to each other?
For each outcome the framework offers a spectrum of five levels ranging from:
Level 0 which is equivalent to a service not being developed within the remit of the outcome up to Level 4 which suggests a highly developed service in relation to the outcome.
The levels are cumulative, each level building on the previous one to enable service improvement. In developing these we looked at how we knew LKS work at various levels of library service development. The step-progression from 1 and 2 to 3 and 4 is logical and you will probably find that you what is needed for levels 1 and 2 need to be in place before you can address what is needed for 3 and 4
How do we use the levels in each outcome for self-evaluation and service improvement?
Within each level there are a series of indicators which suggest whether the level has been met. If all indicators within a level have been met, and can be evidenced, this suggests the organisation and library and knowledge specialits have achieved that level and may be working at the next level
How do we apply Low, Medium and High to each level?
If only some of the indicators within a level appear to have been met then the low, medium and high sub levels may be considered to provide a further way of tracking progress.
Low level suggests that you are at the initial stages of the particular level and can evidence this. High suggests that you can demonstrate you are working fully within a level but as yet you are not working at the next level which will be identified for service improvement and development.
Which organisations need to do a self-evaluation for submission?
Only those NHS organisations that currently have a Learning and Development Agreement with Health Education England will need to carry out a self-evaluation for submission.
A submission will be required for each of these organisations.
How will we report to our senior stakeholders if there is no overall score?
The validated self-evaluation for your organisation will give you a level for each of the outcomes. This will include a visual radar chart/spider graph demonstrating current levels and overtime where improvements have been achieved and can be shared with stakeholders. The report will highlight areas of strength and good practice against all the outcomes, as well as areas for improvement. This will provide an accurate and more meaningful way of reporting.
Are you able to share good examples from LKS services that undertook the pilot last year?
We did try to do this for the pilot, however some of the pilots tried to follow the example too closely with the result that it didn’t always fully demonstrate their own service and organisation. It is difficult to pull together evidence examples as each organisation is different and will be working at different levels for each outcome.
For each outcome a list of suggested outcomes evidence has been provided. Don’t see these evidence suggestions as a prescriptive list as there may be other things of relevance for your organisation and service.
I want to do a preliminary self assessment, is there any documentation that we can use?
It is fine to undertake a preliminary self assessment and the self evaluation template has been made available on the documentation page.
I think that some workshops for people to work together to look at this would be a good idea
We would very much welcome suggestions from colleagues around which workshops/topics might be of use to explore further once the introductory webinars are complete.
Please discuss with your local HEE LKS Lead in the first instance as there may be opportunities to incorporate into forthcoming meetings and events
If there is no overall score how will we benchmark our services?
The Quality and Improvement Outcomes Framework is a service improvement and development tool. Benchmarking is not within the remit of the project.
We are working with HEE Communications and HEE Quality teams to ensure that the Communications to Trust explain the new framework and Quality leads understand the new framework, levels and how the new process works. This will be reported through to the national quality team and integrated with the wider quality education processes with which Trusts are already familiar.
What guidance is there for HEIs?
There is no separate guidance for HEIs. The generic guidance is likely to be detailed enough to assist in the self-evaluation process. As with NHS services, however, any questions and concerns should be directed to the local LKS Lead as they will have local knowledge and be better placed to handle specific queries.
If you have a piece of evidence, for example an annual report, that covers all the organisations you cover how will that work if you are supposed to have a submission for each organisation?
The same piece of evidence can be used for each submission. However, you should ensure that, if it is generic, it covers the relevant organisations in sufficient detail to evidence the narrative and levels you are demonstrating.
What happens when you are incorporated into a wider team for strategy, plan etc? For example, I have 2 points in the corporate strategy.
If you are restricted in this way within wider corporate documents you will need to consider how you can further outline the plans and strategies of your service. This may include using tools such as the Plan on the Page.
What will be the process for submitting the self assessment?
We can say that it will not be a spreadsheet. The final submission structure is not in place at the moment. The blog includes the forms as seen in screenshots, and you can use these for preparing self-evaluation now. There is ongoing work in HEE to restructure our web presence and we hope that this will enable everyone to submit their documentation through a single web portal.
Can you involve teams outside the LKS in the self assessment process?
The Trust is responsible for the submission of the Outcomes framework even though, in practice, the LKS manager will be completing it. This gives an opportunity to work with others across the organisation. Within your LKS team involve everyone. It is not just for the manager to complete.
Will the Strategy and annual report be assessed separately?
No, this will be included as part of the whole valuation that is done. These documents are key documents. These documents give a good overview of your LKS, what has been done over the last 12 months, and shows the direction of travel. Cross-reference if needed.
Levels – how do the levels work?
We spent a long time thinking about and devising the levels and indicators. We looked at how we know LKS work at various levels of library service development. The step-progression from 1 and 2 to 3 and 4 is logical and you should find that you what is needed for levels 1 and 2 need to be in place before you can address what is needed for 3 and 4.
Evidence – can you use the same piece of evidence for more than one outcome?
Yes and we would encourage this approach. Think quality not quantity and cross reference against it for the different outcomes. As the “so what” question of the evidence. Why is it relevant and what does it tell us?
What is the maximum age of the evidence? For example in outcome 4 if you did a role redesign in 2017 would that count?
The evidence is about application and outcomes to the year being reviewed
Will validations be subject to the same traffic light system, i.e. red, amber and green services that the LQAF was?
There won’t be any overall “score” from the new process. There will be a level for each Outcome – so Level 3 for Outcome 1, Level 2 for Outcome 2, etc. HEE and the LKS Leads would have concerns about any services and Trusts who don’t engage with the process and provide a self-evaluation and plans for development. There may also be interventions from HEE where specific problems or issues have been identified.
I want to set myself a SMART target relating to achieving a level for the Outcomes Framework in the baseline year. How do I know what a realistic target might be?
The Outcomes are not just about the LKS, and certainly not just about individuals working within it. Rather it is an approach to looking at how the organisation as a whole supports and makes use of its LKS. Therefore placing the responsibility for achieving certain levels in the Framework on the service – or individuals within it would be innappropriate.
The organisation will not evaluate as having a single level but rather a level per Outcome. It is not necessarily the case, or even likely, that the levels will be the same for every Outcome.
To set a realistic, meaningful target for any of the Outcomes in terms of a level for next year you would need to know where you are now in September 2019. Otherwise the target could mean no progress at all or even going backwards. This would require a self-evaluation on all the Outcomes now and getting this verified by HEE’s library leads. HEE have stated that we will not be undertaking advance verifications of self evaluations as we don’t have the capacity to offer this equitably across the country.
If there is a desire to set an Outcomes Framework associated SMART target we would advise looking at the indicators associated with the Outcomes and linking the aims and objectives to these. With Outcome one for example the objective might be involving specific senior stakeholders within the organisation(s) served in the development of any LKS strategy or associated plans by X date.
Can HEE review my submission in advance for one or more Outcomes and let me know how my organisation is performing?
While we are happy to offer advice on specific issues, HEE does not have the capacity to undertake a comprehensive review of organisation’s self-evaluations in advance of the baseline in 2020.
Does the narrative need to describe and evidence why all levels have been reached or only the highest?
Yes it needs to evidence both the level at which you have assessed your service and those below it. In some cases it is not possible to evidence the higher level without evidencing the levels below.
However where items are mentioned in lower levels and don’t automatically appear in the higher it would be expected that the narrative and evidence captures these too.
Due to staffing issues / organisational changes we would like to request an extension to the Outcomes Framework self-evaluation process
The baseline self-evaluation is intended to provide a snapshot of where the organisation and LKS is at this moment in time in relation to the Outcomes. We are unable to extend the deadline for submission beyond 26th June 2020.