Frequently Asked Questions

Frequently Asked Questions: Quality and Improvement Outcomes Framework

Organisations

Which organisations need to do a self-evaluation for submission? 

Each NHS organisation with a Learning and Development Agreement with Health Education England will need to carry out and submit a self-evaluation.

What guidance is there for Higher Education Institutions (HEIs)? 

There is no separate guidance for HEIs.  The generic guidance is detailed enough to assist in the self-evaluation process.

Any questions and concerns should be directed to the local LKS Lead.

Timeline and submission deadlines

What is the timeline for the baseline self-evaluation and validation process? 

Submission deadline deferred until 2021. 

Given Covid-19 pressures on NHS organisations, the baseline submission for all NHS organisations will now be June 2021.

Any work from June 2019 will  be relevant for the submission in June 2021.

Due to staffing issues / organisational changes we would like to request an extension to the Outcomes Framework self-evaluation process 

The baseline self-evaluation provides a snapshot of how the organisation and LKS meets the Outcomes at the time of submission.

Consequently we are unable to extend the deadline for submission.

Self-assessment process

I want to do a preliminary self-assessment, is there any documentation that I can use? 

Yes this is fine. Use the self-evaluation template from the documentation page and associated documents.

What will be the process for submitting the self-assessment? 

The final submission structure is not in place at the moment.

HEE is developing the LKS web presence.  In time, we hope that this will enable submissions through a single web portal.

Can you involve teams outside the LKS in the self-assessment process? 

While the Trust is responsible for the submission of the Outcomes Framework evaluation, the LKS manager will most likely complete it.

You may  work with others across the organisation as well as other members of the team.  It is not just for the manager to complete.

I think that some workshops for people to work together to look at this would be a good idea.  

We welcome suggestions from colleagues about the workshops/topics might be of use, once the introductory webinars have been completed.

Discuss with your local HEE LKS Lead first.  There may be opportunities to incorporate this into forthcoming meetings and events.

Can HEE review my submission in advance for one or more Outcomes and let me know how my organisation is performing? 

While we are happy to offer advice on specific issues, HEE does not have the capacity to undertake a comprehensive review of organisations’ self-valuations in advance of the baseline in 2021.

Outcome Levels

Levels – how do the levels work? 

We spent a long time thinking about and devising the levels and indicators.

Using our knowledge, we considered how library and knowledge specialists operate at various levels of LKS development.

The step-progression from 1 and 2 to 3 and 4 is logical. You should find that you what is needed for levels 1 and 2 needs to be in place before you can address what is needed for 3 and 4.

How do the levels in each outcome relate to each other? 

For each outcome the framework offers a spectrum of five levels. These range from:
Level 0, where a service is not developed, up to Level 4 which suggests a highly developed service in relation to the outcome.

Levels are cumulative.  Each level builds on the previous one to enable service improvement.

How do we use the levels in each outcome for self-valuation and service improvement? 

Within each level there are a series of indicators which suggest whether the level has been met.

If all indicators within a level have been met, and can be evidenced, this suggests the organisation and library and knowledge specialists have achieved that level and may be working at the next level.

How do we apply Low, Medium and High to each level? 

If only some of the indicators within a level appear to have been met then the low, medium and high sub levels may provide a further way of tracking progress.

Low level suggests that you are at the initial stages for the particular level and can evidence this.

High suggests that you can demonstrate you are working fully within a level but as yet you are not working at the next level.  This will be identified for service improvement and development.

Does the narrative need to describe and evidence why all levels have been reached or only the highest? 

Yes the narrative needs to evidence both the level at which you have assessed your service and those below it.

In some cases it is not possible to evidence the higher level without evidencing the levels below.

However, where items are mentioned in lower levels and don’t automatically appear in the higher it is expected that the narrative and evidence captures these too.

I want to set myself a SMART target relating to achieving a level for the Outcomes Framework in the baseline year. How do I know what a realistic target might be? 

The Outcomes are not just about the LKS or individuals working within it. Rather it looks at how the organisation as a whole supports and makes use of its LKS.

Therefore, placing the responsibility for achieving certain levels in the Framework on the service – or individuals within it would be inappropriate.

The organisation will not evaluate as having a single level but rather a level per Outcome.  It may not be the case, or even likely, that the levels will be the same for every Outcome.

To set a realistic, meaningful target for any of the Outcomes in terms of a level for next year you would need to know where you are now.  Otherwise the target could mean no progress at all or even going backwards.

This would require a self-evaluation on all the Outcomes now and having this verified by HEE’s LKS leads.  HEE will not be undertaking advance verifications of self-evaluations.

If there is a desire to set an Outcomes Framework associated SMART target we would advise looking at the indicators associated with the Outcomes and linking the aims and objectives to these.

With Outcome one for example the objective might be involving specific senior stakeholders within the organisation(s) served in the development of any LKS strategy or associated plans by X date.

Will validations be subject to the same traffic light system, i.e. red, amber and green services that the LQAF was? 

There won’t be any overall “score” from the new process.

There will be a level for each Outcome – so for example Level 3 for Outcome 1, Level 2 for Outcome 2.

HEE and the LKS Leads would have concerns about any services and Trusts who don’t engage with the process or provide a self-evaluation and plans for development.

There may also be interventions from HEE where specific problems or issues have been identified.

How will we report to our senior stakeholders if there is no overall score 

The validated self-valuation for your organisation will give a level for each outcome.

This includes a visual radar chart/spider graph demonstrating current levels and over time where improvements have been achieved.

This can be shared with stakeholders. The report will highlight areas of strength and good practice against all the outcomes, as well as areas for improvement.  This will provide an accurate and more meaningful way of reporting.

If there is no overall score how will we benchmark our services? 

The Quality and Improvement Outcomes Framework is a service improvement and development tool focusing on an individual organisation. Therefore benchmarking is not appropriate.

We are working with HEE Communications and HEE Quality teams on the communications to Trusts explain the new framework.

Quality leads will understand the new framework, levels and how the new process works.

Results will be reported to the national quality team and integrated with the wider quality education processes with which Trusts are familiar.

Evidence

Are you able to share good examples from LKS services that undertook the pilot last year? 

We did for the pilot. However, some pilots followed the example too closely with the result that  they didn’t fully demonstrate their own service and organisation.

It is difficult to pull together evidence examples as each organisation is different and will be working at different levels for each outcome.

For each outcome a list of suggested evidence has been provided. Don’t see these as a prescriptive list; there may be evidence more relevant to your organisation and service.

What is the maximum age of the evidence? For example in outcome 4 if you did a role redesign in 2017 would that count? 

The evidence shows what you did and the outcomes for your service in the period being reviewed.

In this example it would only be relevant if the outcome of the role redesign was achieved during the period covered by your self-evaluation.

If you have a piece of evidence, for example an annual report, that covers all the organisations you cover how will that work if you are supposed to have a submission for each organisation?

The same piece of evidence can be used for each submission. You should ensure that, if it is generic, it covers the relevant organisations in sufficient detail to evidence the narrative at the levels you are demonstrating.

Can you use the same piece of evidence for more than one outcome? 

Yes and we would encourage this approach. Think quality not quantity and cross reference against it for the different outcomes.

Ask the “So what?” question of the evidence. Why is it relevant and what does it tell us? This will help you identify “evidence” that is not appropriate so it can be omitted.

What happens when you are incorporated into a wider team for strategy, plan etc.?  For example, I have 2 points in the corporate strategy. 

If you are restricted in this way within wider corporate documents you will need to consider how you can further outline the plans and strategies of your service.

This may include using tools such as the Plan on the Page.

Will the Strategy and annual report be assessed separately? 

No they will not. These are key documents giving a good overview of your LKS.

The strategy shows how the service is intended to develop and improve and the annual report shows what has been done over the last 12 months.

Elements of the documents could act as evidence for several of the Outcomes. Cross-reference in relevant Outcomes if needed.

Further information

We recommend you also look at the six Outcomes Framework webinars.