REF 2014 results

The Research Excellence Framework (REF) was introduced in 2014 as a new system for assessing the quality of research in UK higher education institutions (HEIs). The REF 2014 results provide four quality profiles for each submission: Overall, Outputs, Environment, and Impact. Each profile gives the percentage of the submission judged to be of 4*, 3*, 2*, and 1* quality. This page provides a graphical presentation of these results.

Volume

The volume of work in each submission is measured by the number of category A staff submitted (CATA). For each profile, we compute the volume of work at each level by multiplying the profile percentages by CATA. These volumes measure concentrations of research excellence. They are also used to determine one component of Research Funding in the UK. For example, the Scottish Funding Council (SFC) Research Excellence Grant (REG) has recently been allocated according to a score that allocates a weight of 3.11 to 4* volume, and a weight of 1 to 3* volume. This site allows you to compare the effects of various weightings.

SFC is currently reviewing the underlying model for the distribution of REG (deadline for responses 14th Jan). The volume computation with SHEFC weighting given here corresponds to the Vstaff ⨉ Q component of the current allocation model (Vstaff ⨉ C + X) ⨉ Q). It does not include the Weighted Other Activity Indicators (X) nor the Cost Factor (C).

Efficiency

To compare submissions of different sizes, we could simply compare the raw percentages from the quality profiles. However, the number of staff submitted (CATA) is generally smaller than the number of staff eligible for submssion, and the proportion of eligible staff submitted varies widely.

Instead, we express the volume of research at each level as a percentage of the FTE number of staff eligible for submission, as given by the Contextual data for the Research Excellence Framework 2014 provided by the Higher Education Statistics Agency (HESA). We then provide weighted comparisons based on these percentages. This provides a measure of efficiency, or research productivity.

The HESA data carry important health warnings. In particular, "There are known to be staff employed by the Colleges of Oxford, Cambridge and the University of the Highlands and Islands, and the Institute of Zoology who are eligible for submission to REF but are not included in the HESA Staff Record."

Where the CATA number of staff submitted is greater than the HESA number of FTE eligible staff, and for the four submissions for which HESA gives no data, we use the CATA number as an adjusted FTE, but this may still be an underestimate of the true number of eligible staff.

From a funding perspective, our efficiency figures correspond to the funding per eligible FTE that a given submission would attract.

We also provide an option, "per CatA" for the use of CATA as the denominator.

Comparisons

You can focus on a particular UoA and a particular institution, and choose a criterion for comparison—volume or efficiency—using various weightings for the different quality levels.

We give four graphs.

• The first graph presents the distribution of research quality and volume by unit of Assessment. There is an entry for each UoA. The overall quality profile varies significantly from one UoA to another.
• The second graph presents the distribution of research quality and volume across institutions. There is an entry for each institution with a 4* volume of at least 20 FTE (summed across all UoA).
• The third graph shows the results for a chosen UoA. It has an entry for each institutional submission, or pair of institutions making a joint submission. Using the selector, you can change the UoA.
• The final graph compares the results for a chosen institution, for each UoA, with the results averaged over the chosen region. This compensates for the differences in quality profiles for different UoA, to allow the relative performance of submissions in diferent UoA to be compared. When "volume" is selected it shows the volume and score as a percentage of the total volume across the selected region; When "efficiency" is selected, the bars still show volume as a percentage of the total, but the discs show the score as percentage of the average score across the selected region. There is an entry for each submission (split submissions are shown separately). Using the selector, you can focus on a particular institution.

By defaut, the last two of these show results across the UK, but you can use the selector to focus on a particular region.

Presentation

The controls allow you to select the profile (overall, outputs, environment, impact, see the Assessment criteria and level definitions) you want to view, to compare volume or efficiency, and to select a sorting criterion. Each criterion corresponds to a particular weighting of the different starred levels 4*—1*. The selected weights are shown graphically by the lengths of the coloured bars in the legend.

Each entry consists of a bar and a disc. Each bar shows the adjusted FTE staff eligible for submission; the coloured segments show the volume of research classified at each of the four levels, with 4* at the base. These volumes are measured in units of FTE staff, shown on the left-hand vertical axis. Each disc shows the score for the selected criterion, referred to the scale on the right-hand vertical axis.

The entries are ordered accordingly to the selected criterion—by default by Research Fortnight Power, which corresponds to the (3, 1, 0, 0) weightings for 4*, 3*, 2*, and 1* research volume used for funding by HEFC (quality weighting for mainstream QR funding). The SHEFC model uses slightly different weightings of (3.11, 1, 0, 0).

The weighting in use is shown graphically by the lengths of the bars in the legend.

You can click on the bar for each entry to see the corresponding data and score.

The bookmarking button (top-right) will add parameters to the URL. Copying this url will enable you recreate the settings you have chosen, or send them to others.