Advice on RAE 2008 Outputs

For RAE 2008, each submitted member of research active staff (so called, Category A or D) will nominate up to four research outputs. This note provides some background to help you select and optimise these outputs.

This advice draws on the Criteria and Working Methods documents of Panel F and Subpanel 23, the panels that mostly concern us, as well as generic RAE guidance.

What is a Research Output?

Most research outputs in our field will be research papers that have been published in journals, books or conferences. However, many other kinds of output are allowed. This list is taken from Subpanel 23's criteria.

"All forms of research output will be treated equally. The sub-panel acknowledges the breadth of technology transfer and dissemination practice in computer science and informatics. Consequently it will accept outputs in any form including, but not necessarily limited to books, chapters in books, articles in journals, conference contributions; creative media and multimedia; standards documents; patents, product or process specifications; items of software, software manuals; and technical reports, including consultancy reports or independent evaluations."

How are these Research Outputs Assessed?

The result of the 2008 RAE will be the assignment of an overall profile to each submission. This will consist of 100 percentage points divided between each of the five categories: unclassified, 1*, 2*, 3* and 4*, where 4* is the top category. This overall profile is a weighted sum of three sub-profiles on the same scale, namely for: research outputs, environment and esteem. The weight for research outputs in our area is 70%, i.e. it is by far the most important of the three sub-profiles.

The profile for research outputs is calculated by assigning a category on the same five-point scale to each of the submitted outputs. Each output will be examined by each of the three or more spokespeople assigned to each submission. At least 25% of the outputs will be examined in detail by at least one of these spokespeople. Note that no score or profile will be assigned to either individuals or research groups --- only scores to individual outputs by each spokesperson en route to assigning a profile to each submission.

Getting a good profile is enormously important to the School and the University, both because our RAE profile will determine a great deal of our income and because of the kudos it will bestow. It is thus vital that we submit the very best research outputs that the School has produced during the census period.

How to Choose Your Research Outputs

To be eligible for submission as one of your four research outputs, an output must have been published in the census period of 1st January 2001 to 31st December 2007, and you must be one of the co-authors. For non-paper outputs the meaning of "publication date" may have to be discussed with the RAE Coordinator.

Research outputs will be assessed on the three criteria of originality, rigour and significance. You should choose four outputs that score highest on these criteria. No particular kind of output is preferred, a priori, over any other, e.g. there is no ranked list of journals or conferences, electronic journals, hard copy journals and conferences are treated equally. However, it is clear that RAE panel members will have to examine a vast number of papers in a short period. They will also have to examine papers on topics on which they are not experts. It is, therefore, inevitable, that they will use a variety of heuristic measures in assessing papers. You should be sensitive to the likely heuristic measures when both preparing and selecting your RAE submissions.

Among the heuristic measures most likely to be used are: what you say about the impact of your work in the "why I chose this output" text box, title, abstract, introduction and conclusion of a paper; the reputation, impact factor and success rate of the outlet; the citations to and citation index of your output; the appropriateness of your chosen methodology to establishing any claims you make; etc. You can look up the impact factors of journals in your research area and citation indices of your papers in ISI's Web of Knowledge. Google Scholar is also a useful source of information about citations to your papers. The Publish or Perish software provides a nice interface to the Google citation information.

How Do I Nominate a Research Output?

Research paper outputs should be entered in the Informatics@Edinburgh Publications Portal. The submission form contains fields for nominating a paper as an RAE submission, estimating a star ranking for it and explaining why you chose it. Filling in this last field is especially important, as its claims will guide readers as to how to assess your paper. There are web pages giving special instructions on submitting RAE papers to the Portal, including estimating star rankings and also on filling in the "why I chose this paper" field.

The RAE relevant information will not appear in the publication portal, but will be stored in the School Database and eventually used to populate our RAE submission. In the preparation for the RAE it will appear in an RA2 form for each individual, where it will inform preparatory discussions between those preparing our submission and the potential Category A, etc staff. You can access your own form by clicking on your name in the potential Category A staff list.

Non-paper outputs will also be stored in the School Database and entered on the RA2 forms, but they will not appear in the publication portal. To nominate them, send the details to Rosemary Soutar. As far as possible, provide fields analogous to those for paper-based outputs, namely: title, co-authors, publication details, url, estimated star ranking, why I chose this output. Depending on the nature of the output, some of these may not make sense or additional information may be appropriate. Please use your common sense and discuss any problems with Rosemary.

Please also submit software outputs to the School of Informatics Software Download Database (ISDD). Then provide a link to its ISDD entry as your url when sending the details about it to Rosemary. The ISDD will help us to monitor downloads of your software and, hence, measure its impact, which is vital for RAE assessment. If you have any questions about the ISDD, please refer to the FAQ.

Tips on Preparing and Selecting Research Output

Here are some tips you might want to follow when preparing research outputs for RAE submission and/or choosing which outputs to submit.

Special Circumstances

Normally, if someone submits fewer than 4 outputs then the School will be punished by having the 'missing' outputs classified as 'unclassified', which will adversely affect our RAE profile.

However, exceptions will be made for those staff who, for one reason or another, were unable to be research active for a significant part of the census period. You can find more details in the document about special circumstances. I have extracted possible grounds for exemption from the RAE's generic statement on criteria and working methods.

"Panels will consider the following individual circumstances to the extent that they are stated to have had a material impact on the individual's ability to produce the expected volume of research outputs in the assessment period:

If you think you may qualify for such an exemption, then please email Tamise Totterdell with (a) the grounds for the exemption and (b) the period(s) over which this operative. Obviously, this information may be confidential -- and we will restrict it to those who need to know for RAE preparation purposes. However, you don't need to go into unnecessary detail, e.g. you should tell us you were on sick leave, but not what the illness was. If you can, just cut and paste the grounds from the list above then that is fine, but the list is not meant to be exhaustive. The wording will appear on your confidential mock RA2 form. Check this wording to ensure you are happy with it.

Alan Bundy
Last modified: Mon Aug 27 13:17:03 BST 2007