Advice on RAE 2008 Outputs

This advice draws on the Criteria and Working Methods documents of Panel F and Subpanel 23, the panels that mostly concern us, as well as generic RAE guidance.

What is a Research Output?

"All forms of research output will be treated equally. The sub-panel acknowledges the breadth of technology transfer and dissemination practice in computer science and informatics. Consequently it will accept outputs in any form including, but not necessarily limited to books, chapters in books, articles in journals, conference contributions; creative media and multimedia; standards documents; patents, product or process specifications; items of software, software manuals; and technical reports, including consultancy reports or independent evaluations."

How are these Research Outputs Assessed?

The profile for research outputs is calculated by assigning a category on the same five-point scale to each of the submitted outputs. Each output will be examined by each of the three or more spokespeople assigned to each submission. At least 25% of the outputs will be examined in detail by at least one of these spokespeople. Note that no score or profile will be assigned to either individuals or research groups --- only scores to individual outputs by each spokesperson en route to assigning a profile to each submission.

Getting a good profile is enormously important to the School and the University, both because our RAE profile will determine a great deal of our income and because of the kudos it will bestow. It is thus vital that we submit the very best research outputs that the School has produced during the census period.

How to Choose Your Research Outputs

Research outputs will be assessed on the three criteria of originality, rigour and significance. You should choose four outputs that score highest on these criteria. No particular kind of output is preferred, a priori, over any other, e.g. there is no ranked list of journals or conferences, electronic journals, hard copy journals and conferences are treated equally. However, it is clear that RAE panel members will have to examine a vast number of papers in a short period. They will also have to examine papers on topics on which they are not experts. It is, therefore, inevitable, that they will use a variety of heuristic measures in assessing papers. You should be sensitive to the likely heuristic measures when both preparing and selecting your RAE submissions.

Among the heuristic measures most likely to be used are: what you say about the impact of your work in the "why I chose this output" text box, title, abstract, introduction and conclusion of a paper; the reputation, impact factor and success rate of the outlet; the citations to and citation index of your output; the appropriateness of your chosen methodology to establishing any claims you make; etc. You can look up the impact factors of journals in your research area and citation indices of your papers in ISI's Web of Knowledge. Google Scholar is also a useful source of information about citations to your papers. The Publish or Perish software provides a nice interface to the Google citation information.

How Do I Nominate a Research Output?

The RAE relevant information will not appear in the publication portal, but will be stored in the School Database and eventually used to populate our RAE submission. In the preparation for the RAE it will appear in an RA2 form for each individual, where it will inform preparatory discussions between those preparing our submission and the potential Category A, etc staff. You can access your own form by clicking on your name in the potential Category A staff list.

Non-paper outputs will also be stored in the School Database and entered on the RA2 forms, but they will not appear in the publication portal. To nominate them, send the details to Rosemary Soutar. As far as possible, provide fields analogous to those for paper-based outputs, namely: title, co-authors, publication details, url, estimated star ranking, why I chose this output. Depending on the nature of the output, some of these may not make sense or additional information may be appropriate. Please use your common sense and discuss any problems with Rosemary.

Please also submit software outputs to the School of Informatics Software Download Database (ISDD). Then provide a link to its ISDD entry as your url when sending the details about it to Rosemary. The ISDD will help us to monitor downloads of your software and, hence, measure its impact, which is vital for RAE assessment. If you have any questions about the ISDD, please refer to the FAQ.

Tips on Preparing and Selecting Research Output

• Be sure to emphasise not only the content, but also the impact of your work in title, abstract, introduction, conclusion and "why I chose this output" field. I have provided a list of impact indicators to help you. It's not enough just to say what you did -- you must say why it is significant.
• Avoid selecting outputs that are similar, or that make incremental improvements on each other. Doing so will adversely affect your originality rating.
• Be generous on co-authorship. This will increase our flexibility when it comes to choosing outputs to submit. It is anyway a Good Idea, since it ensures that nobody who has contributed to the work feels slighted and usually repays itself when your co-authors return the generousity. Washington University in St Louis has a policy on authorship that reflects the scientific consensus on who should be the authors of a paper and what their rights and duties are.
• The RAE has made it far more important to have four 4* publications than forty 1* publications. Aim for the top quality outlet that your research achievements will support.
• Try to include at least one journal paper, if you can.
• Consider whether one or several short conference or workshop papers can be refined into an archival publication in a major journal.
• Workshop papers would normally be categorised as 1* or at best 2*, so should be avoided, if possible. Sometimes a high-quality conference is called a "Workshop" for historical reasons. If so, you should explain this in the "why I chose this paper" field.
• If you can do so without compromising on quality, it is good to have papers drawn from throughout the census period, rather than restricted to a small subset of the period.
• Similarly, a mixture of single and multi-authored papers is desirable, provided you don't compromise on quality.
• Avoid submitting papers that other people from Informatics have submitted. Although such duplication is allowed, it will be frowned upon by our Subpanel. You may need to discuss with your co-authors how to divide up your papers between you. Remember, we are trying to maximise income to the University. Individuals will not be ranked. So give up your best paper to a co-author if this ensures that the School's best papers can all be submitted. Note that duplication across different universities or different units of assessment is fine.

Special Circumstances

Normally, if someone submits fewer than 4 outputs then the School will be punished by having the 'missing' outputs classified as 'unclassified', which will adversely affect our RAE profile.

However, exceptions will be made for those staff who, for one reason or another, were unable to be research active for a significant part of the census period. You can find more details in the document about special circumstances. I have extracted possible grounds for exemption from the RAE's generic statement on criteria and working methods.

"Panels will consider the following individual circumstances to the extent that they are stated to have had a material impact on the individual's ability to produce the expected volume of research outputs in the assessment period:

• a. Family and domestic matters, including:
  • i. Absence on maternity, paternity, parental or adoption leave and arrangements on return to work following these periods of leave.
  • ii. Part-time working or other flexible working arrangements.
  • iii. Time spent acting as a carer or other domestic commitments.
• b. Disability, ill-health and injury, including:
  • i. Any disability to which the Disability Discrimination Act 1995 applies, including both permanent disabilities and any temporary disability with a duration of 12 months or more.
  • ii. Absence from work on the advice of a registered medical practitioner.
• c. Engagement on long-term projects of significant scale and scope.
• d. Status as an early career researcher. These are individuals of any age who first entered the academic profession on employment terms that qualified them for submission to RAE2008 as Category A staff on or after 1 August 2003.
• e. Prolonged absences (absences for more than six months consecutively in the assessment period) which were agreed by the individual with the institution but which do not fall into one of the categories above. They include:
  • i. Secondment to non-academic positions outside the higher education sector.
  • ii. Career breaks for purposes unconnected with research, teaching or other academic duties.
• f. Other absences which the institution is legally obliged to permit, such as absences for religious observance or absence arising out of involvement as a representative of the workforce.
• g. Any other personal circumstances which are considered to have had a significant impact on an individual's ability to produce the expected volume of research outputs in the assessment period."

If you think you may qualify for such an exemption, then please email Tamise Totterdell with (a) the grounds for the exemption and (b) the period(s) over which this operative. Obviously, this information may be confidential -- and we will restrict it to those who need to know for RAE preparation purposes. However, you don't need to go into unnecessary detail, e.g. you should tell us you were on sick leave, but not what the illness was. If you can, just cut and paste the grounds from the list above then that is fine, but the list is not meant to be exhaustive. The wording will appear on your confidential mock RA2 form. Check this wording to ensure you are happy with it.