Readiness for REF

3

Dr Mark CoxKing’s College London

What is REF?

4

• Research Excellence Framework• Successor to RAE 2008 – Research Assessment Exercise• Peer assessment of the quality of research, split into specific

subject areas – Units of Assessment (UoA)• Judged on a number of factors:

– individual research outputs– research funding– postgraduate student numbers and funding– research “environment” and esteem

• Two significant new factors in this exercise: bibliometrics and research “impact”

Why is REF important?

5

• Main factor in determining the amount of money received from UK central government by institutions to fund research

• Used by parts of the media in their calculations of institutional league tables

• Institutional reputation• Can show strengths and weaknesses within

multidisciplinary institutions

How is data collected?

6

• Each institution prepares data for each of the the UoAs it wishes to submit to

• Data is “person-centric” – if a staff member is submitted to a UoA, the funding they have obtained and the students they supervise count towards that UoA

• Information on environment and impact is/will be prepared as narrative for each UoA

• Data is uploaded and/or directly entered to central data collection system hosted by HEFCE

Research Management at UK institutions• Many different systems!• Most institutions have central business

systems: HR system, student system, finance system

• Many institutions have an Institutional Repository, usually hosted by library staff, holding details of staff publications

• Some institutions have in-house built research databases, sometimes integrated with other systems

• A few institutions have purchased commercial CRIS systems

7

Streamlining research management

8

• Diversity of systems across the UK makes collaboration difficult

• Institutions find themselves “reinventing the wheel”• Locally built systems require expertise; loss of this

expertise can be a serious blow• As the requirement for data relating to research

increases, many institutions are looking for a more co-ordinated approach to data management

• There is a growing appreciation that collaboration and harmonization will provide greater value for money, both for institutions and for UK research in general

Readiness for REF (R4R)

9

• The R4R Project between King’s and Southampton is examining CERIF as a the basis of a mechanism for delivering data to the REF

• At the core of this will be a “lightweight “ data schema based on the CERIF model, adapted to the needs of data collection for the REF – “CERIF4REF”

• The desired outcome is that the CERIF data model will:– form the “glue” that will hold disparate information systems

together– deliver repository and CRIS interoperability for the REF– allow for interoperability and exchange of data within and across

institutions, including external systems

Why CERIF?

10

• Institutions do not have the resources to develop a completely new UK data model

• CERIF has been successfully used as the basis for a number of national systems across Europe

• CERIF has evolved over ten years via the experience of developers across Europe

• euroCRIS, as the custodians of the CERIF standard, form an active and enthusiastic community which encourages participation in the development of the model

R4R work streams

11

• Carry out an analysis of current practice in UK institutions

• From REF requirements, develop the CERIF4REF data schema

• Evaluate CERIF4REF schema applicability• Examine mapping this data model to external sources• Create “plug-ins” for the three most common types of

repository (ePrints, DSpace, Fedora)• Demonstrate data exchange between systems at

King’s and Southampton

R4R: Analysis of Current Practice

12

• Analysis of a “Towards the REF” survey, gathered from attendees at REF conferences at King’s– what current systems are in place?– do you have an institutional repository?

• Many institutions had a repository, and many of these had been used to support the RAE

• Discussions were held with institutions on data models in repositories and CRISes

• Where institutions had some from of CRIS, many had recently started to look the CERIF data model

R4R: Develop CERIF4REF schema

13

• Prior to definition of final REF requirements, an initial model was developed against data required for RAE2008

• A mapping of RAE2008 to CERIF indicated where REF elements may be missing in CERIF

• Discussions with euroCRIS developers helped resolved a number of the unmapped elements

• “Date of birth” element added to most recent release of CERIF

• CERIF4REF XML schema produced, including stylesheets to convert CERIF4REF data to RAE upload XML or native CERIF

R4R: Evaluate R4R schema

14

• CERIF4REF model used to create Oracle database on King’s systems

• Data mapping to existing King’s CRIS (Research Gateway) carried out

• CERIF4REF database successfully populated from King’s Research Gateway system

• CERIF4REF model being used to inform development of local ePrints repository at Southampton

• Case studies carried out with a number of other institutions to determine if CERIF4REF elements could be populated from existing systems

• Agreement that CERIF4REF covered a majority of areas

R4R: External Sources

15

• Publication data (Thomson Reuters)– Tool developed at Southampton to allow population of

repositories from Web of Science (WoS) basic API via the SWORD protocol

– This may circumvent the need to map this data to CERIF

• UK Research Councils– RCUK are developing a new system to collect research

output from institutions; Research Outcomes Project– Discussions have recently been held with members of this

project team to determine the overlap of the ROP requirements with the R4R developments

R4R: Current work

16

• Repository plug-ins– Now working with developers at Southampton

(ePrints), Edinburgh (DSpace) and Oxford (Fedora) to start development of these tools

• Data exchange– Hoping to use real data – Data on staff and publications likely to be

reasonably simple– Data on awards and students will be more

complex

R4R: Conclusions so far

17

• In the UK, there is much greater awareness of, and interest in, the role of CRISes in research management and the CERIF model

• CERIF being increasingly viewed as the best solution for data harmonization within the UK (EXRI report, JISC RIM agenda)

• CERIF4REF model has potential to provide a step towards greater CERIF uptake in UK institutions

R4R: Conclusions so far

18

• Work in this area has highlighted the need for different areas of institutions to communicate better

• The delay in the release of the final REF requirements has so far precluded any development in the areas of research impact and bibliometrics

• Amongst the biggest hurdles to widespread development are a lack of core semantics, and a workable national and/or international system of staff identifiers

Finally – thanks to…

19

• JISC • King’s team

– Sheila Anderson– Richard Gartner– Sudden Martes

• Ex-King’s– Mary Davies– Stephen Grace

• Southampton team– Tim Brodie– Les Carr

• euroCRIS– Keith Jefferies– Brigitte Joerg– Geert van Grootel

And many others – apologies to anyone I’ve missed!

Questions?

?20

21