The University of Sheffield is a remarkable place to work. Our people are at the heart of everything we do. Their diverse backgrounds, abilities and beliefs make Sheffield a world-class university.
We offer a fantastic range of benefits including a highly competitive annual leave entitlement (with the ability to purchase more), a generous pensions scheme, flexible working opportunities, a commitment to your development and wellbeing, a wide range of retail discounts, and much more. Find out more about our benefits (opens in a new window) and join us to become part of something special.
Overview
We are recruiting a Research Associate to the ESRC-funded Metascience project AI Peer: Large language models (LLMs) and academic peer review outcomes. The project involves developing and evaluating LLM-based evaluations of published academic research to support quality control and research assessment tasks.
The role includes running a simple survey to get permission from authors to analyse their texts, managing the existing LLM pipeline to get LLM scores, and writing up the results. It also involves organising an international network of research evaluators that are interested in applying LLM technologies. The postholder will need to attend and present at meetings internationally.
Applicants will need excellent communication, networking, and organisational skills as well as some knowledge of Python, LLMs, and research evaluation or bibliometrics.
Main duties and responsibilities
Liaise with managers at the University of Sheffield to get permission to survey staff about their research outputs.
Manage a survey of research staff to get permission to use their research outputs in a study.
Obtain, securely store and process the designated research outputs, feeding them into an existing Python pipeline to obtain research evaluation scores.
Statistically analyse the results, such as with correlation tests, and write up the results for publication.
Collaborate with others in the project on shared tasks, including with project partners in Norway.
Manage an international network of researchers and others interested in the use of AI to support quality evaluations of journal articles.
Attend project meetings internationally.
Present at conferences and workshops internationally and in the UK.
Observe the project's Data Management Plan and contribute to maintaining a well-organised shared file system.
As a member of staff you will be encouraged to make ethical decisions in your role, embedding the University sustainability strategy into your working activities wherever possible.
Any other duties, commensurate with the grade of the post.
Carry out other duties, commensurate with the grade and remit of the post
Person Specification
Our diverse community of staff and students recognises the unique abilities, backgrounds, and beliefs of all. We foster a culture where everyone feels they belong and is respected. Even if your past experience doesn't match perfectly with this role's criteria, your contribution is valuable, and we encourage you to apply. Please ensure that you reference the application criteria in the application statement when you apply.
Essential Criteria
A PhD (or equivalent) in Information Science or a relevant field. [Application]
Experience of Python programming. [Application/ Interview]
Expertise in basic statistics. [Application/ Interview]
Knowledge of bibliometrics, scientometrics, research evaluation, or science policy. [Application/ Interview]
Excellent communication skills, both written and verbal. [Application/ Interview]
Ability to organise groups of people for a shared task. [Application/ Interview]
Desirable Criteria
Experience of using Large Language Models or AI as part of the methods of a research project. [Application]
Experience of working successfully with project partners [Application/ Interview]
Further Information
Grade: 7
Salary: Grade 7 - £37,999 - £46,485 per annum pro rata
Work arrangement: Full-time
Duration: Fixed-term (13 months)
Line manager:Principal Investigator
Our website: https://www.sheffield.ac.uk/is
For informal enquiries about this job contact
Mike Thelwall, Principal Investigator : on m.a.thelwall@sheffield.ac.uk
Next steps in the recruitment process
It is anticipated that the selection process will take place on 9th May 2025. This will consist of a short presentation and an interview. We plan to let candidates know if they have progressed to the selection stage on the week commencing 28th April 2025 If you need any support, equipment or adjustments to enable you to participate in any element of the recruitment process you can contact inf-operations-team@sheffield.ac.uk .
Our vision and strategic plan
We are the University of Sheffield. This is our vision: sheffield.ac.uk/vision (opens in new window).
What we offer
A minimum of 41 days annual leave including bank holiday and closure days (pro rata) with the ability to purchase more.
Flexible working opportunities, including hybrid working for some roles.
Generous pension scheme.
A wide range of discounts and rewards on shopping, eating out and travel.
A variety of staff networks, providing opportunities for social interaction, peer support and personal development (for example, Race Equality, LGBT+, Women's and Parent's networks).
Recognition Awards to reward staff who go above and beyond in their role.
A commitment to your development access to learning and mentoring schemes; integrated with our Academic Career Pathways.
A range of generous family-friendly policies
paid time off for parenting and caring emergencies
support for those going through the menopause
paid time off and support for fertility treatment
and more
More details can be found on our benefits page: sheffield.ac.uk/jobs/benefits (opens in a new window).
We are a Disability Confident Employer. If you have a disability and meet the essential criteria for this job you will be invited to take part in the next stage of the selection process.
Closing Date : 27/04/2025
We are a research university with a global reputation for excellence. Our ideas and expertise change the world for the better, making a real difference to society. We know that when people come together with different views, approaches and insights it can lead to richer, more creative and innovative teaching and research and the highest levels of student experience. Our University Vision ( www.sheffield.ac.uk/vision ) outlines our commitment to building a diverse community of staff and students that recognises and values the abilities, backgrounds, beliefs and ways of living for everyone.