In recent years there has been a (welcome) trend towards conducting research “with” or “by” the public rather than “to”, “about” or “for” them (NIHR). While there is a move to standardize this across all health research (for example, many funders now require evidence of patient and community involvement), there remains disparity in the types and stages of research involving the community.
Although patient and community participation is advocated at all stages of the research cycle (NIHR), such as the selection of outcome measures, item development and constructability, experience inputs are rarely experienced (Wiering, de Boer & Delnoij, 2017). This gap poses a major challenge for health research. Why should only researchers, funders or policy makers decide which outcomes are most important? This approach risks leading us astray— How can we make sure we prioritize the right questions and measure the right outcomes? For example, instead of just evaluating whether a new talk therapy reduces clinical symptoms of depression, might there be deeper, more fundamental aspects to people's experiences that we should be investigating?
This blog explores this particular issue by examining a systematic review (2025) by Molloy and colleagues published today in The Lancet Psychiatry. “Identify currently used mental health outcome measures that meet a strict definition of co-operation.” (p. 2).
Is our current approach to selecting outcome measures leading us down the wrong path?
Methods
The authors (including three who identified as having a mental disorder) conducted a systematic review of papers describing the development of new disorders. Patient Informed Outcome Measures (PROMs) using quantitative, qualitative or mixed methods. Searches were conducted in MEDLINE, Web of Science, Scopus, PsycINFO, and Embase, as well as a gray literature search. Studies are excluded if: they are not primarily in a mental health population or condition; did not describe original development or psychometric testing and used collaboratively developed PROMs as endpoints for data collection. All articles were screened independently by two authors at both title, abstract and full paper stages.
The measures included were rated on a scale of one to three, where one indicated consultation with the live practice team, two indicated the presence of a service user-researcher in the team, and three where both were evident. No points were given if there was no report of lived experience. Assessments were provided at each stage of collaborative scale development, including item generation, item feedback, psychometric testing, and writing. The GRIPP2-SF (Staniszewska et al., 2017) was used to assess the quality of reporting of lived experience participation in the included papers.
Results
Total 34 articles describing 23 PROMs they had the right to enter the review. Most of the measures were specific to mental disorders and were designed for adults in the UK. On the GRIPP-2 SF checklist, documents had an average score of 9 (good), with only two indicators rated as excellent. Overall, the amount of experience involvement was highest in item development and lowest in the psychometric testing phase.
for generation of itemsservice user researchers were the most common type of involvement (11 PROMs), although some PROMs included advisory groups (n=4), steering committees (n=2), reference groups (n=2) or advisory groups (n ) included. =1). Eight PROMs did not identify specific lived experience groups, but described participation through focus groups or qualitative interviews.
Feedback on items collected through focus groups, interviews and Delphi exercises. For 11 PROMs, such feedback led to modification of items, removal of items, and formatting of final measures.
Thirteen PROMs involved people with expertise in tasks beyond completing measures. Other ways in which people with lived experience are involved psychometric test phase including designing feedback forms, selecting comparison measures, reviewing analysis results, and refining items.
At least one experienced person participated write and distribute 14 for PROM.
This study shows that the UK is leading the way in collaborative development of outcome measures in mental health research.
Results
Traditional methods of measurement development often exclude people with past experience or involve them only minimally – for example, by including service users in item creation or a qualitative focus group, but without any further contact before or after these activities. But this review showed it It is both possible and beneficial to engage meaningfully with people who have prior experience in developing PROMs.
It is both possible and beneficial to meaningfully collaborate with people who have past experience in the development of patient-reported outcome measures.
Strengths and limitations
It was a well-executed, well-conducted systematic review that demonstrated a thorough and rigorous commitment to methodology. From the initial conception of the study, it is advisable to involve those with expertise in the research team; This paper clearly focuses on the experienced and that's great to see.
However, it would be nice to see more justification from the authors regarding the scoring criteria they chose to rank the level of participation in the included articles. Is a service-using researcher always a better form of participation than a lived experience group? Maybe so, but it would be nice to know a little more about what led to this decision and how much experience really played into it.
Overall, the main limitations of this study exist in the composition of the studies reviewed in this review; It is unfortunate to see so few of the included papers receiving an “excellent” rating on the GRIPP-2 SF measure.
Are service-using researchers necessarily a better form of participation than practice groups? Maybe, but more justification is warranted.
Implications for practice
Personally, I think that the conversation around user results is not new. Indeed, I remembered this excellent article published in 2017 by one of our fantastic Elves. Alison Faulknerhe notes:
There is little research on issues of identity, considerations of queer culture, or the impact of treatments/services on user-defined outcomes other than service- or symptom-related. Interestingly, this contrasts significantly with Disability Studies, where interdisciplinary work including and by people with disabilities, plus respect for first-person narratives, results in respect for knowledge from “experts on the ground” – in this case disabled people. people. (Faulkner, 2017)
I'm glad to see that we're now entering a period where we're measuring and reviewing the ground level in this area, but I can't help but feel that more is needed. Even at the most basic level, I would ask researchers to consider this; Have you ever asked people who lived what they thought the main results of your research should be? I started doing this in my PhD (Hemming et al., 2021; Nedoma, 2021) and continue to do so in every research I receive funding for. The reaction I get from other researchers is always surprise and congratulations – but I would argue that this has become commonplace by now and shouldn't come as a surprise to other researchers.
Have you ever asked experienced people what they think the main results of your research should be?
Statement of interest
The author has no interest to declare.
Connections
Primary paper
Molloy, N. Kilcoyne, I., Belcher, H. & Wykes, T. (2025). Exploring the participation of people with experience of mental health disorders in the co-development of outcome measures: a systematic review. Lancet Psychiatry. 10.1016/S2215-0366(24)00376-6
Other references
Faulkner, A. (2017). Survivor research and Mad Studies: the role and value of experiential knowledge in mental health research. Disability and Society, 32(4), 500-520.
Hemming, L., Shaw, J., Haddock, G., Carter, LA, & Pratt, D. (2021). A cross-sectional study examining the relationship between alexithymia and suicidality, violence and dyadic harm in male prisoners. Frontiers in Psychiatry, 12670863.
Nedoma, R. Alexithymia and suicidality, violence and dual harm in male prisoners. Mental Elf, September 2021.
Staniszewska, S., Brett, J., Simera, I., Seers, K., Mockford, C., Goodlad, S., … & Tysall, C. (2017). GRIPP2 reporting checklists: tools for improving reporting of patient and community involvement in research. bmj, 358.
Wiering, B., de Boer, D., & Delnoij, D. (2017). Patient involvement in the development of patient-reported outcome measures: a scoping review. Health Expectations, 20(1), 11-23.