“It cannot be boring!”: Developing a measure of function for young adults accessing integrated youth services
Journal of Patient-Reported Outcomes volume 6, Article number: 92 (2022)
In Canada, the prevalence of mental health challenges is highest in young people aged 12–24. Mental health challenges frequently cause marked functional impairment. Despite this, we are unaware of any existing conceptualization and/or measures of function that has been developed from the perspective of young people. The objective of this paper is to develop a conceptual and measurement model, including a preliminary set of items, for an outcome measure of function for young adults accessing mental health services.
We conducted this study in three phases. In phase 1, we conducted three focus groups to conceptualize function as a construct from the perspective of young adults. In phase 2, we co-designed a set of items with youth (n = 4) to capture the construct. In phase 3, we invited young people (n = 12) accessing mental health services to complete workbooks and participate in one of two focus groups to evaluate whether items were clear, captured function comprehensively, and were relevant. We transcribed and compiled all data to eliminate, refine and generate new items.
In phase 1, a conceptual model of function was developed with three main themes: basic needs, roles and responsibilities, and social connections. In phase 2, 97 candidate items were developed, and in phase 3, a candidate pool of 50 items resulted for psychometric testing.
This youth-centred conceptualization of function and preliminary item bank has the potential to advance person-centred care, outcomes, and experiences for youth seeking mental health services.
An estimated 1 in 5 Canadians experience mental health issues annually, with the highest prevalence in those aged 15–25 . Mental illness costs the Canadian healthcare system upwards of $51.1 billion in direct healthcare services, excluding peripheral costs associated with caregiving and lost employment . In response, British Columbia has responded by developing an integrated primary model of care to support the needs of youth (12–18 years), young adults (19–25 years), and their families/caregivers. The service model is called Foundry (foundrybc.ca). Since 2018, Foundry has provided integrated primary care (including sexual health), mental health, substance use, peer support, and social services (i.e., work/study support) to nearly 40,000 young people. Currently, twelve Foundry centres exist in British Columbia, with eleven more in development .
Early learnings at Foundry highlight that ‘function’ is an important outcome for young people, their families, service providers, and funders [3–6]. Function, including physical, psychological, social, and occupational function, has been shown to have a significant association with health (mortality and morbidity) and social outcomes (i.e., social relationships, living circumstances, education and employment [4, 5, 7, 8]. Youth and young adults with mental health challenges frequently face barriers to function, including barriers to education/employment, safe and affordable housing, and timely access to health services, [9–11]. They also face barriers to communicating function to health providers, including a lack of youth-centred measures [4, 5], lack of co-designed measures that are culturally relevant , and measures that create a common language for guiding care . “Function”, as it is currently conceptualized and measured, is often associated with mental well-being and/or quality of life [14–20]. However, to measure function from the perspective of youth and young adults, a hypothesized conceptual framework and youth-centred language of outcomes is needed. As Patrick and colleagues  note in their landmark paper Patient-Reported Outcomes to Support Medical Product Labeling Claims: FDA Perspective, we must “Begin with the End in Mind” (p.S127). The goal of this project is to develop a youth-centred model and measure of function that enable treatment plans that are fit for purpose for this population and driven by what is meaningful to diverse youth and young adults .
The universal model of function, the International Classification of Function (ICF) defines the term as “all bodily functions, activities and participation,” and conceptualizes these facets as interacting with contextual and individual factors (World Health Organization [WHO] , p. 2). Since the model depicts all activities and participation as equally important to function, it is not surprising that current measures of function measure varying constructs. However, existing measures of function are predominantly clinician-reported, including the Global Assessment of Functioning (GAF), the Personal and Social Performance Scale (PSP), the Social and Occupational Functioning Assessment Scale (SOFAS), and the Life Functioning Assessment Inventory (L-FAI) [24–28]. Patient-reported measures that do exist [(i.e., WHODAS, Kessler Distress Scale (K10)] focus on general domains of function relevant to the adult population or are designed to capture epidemiological trends rather than designed to evaluate the effectiveness of interventions over time [29–31]. The substantial gap between the outcomes captured in research and in youth-centred health care are increasingly shown to be detrimental for youth, providers, clinics, and health care systems. At this time, there is a need for a measure of function that can be used to evaluate the effectiveness of integrated youth services and to inform quality improvement initiatives . It is also paramount that a new measure considers the modern day priorities of youth, including considerations for equity and diversity, co-deign methods, and including those who may have barriers to accessing services in the first place [12, 33].
Patient Reported Outcome Measures (PROM’s) are one type of assessment tool that can help young people communicate concerns to their healthcare providers and monitor their own progress . By bolstering young peoples’ self-efficacy through meaningful measurement, assessments of function can empower young adults and families to engage in care that is contextualized based on their needs and priorities . The objective of this study was to develop an initial item set to capture the concept of “Function” from the perspective of young people accessing integrated health services.
This study describes the first three phases of measurement development as depicted in Fig. 1.
Conceptualization of function (Phase 1)
We used a qualitative study design guided by focus-group methodology to answer our research questions. Before beginning, we hired a youth research lead (MK) and a team of youth research assistants to co-design the interview schedule and to suggest opportunities to maximize opportunities for diverse youth to participate in the project. The team co-designed the questions (see Additional file 1: Table S1). Using posters, we championed the project and recruited young people from an integrated youth service network called Foundry (foundrybc.ca) that provides health and wellness services and support to youth and young adults with mental health and substance use disorders in British Columbia, Canada. For this study, we recruited young people from one centre located in Vancouver, an urban centre, between November and February 2018/2019, with the goal of first identifying function from the perspectives of young adults who experience mental health and/or substance use disorders. Participant inclusion criteria was young people between 19 and 25 years of age diagnosed with a mental health and/or substance use disorder, willingness and ability to read and respond in English, and provide informed consent. Based on youth engagement and a review of the literature, it was felt to be important to focus uniquely on young adults separately before youth, as their developmental needs were different (Kwan et al. ). Prior to study enrollment, participants were explained the study purpose, expectations of them as a participant, and given contact information of the lead author and University of British Columbia (UBC) research ethics board.
Our team, including two trained youth research assistants and the primary investigator, conducted two 90-min focus groups, with a plan for more if saturation was not achieved. We aimed to have 5–9 participants in each focus group to allow for meaningful participation of each young person and to account for the new experience of youth with this methods. Participants were invited to one focus group. We recorded and manually transcribed the focus groups verbatim. Two team members and two participants (one for each focus group), reviewed the transcripts. Subsequently, two researchers not present for the focus groups, analyzed transcripts to code for themes, employing an inductive thematic approach to identify themes representative of function in young adults. To assess preliminary evidence of saturation, we reviewed each focus group discussion in order in which groups were conducted and documented the development of codes. Specifically, we documented all new codes that were developed, their characteristics, including the code definitions, type of code, notes about rationale for new codes (youth-centred, clarity of the code, completeness of the code), and whether any previously developed codes were present in the transcripts. We also took notes on the evolution of the coding, recording changes made to the codes and rationales where appropriate. As outlined by Korstjens and Moser ’s practical guide, we also implemented several trustworthiness strategies, including triangulation and member checks with five randomly selected participants.
For triangulation, three researchers interpreted the data independently, after which we compared interpretations. Where there were differences, we came together as a team to discuss them until the most suitable interpretation was found that represented the data (Korstjens and Moser). The researchers held weekly meetings during the analytical process and we held monthly analysis sessions with the research team. For member checking, halfway through (n = 3) and at the end of the study period (n = 2), we went back to participants for feedback and asked them to review the interpretation of the data and challenge what they perceived as correct or in accurate interpretation of the data. This iterative process continued until the codebook was complete and our team agreed that the trustworthiness of the data was excellent. We developed an understanding of the broad themes that constitute function in young adults by (1) analyzing the data independently before comparing respective codes and co-constructing the overarching themes through an iterative process of ongoing discussion (2) reflecting on our positionality as researchers, and (3) incorporating varying participant descriptions of function with written reflections from the focus group interviewers to build successively more sophisticated reconstructions. This process was modelled on the work of Klassen and colleagues .
Item development (Phase 2)
Our team (including one youth researcher, a young adult with lived experience of mental health challenges and who has accessed integrated youth services) developed items based on nouns and verbs collected from the phase 1 transcripts and conceptual model. As a team, we endeavored to develop at least one item for each theme identified to ensure the item bank covered the full breadth and range of function in young adults, from low to high. As outlined by Thrush , we used plain English to increase transferability of the item bank’s language to different social and geographical contexts and increase accessibility to non-native English speakers. However, we also judiciously retained the word choices used by participants in our prior research when appropriate, to better capture participants' original sentiments. At this point, we also designed the instructions for completing the tool and proposed category response scales.
Cognitive debriefing (Phase 3)
We held two focus groups at a conference room in a central location accessible to young people. Participants were recruited purposefully from a local integrated youth service. The youth peer researcher (MK) was available to any young person to ask questions about the study. The youth peer researcher also supported screening of young people to ensure they met the criteria for participating in this phase (age 19–25, accessing integrated youth services, and have current/past experience with mental health challenges) and that our sample for this phase was diverse and representative of young adults accessing this health centre, including youth who were precariously housed, racialized, and/or gender-diverse.
At the start of each focus group, participants individually completed a workbook containing all candidate items from Phase 2 (n = 84). Without conferring, we asked participants to circle yes/no in response to two questions: From your perspective, (1) “is it [the item] clear?” and (2) “does it [the item] capture function and (sub-question) does it capture function comprehensively? This generated quantitative data for researchers to calculate concordance rates between participants from both focus groups. Qualitative feedback and input about item comprehensive was also collected and reported back to each group. Based on feedback from the first focus group participants, a third category was added to the workbook to assess whether each item was youth friendly or meaningful. To preserve original phrasing and the wishes of our participants, we added a third question: (3) “is it [the item] lame?”. Additionally, space was provided below each item for participants to provide more detailed feedback or alternate wording suggestions or apparent missing items to capture comprehensiveness. Of note, the use of the term “lame” was discussed at length throughout this. Our team returned to participants several times to clarify if changing the term to “boring” was sufficient to capture the intent of participants, not so to promote ableist language. Participants accepted the change, but emphasized that “items cannot be lame/boring or youth won’t buy into the process of filling in measures”. See Fig. 2 for an example of how this was worded in the participant workbook.
Participants subsequently expressed their interpretations of each item in a focus group discussion. Our team used verbal prompts to explore varying interpretations of each item and promote equitable participation. Prompts included, “what does this item mean to you?” and, “can you elaborate on any confusion you had in understanding this item?” based on recommendations from Patrick et al. . At the end of the focus groups, we asked participants whether the items any content salient in capturing function from the perspectives of young adults. Two sources of data included the workbooks and the focus group data.
Workbook data and focus group data analysis
Our team recorded and transcribed the focus groups verbatim. Subsequently, two researchers coded the focus groups separately to ensure the credibility of the findings, which is pertinent in the development of patient-reported outcome measures . As outlined by Korstjens and Moser ’s practical guide, our team implemented several trustworthiness strategies, including method triangulation and investigator triangulation. We triangulated the concordance rates from the workbook with the comments from the focus groups. The participants completed the workbook prior to the focus group, enabling them to share their initial impressions of the item in private and without external influence. In the subsequent focus group, participants explored their ideas in more depth and elaborated on responses that needed clarification . Second, our team triangulated findings by separately coding focus group transcriptions, analyzing the findings, deciding which items to keep, modify, or eliminate, prior to corroborating their findings .
We pre-determined that any item to which less than 70% of participants answered “yes” to the workbook questions “is it clear?” and “does it capture function?” would be eliminated. We also eliminated any item where more than 70% of participants answered yes to the question “is it lame?”. We used qualitative data from the focus groups to eliminate or refine items that fell above/below the cut-off rates. We retained items that could not be agreed upon for member phases of measurement development.
Conceptualization of function (Phase 1)
Nineteen participants made up two focus groups. Participants ranged from 19 to 25 years old, with a median age of 24 years old. Seven (37%) participants identified as female, twelve (63%) identified as male and one participant (5%) identified as non-binary. The majority of participants (63%) identified as White. Six (32%) identified as First Nations/Metis/Inuit, three each identified as South Asian, Black/African and Hispanic/Latino (16%), and two identified as Middle Eastern/North African (11%). Two participants (11%) were attending school only, three (16%) were employed, one (5%) was attending school and had employment, while twelve (63%) participants were unemployed and not in school. Seven (37%) participants were looking for employment, and six (32%) were not. Six (32%) participants had a high school diploma and seven (37%) had some high school. Three (16%) participants had some college or technical school education, two (11%) had some university education, and one (5%) had a bachelor’s degree. Ten (53%) reported living in a single room occupancy (SRO) hotel, group home, or Covenant House. SROs are a low-cost housing option in Vancouver typically made up of single rooms ranging from 8-12m2 in size, including a sink, hot plate, and shared washroom facilities [4, 5]. Four (21%) participants were living with someone else and six (32%) were living in an apartment. Participants’ self-reported mental health diagnoses included mood disorders (79%), anxiety disorders (74%), post-traumatic stress disorder (PTSD) (32%), other disorders (32%) and psychotic disorders (16%). Most participants (69%) reported using alcohol, 53% reported using cannabis, and 31% reported using substances in the last two weeks.
Participants provided diverse and wide-ranging definitions of function, which fit into three broad themes of (1) basic needs (2) roles and participation and (3) social connection described in Fig. 3. Basic needs include the subcategories: diet, sex, self-regulation, substance use, personal hygiene, sleep and exercise. Roles and participation include the subcategories of school and work, exercise, goals, engagement and enjoyment, service use and managing daily life. Finally, social connection includes the subcategories of healthy relationships, communication, social norms and support networks.
Spectrum of function
Participants described a spectrum of function, from low to high. Additionally, participants depicted various elements, such as self-regulation, participation and social relationships as interacting, leading to downward spirals. One participant described, “if lots of things are not going the right way in your life, you might be stressed which would lead to more low function.” Conversely, participants described how elements can snowball and build functional gains, describing “let’s say you finish part of your studies, or you find something that was important to you… I feel it gives you the confidence and inspires the drive to pursue more or go further down the path.” Numerous other elements were described as interrelating; components of managing daily life, such as organizing time, were depicted as highly related to basic needs such as diet or sleep, with participants describing the importance of “eating habits and sleep routines.” Participants framed goals relative to “societal productivity standards,” and “milestones” therefore sharing a close relationship with social norms and “fitting in.” Additionally, one participant noted how healthy relationships shape self regulation, stating that when “surrounded by people who have confidence in you, you have confidence in yourself.” Given this, our conceptual model depicts these themes as dynamically interacting (Fig. 3).
Participants highlighted the importance of a sleep schedule, and expressed that low function involved “not sleeping for 3 days straight.” In addition to getting enough sleep, participants noted that during periods of higher function they were “able to just get up.”
Participants spoke of how function well involves actively controlling thoughts, emotions, attention and focus. Participants shared that function entails “establishing and maintaining emotional stability,” “being aware of your thought patterns” and having “control of your thoughts.” Participants highlighted that controlling your thoughts might entail “different coping skills for when the situation occurs.” Other self-regulation strategies include “building exposure to your triggers”, “mindfulness”, being “aware of self-harm” and “processing things, in words of, like, how an operation or situation goes.”
Safe substance use
One participant said low function usually entails “trying to sedate myself with drugs.” Others suggested higher function entails “controlling your substance use.” Substance use was not restricted to elicit substances, with participants noting it “could be from cigarettes or hard drugs.”
Participants described that function well entails one to “brush your teeth and maintain proper hygiene”, while the lower end of function involved “not partaking in personal hygiene habits.”
Participants noted that exercise is a central component of health that is integral to function. One participant described that during high function periods they “move daily.”
Participants highlighted that function well entails “eating right”; however, on the lower end of the spectrum, “you don’t keep good eating habits.”
Sex and intimacy
Participants also expressed that having safe sex is integral to function. As one participant noted, “This is important S@$* that not one talks to us about, but man it is all I care about, all that is important to me at this phase in my life, everyone dances around it- I would give up showers, eating, and work….this is what I care about as a 20-year old man”.
Roles and participation
Participants noted that higher function entails “knowing where you want to go and trying to achieve that.” Additionally, participants spoke of the importance of “hope” or “vision” for high function. Beyond setting goals, participants expressed that higher function entails “following through with plans or goals,” and that when functioning well, “plans that are in your head come into fruition and actions are executed fully.” Participants noted that this was a “large range” and “can be as simple as getting out of bed or working for a year.” Goals were framed in relation to life transitions and societal roles, with a participant expressing that low function entails not reaching “what’s expected when you grow up.” Participants provided examples such as “dogs, kids, house,” and “moving out.”
Participants mentioned that high function means to “go to school” or “go to work.” Additionally, participants depicted school and work as key goals, with one participant describing a goal of “being a social worker.” Many participants reporting being out of work or school, but highlighting this was critical to high levels of function. However participants also noted many barriers to returning to school/work, including disclosing a mental health challenge, low pay, complications with disability laws, and extended time away from the workforce or education system.
Daily life management
Participants expressed that function necessitates “life skills.” Examples given were “to keep reminders”, “cooking” and “cleaning”. Additionally, managing finances and medication was also mentioned. Being able to organize time was also brought up by participants, as was “balancing between needs and wants.” Managing daily life did not always entail maintaining the status quo, with participants noting that higher function may encompass “leaving your comfort zone,” and “breaking, maybe, a habit that would’ve caused you issues to begin with.”
Low function was depicted as “not making it to appointments.” Services highlighted as important to function include “psychiatrists,” and “case managers, peer support workers, doctors,” and “outreach workers.” Participants also spotlighted that function entails being able to use transit to get to appointments. Participants stressed that often appointments were not coordinated and in many locations across the city. One participant noted “my entire functioning in my day could often be just me getting to where I am supposed to go to see my worker, my doctor, my pharmacist for methadone, and my friends”.
Engagement and enjoyment
Participants highlighted that when on the higher end of the spectrum of function, they participated in activities for their own engagement and enjoyment including “being mentally active” and “doing things you like to do.” Examples included hobbies, sports or “learning, not even specifically from schooling but just from like other people.”
Participants described that to have high function, you “gotta find the right people.” Low function was depicted as “inflicting unfairness towards yourself that impacts others around you,” while high function entails “feeling connected to others.” Participants described that functional gains involved being “away from toxic people” or “people who want to hurt you,” highlighting the importance of healthy relationships.
Participants indicated that when they are functioning well, they “behave in a socially acceptable manner,” but during periods of low function, “social cues and behaviours” are a struggle. Participants also described the importance of “fitting in” and reaching “societal productivity standards”.
Several participants noted that function means to “communicate clearly.” Participants also noted that higher function necessitates “taking action and engaging with others”, as well as “the ability to resolve conflict”.
Participants described “reaching out” as key to functional gains. Lower states of function involve being “isolated” or feeling “no one they share their feelings with is there to actually help.” Participants also described “not being heard” by healthcare workers during periods of low function but having “people who are there to sincerely help you,” when function highly.
Item bank development (Phase 2)
Based on Phase 1, our team co-developed 97 items with our youth of researchers including the youth peer research and four additional youth researchers (n = 4). Based on the conceptual model, the item set comprised of items from basic needs (n = 50), roles and responsibilities (n = 30), and social connections (n = 17). Based on a preliminary internal review, our team revised 15 items for clarity, and eliminated 13 for repetition, which resulted 84 (basic needs n = 37, roles and responsibly n = 30, social connections (n = 17). We made a choice to include several items for each category to provide a variety of options to participants in phase 3 to consider and choose.
Cognitive debriefing (Phase 3)
Participants were aged 20–24 and identified as men (n = 8), women (n = 3), female and non-binary (n = 1), with a variety of ethnic backgrounds (see Table 1). Nine of the participants had high school diplomas and, of these, three had attended university or another form of education. Participants had a range of mental health diagnoses and seven participants were homeless or precariously housed at the time of the study. The breakdown of other participant characteristics are in Table 1.
Following the cognitive debriefing focus groups, we retained 31 items, eliminated 38 (for being not clear, not relevant to function, or lame), revised 15 (for clarity and lameness) and added 4 new ones. Table 2 describes the qualitative criteria considered for item removal or modification. After this phase, 50 items were deemed ready for next phases in the PROM development process: experts by experience review (including young adults accessing services and health providers) and psychometric testing (see Additional file 2: Table S2 for item list). All items were mapped back to the conceptual model from phase 1 (see Fig. 3) including basic needs (n = 26), roles and responsibilities (n = 16), and social connections (n = 8). Participants also noted in this phase that instructions were clear and there was value in having a response scale that had both a numerical options (e.g., 0, 1, 2, 3, 4) and associated descriptors with each number (e.g., none of the time, some of the time, half of the time, most of the time, all of the time). They also noted the value of “time” as descriptors, as it is something most young people have in common and can understand.
Function is integral to well-being and mental health, and as such, a PROM measuring this concept has clinical utility in monitoring progress, guiding treatment and communication, and ultimately, helping bridge a road to health and wellness for young adults [41, 42]. However, as outlined in theories of emerging adulthood, what it means to function during this period of life may be distinct from that of adults. As well, what it means to measure for population epidemiological purposes, may be distinct from measuring the effectiveness of treatment over time for individuals [4, 5, 32]. As such, a PROM developed specifically for young adults in this unique stage of life is warranted [43, 44]. In this study an item bank of 50 items was co-developed and refined in collaboration with young adults.
Integrated youth health is a nascent discipline, with significant potential for growth in the next decade [3, 7]. Involving young adults in measurement research leads to improved quality and range of data that is more relevant to this population’s needs [32, 45, 46]. As such, we sought to develop this PROM with, and not just for, young adults. We achieved this by consulting young adult participants during the first two stages of item development and working alongside a peer researcher. We also ensured that all young people, including those who have barriers to accessing mental health services, could participate if they wished. Beyond representing diverse young adults' perspectives, as highlighted by Couch and Frances , including marginalized populations in research enables young people to have a voice, self-advocate, and be empowered. Young people in our study emphasized the importance of the concept being clear, meaningful, and not “not lame!”). Being clear and meaningful is in line with studies that highlight that the function construct should be captured from the perspective of the person: Eklund and Argentzell  and Edgelow and Krupa  both highlight the importance of function capturing the amount of time spent in varying activities, both those they need to do and want to do. Meanwhile, Wada et al.  showed that function was tied to the degree to which time allocated to the activity aligned with personal goals and values. Our study also made our team reflect on the importance of items being relevant, engaging, and current (i.e., not lame!). Participants in our cognitive interviews reflected often on how many times they “have to fill out boring items that do not mean anything to me” or have a feeling that “data gets collected for the government and I never see it!” Participants stressed that it was not only about having a good measure of function, but a process that can move the function data into practice.
Collectively, our study showed that as the demand increases for health services to be youth- and family-centred, there is opportunity for Canada to build a measurement-based care model that is driven by youth-centred data collection, governance, and mobilization efforts. Ultimately, the benefits of such measurement practices could be improved outcomes and experiences for young adults. However, this can only be tested once there is consensus that person-centred outcome measurement be incorporated into performance measures and payment reforms of all integrated youth health services. Without such an approach, youth, families, and health providers may not recognize the improvements) or potential lack of improvements) of young people in Canada and beyond, and young adults may endure ineffective treatment. As Forney and colleagues  explain, this is “particularly problematic for patients from low income and minority groups who face persistent health disparities. The time in long overdue for the field of mental health to embrace measurement based care and live up to medical testing and treat to target principles applied by other medical specialties” (p.8).
Developing a measure of function for young adults is timely. Compounded by the drug toxicity crisis and COVID-19 in Canada, emergency department visits, hospitalization and suicide in youth have been on a sharp rise since 2007 [50, 51]. The suicide rate among young adults increased from 10.9 per 100,000 population to 11.8 per 100,000 in 2016 . Additionally, according to the latest data from the Canadian Institute for Health Information, emergency department mental health visits for patients aged 12–24 have increased by 85% since 2007. Across Canada, service providers, organizations and decision-makers have struggled to understand what interventions are required for young people (including marginalized and racialized youth) to thrive and succeed . This new item bank leverages existing work in youth engagement and measurement to understand health systems transformation for young Canadians [4, 5, 14, 52]. In partnership with young people, our research highlights the importance of co-creating outcomes and measures that align with their needs. We anticipate PROMs, that are fit for purpose for their context of use, will inform how health services for young people should be developed, implemented, evaluated, and scaled.
One limitation of this study is that we had a relatively small sample size and only conducted focus groups in one geographical area with young adults. This could limit the transferability of our results since as evident in research by Paul  and Bolton and Tang , what it means to function is bound not just by age, but also geographical, historical, social and cultural context. We also recognize that it is unlikely that saturation may have occurred in two focus groups alone. This is particularly relevant to items relating to technology, which changes rapidly. Likewise, Romaine  suggests that language is shaped by social and cultural factors, beyond age. Variable language interpretation was evident in our study, since some items elicited connotations unintended by our team, including our peer researcher, and needed to be modified. Despite the drawbacks of our small sample size, one strength of this study is that our sample had varied ethnic and educational backgrounds, thereby increasing the variability in perspectives imbued. Steel  highlights that major barriers to recruiting minorities include utilizing mainstream media for recruitment rather than local connections and failing to convey the value of the research. Our study attracted a diverse group of participants owing to the communication and networking skills of our peer researcher, who carried out recruitment. Nonetheless, further studies would be warranted to verify the conceptual and linguistic applicability in different countries, age groups (notably youth age 12–18), and cultures.
Another potential limitation of this study is that, for practical purposes, we used focus groups rather than individual interviews for cognitive debriefing. Focus groups have been found to generate fewer ideas that are more homogenous than an equivalent number of individual interviews [56, 57]. However, by running two different focus groups, we were able to compare data between them to increase the variability in the information collected. Furthermore, by administering the workbook prior to discussion commencing, we were able to compare private written feedback with opinions shared in a group context. An advantage of focus groups is that they elicit more critical comments, particularly from disempowered groups, thereby maximizing the potential that issues regarding the clarity and conceptual content of items is explored [58, 59]. Nonetheless, it is possible that we may have uncovered additional issues with the item bank had we conducted individual interviews.
A question that arose during the focus groups is whether we are conceptualizing function from a normative perspective, or whether we are addressing how well someone functions given the resources available to them. Participants expressed concern in both focus groups that young peoples’ level of function is dictated not only by capability, but also by access to the services and resources they need to thrive. For example, participants expressed being capable of cooking or showering, but not having access to the amenities to do so. Based on this feedback, where possible, we modified the items to be more inclusive to those with limited access to resources. This has important implications for the clinical utility of this measure; it is a tool to monitor progress, set goals and facilitate conversation regarding function. However, to be of utility in predicting the level of care someone may require, or their capacity to function in the community, we recommend that the data be integrated with subjective report, contextual factors and observational data to optimize how the PROM is interpreted and used.
This paper reports the early development of a PROM measuring the full breadth of function in young adults. After completing the fourth stage, 50 items are ready for expert review and psychometric testing. The finished PROM will provide a measure for young adults to self-monitor functional gains, while providing clinicians with a tool to assess, monitor and plan interventions in collaboration with young adults facing functional challenges. This PROM fills a gap for a clinically important tool in integrated youth services. Furthermore, since this tool is being developed in consultation with diverse young adults, we are providing a platform for self-advocacy to a population frequently excluded from research. As such, this study makes a significant contribution to literature regarding function in young adults. This PROM developed both for and alongside young adults, will provide a tool to address functional challenges in this population, to help them on their personal journey towards health and wellness.
Availability of data and materials
The datasets generated and/or analysed during the current study are not publicly available due to privacy but are available from the corresponding author on reasonable request.
International classification of function
World Health Organization
Global assessment of functioning
World Health Organization disability assessment schedule
Personal and social performance scale
Social and occupational functioning assessment scale
Life functioning assessment scale
Patient report outcome measures
Canadian Institute for Health Information
Pearson C, Janz T, Ali J (2013) Health at a glance: mental and substance use disorders in Canada. Statistics Canada Catalogue, (pp. 82–624)
Smetanin P, Briante C, Stiff D, Ahmad S, Khan M (2015) The life and economic impact of major mental illnesses in Canada. Mental Health Commission of Canada
Mathias S, Tee K, Helfrich W, Gerty K, Chan G, Barbic SP (2021) Foundry: Early learnings from the implementation of an integrated youth service network. Early Interv Psychiatry. https://doi.org/10.1111/eip.13181
Barbic SP, Cano SJ, Mathias S (2018) The problem of patient-centred outcome measurement in psychiatry: Why metrology hasn’t mattered and why it should. J Phys: Conf Ser 1044:12069. https://doi.org/10.1088/1742-6596/1044/1/012069
Barbic SP, Jones AA, Woodward M, Piercy M, Mathias S, Vila-Rodriguez F, Leonova O, Smith GN, Buchanan T, Vertinsky AT, Gillingham S, Panenka WJ, Rauscher A, Barr AM, Procyshyn RM, MacEwan GM, Lang DJ, Thornton AE, Heran MK, Leon AM, Krausz M, Honer WG (2018) Clinical and functional characteristics of young adults living in single room occupancy housing: Preliminary findings from a 10-year longitudinal study. Canadian J Public Health 109(2):204–214. https://doi.org/10.17269/s41997-018-0087-9
Zenone MA, Cianfrone M, Sharma R, Majid S, Rakhra J, Cruz K, Costales S, Sekhon M, Mathias S, Tugwell A, Barbic S (2021) Supporting youth 12-24 during the COVID-19 pandemic: how Foundry is mobilizing to provide information, resources and hope across the province of British Columbia. Glob Health Promot 28(1):51–59. https://doi.org/10.1177/1757975920984196
Hetrick SE, Bailey AP, Smith KE, Malla A, Mathias S, Singh SP, O’Reilly A, Verma SK, Benoit L, Fleming TM, Moro MR, Rickwood DJ, Duffy J, Eriksen T, Illback R, Fisher CA, McGorry PD (2017) Integrated (one-stop shop) youth health care: best available evidence and future directions. Med J Aust 207(S10):S5–S18. https://doi.org/10.5694/mja17.00694
Waclawik K, Jones AA, Barbic SP, Gicas KM, O’Connor TA, Smith GN, Leonova O (2019) Cognitive impairment in marginally housed youth: Prevalence and risk factors. Front Public Health 7(270):1–7. https://doi.org/10.3389/fpubh.2019.00270
Davis M, Vander Stoep A (1997) The transition to adulthood for youth who have serious emotional disturbance: developmental transition and young adult outcomes. J Mental Health Adm 24(4):400–427. https://doi.org/10.1007/BF02790503
Farris MS, MacQueen G, Goldstein BI, Wang J, Kennedy SH, Bray S, Lebel C, Addington J (2018) Treatment history of youth at-risk for serious mental illness. Can J Psychiat 64(2):145–154. https://doi.org/10.1177/0706743718792195
Mei C, Fitzsimons J, Allen N, Alvarez-Jimenez M, Amminger GP, Browne V, Cannon M, Davis M, Dooley B, Hickie IB, Iyer S, Killackey E, Malla A, Manion I, Mathias S, Pennell K, Purcell R, Rickwood D, Singh SP, Wood SJ, Yung A, McGorry PD (2020) Global research priorities for youth mental health. Early Interv Psychiatry 14(1):3–13. https://doi.org/10.1111/eip.12878
Wright M, Getta AD, Green AO, Kickett EC, Kickett AH, McNamara AI, Newman AM, Pell AC, Penny AM, Wilkes UP, Wilkes AS, Culbong T, Taylor K, Borwn A, Dudgeon P, Pearson G, Allsop S, Lin A, Smith G, Farrant B, Mirabelle L, O’Connell M (2021) Co-designing health service evaluation tools that foreground First Nation worldviews for better mental health and wellbeing outcomes. Int J Environ Res Public Health 18(16):8555. https://doi.org/10.3390/ijerph18168555
Cano SJ, Pendrill LR, Barbic SP, Fisher WP (2018) Patient-centred outcome metrology for healthcare decision-making. In CostaFelix RPB (Ed.), 2017 Joint Imeko Tc1-Tc7-Tc13 symposium: measurement science challenges in natural and social sciences (Vol. 1044)
Barbic SP, Leon A, Manion I, Irving S, Zivanovic R, Jenkins E, Ben-David S, Azar P, Salmon A, Helps C, Gillingham S, Beaulieu T, Pattison R, Talon C, Oyedele O, Tee K, Mathias S (2019) Understanding the mental health and recovery needs of Canadian youth with mental health disorders: a strategy for patient-oriented research (SPOR) collaboration protocol. Int J Ment Heal Syst 13(1):6–8. https://doi.org/10.1186/s13033-019-0264-0
Brimblecombe N, Knapp M, Murguia S, Mbeah-Bankas H, Crane S, Harris A, Evans-Lacko S, Ardino V, Iemmi V, King D (2017) The role of youth mental health services in the treatment of young people with serious mental illness: 2-year outcomes and economic implications: Youth mental health services. Early Interv Psychiatry 11(5):393–400. https://doi.org/10.1111/eip.12261
Browne J, Penn DL, Meyer-Kalos PS, Mueser KT, Estroff SE, Brunette MF, Correll CU, Robinson J, Rosenheck RA, Schooler N, Robinson DJ, Addington J, Marcy P, Kane JM (2017) Psychological well-being and mental health recovery in the NIMH RAISE early treatment program. Schizophrenia Res 185:167–172. https://doi.org/10.1016/j.schres.2016.11.032
Forchuk C, Dickins K, Corring DJ (2016) Social determinants of health: Housing and income. Healthcare Quart 18:27–31. https://doi.org/10.12927/hcq.2016.24479
Gmitroski T, Bradley C, Heinemann L, Liu G, Blanchard P, Beck C, Mathias S, Leon A, Barbic SP (2018) Barriers and facilitators to employment for young adults with mental illness: a scoping review. BMJ Open 8(12):e024487. https://doi.org/10.1136/bmjopen-2018-024487
Lambert M, Schöttle D, Sengutta M, Ruppelt F, Rohenkohl A, Luedecke D, Nawara LA, Galling B, Falk A-L, Wittmann L, Niehaus V, Sarikaya G, Handwerk U, Rothländer W, Rietschel L, Gagern C, Lange B, Meigel-Schleiff C, Naber D, Schulte-Markwort M, Krüger H, Unger H-P, Sippel S, Ott S, Romer G, Daubmann A, Wegscheider K, Correll CU, Schimmelmann BG, Bock T, Gallinat J, Karow A (2016) Early detection and integrated care for adolescents and young adults with severe psychotic disorders: rationales and design of the integrated care in early psychosis study (ACCESS III). Early Interven Psychiatry 12(1):96–106. https://doi.org/10.1111/eip.12361
Nelson G, Lord J, Ochocka J (2001) Empowerment and mental health in community: narratives of psychiatric consumer/survivors. J Commun Appl Soc Psychol 11(2):125–142. https://doi.org/10.1002/casp.619
Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, Ring L (2011) Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 2—assessing respondent understanding. Value Health 14(8):978–988. https://doi.org/10.1016/j.jval.2011.06.013
Trombly C (1993) Anticipating the future: assessment of occupational function. Am J Occup Therapy 47(3):253–257. https://doi.org/10.5014/ajot.47.3.25
World Health Organization (2018) Beginner’s guide to the ICF: Towards a common language of function and disability. https://www.who.int/classifications/icf/icfbeginnersguide.pdf
Chiu E, Hung T, Huang C, Lee S, Hsieh C (2018) Responsiveness of the personal and social performance scale in patients with schizophrenia. Psychiatry Res 260:338–342. https://doi.org/10.1016/j.psychres.2017.11.053
Gspandl S, Peirson RP, Nahhas RW, Skale TG, Lehrer DS (2018) Comparing global assessment of functioning (GAF) and World Health Organization disability assessment schedule (WHODAS) 2.0 in schizophrenia. Psychiatry Res 259:251–253. https://doi.org/10.1016/j.psychres.2017.10.033
Hui CL, Li Y, Leung K, Tang JY, Wong GH, Chang W, Chan SK, Lee EH, Chen EY (2013) Reliability and validity of the life functioning assessment inventory (L-FAI) for patients with psychosis. Soc Psychiatry Psychiatr Epidemiol 48(10):1687–1695. https://doi.org/10.1007/s00127-013-0679-x
Romera I, Perez V, Menchón JM, Delgado-Cohen H, Polavieja P, Gilaberte I (2010) Social and occupational functioning impairment in patients in partial versus complete remission of a major depressive disorder episode. A six-month prospective epidemiological study. Eur Psychiatry 25(1):58–65. https://doi.org/10.1016/j.eurpsy.2009.02.007
World Health Organization (2020) WHODAS disability schedule 2.0. https://www.who.int/classifications/icf/more_whodas/en/
Chan EHC, Kong SDX, Park SH, Song YJC, Demetriou EA, Pepper KL, Glozier N, Hickie IB, Guastella AJ (2019) Validation of the social functioning scale: Comparison and evaluation in early psychosis, autism spectrum disorder and social anxiety disorder. Psychiatry Res 276:45–55. https://doi.org/10.1016/j.psychres.2019.03.037
Oude Voshaar MAH, Ten Klooster PM, Vonkeman HE, van de Laar MAFJ (2017) Measuring everyday functional competence using the rasch assessment of everyday activity limitations (REAL) item bank. Qual Life Res 26(11):2949–2959. https://doi.org/10.1007/s11136-017-1627-0
Pan A, Wu C, Chung L, Chen T (2018) Reliability and validity of the self-reported activities of daily living scale for people with mental illness. Hong Kong J Occup Ther 31(2):115–124. https://doi.org/10.1177/1569186118819891
Fortney J, Unutzer J, Wrenn G, Pyne JM, Smith GR, Schoenbaum M, Harbin HT (2017) A tipping point for measurement-based care. Psychiatr Serv 68(2):179–188. https://doi.org/10.1176/appi.ps.201500439
Steel R (2013) Actively involving marginalized and vulnerable people in research. Involving service users in health and social care research. Routledge, London, pp 27–38. https://doi.org/10.4324/9780203865651-9
Lavallee DC, Chenok KE, Love RM, Petersen C, Holve E, Segal CD, Franklin PD (2016) Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff 35(4):575–582. https://doi.org/10.1377/hlthaff.2015.1362
Crepeau EB, Cohn ES, Schell BAB (2003) Willard and Spackman’s occupational therapy, 10th edn. Lippincott, Philadelphia. https://doi.org/10.1111/j.1440-1630.2004.00466.x
Kwan B, Rickwood DJ (2015) A systematic review of mental health outcome measures for young people aged 12 to 25 years. BMC Psychiatry 15:279
Korstjens I, Moser A (2018) Series: practical guidance to qualitative research. Part 4: trustworthiness and publishing. Eur J Gener Pract 24(1):120–124. https://doi.org/10.1080/13814788.2017.1375092
Klassen AF, Longmire NM, Bulstrode NW, Fisher DM, Kasrai L, O’Hara J, Panchapakesan V, Pusic AL, Stewart K, Tsangaris E, Ziolkowski N, Wong Riff KWY (2018) Development of a new patient-reported outcome measure for ear conditions: the EAR-Q. Plast Reconstr Surg Global Open 6(8):e1842. https://doi.org/10.1097/GOX.0000000000001842=
Thrush EA (2001) Plain english? A study of plain English vocabulary and international audiences. Tech Commun 48(3):289–296
Ward VM, Bertrand JT, Brown LF (1991) The comparability of focus group and survey results. Eval Rev 15(2):266–283. https://doi.org/10.1177/0193841X9101500207
Edgelow M, Krupa T (2011) Randomized controlled pilot study of an occupational time-use intervention for people with serious mental illness. Am J Occup Therapy 65(3):267–276. https://doi.org/10.5014/ajot.2011.001313
Eichas K, Montgomery MJ, Meca A, Kurtines WM (2017) Empowering marginalized youth: a self-transformative intervention for promoting positive youth development. Child Dev 88(4):1115–1124. https://doi.org/10.1111/cdev.12866
Arnett JJ (2000) Emerging adulthood: a theory of development from the late teens through the twenties. Am Psychol 55(5):469–480. https://doi.org/10.1037/0003-066X.55.5.469
Arnett JJ, Tanner JL (2016) The emergence of emerging adulthood: The new life stage between adolescence and young adulthood. Routledge handbook of youth and young adulthood. Routledge, New York, pp 50–56. https://doi.org/10.4324/9781315753058-13
Faithfull S, Brophy L, Pennell K, Simmons MB (2019) Barriers and enablers to meaningful youth participation in mental health research: qualitative interviews with youth mental health researchers. J Mental Health 28(1):56–63. https://doi.org/10.1080/09638237.2018.1521926
McLaughlin H (2005) Young service users as co-researchers: Methodological problems and possibilities. Qual Soc Work 4(2):211–228. https://doi.org/10.1177/1473325005052394
Couch J, Francis S (2006) Participation for all? searching for marginalized voices: The case for including refugee young people. Child Youth Environ 16(2):272–290
Eklund M, Argentzell E (2016) Perception of occupational balance by people with mental illness: a new methodology. Scand J Occup Therapy 23(4):304–313. https://doi.org/10.3109/11038128.2016.1143529
Wada M, Backman CL, Forwell SJ (2010) Theoretical perspectives of balance and the influence of gender ideologies. J Occup Sci 17(2):92–103. https://doi.org/10.1080/14427591.2010.9686680
Canadian Institute for Health Information (2018) Child and Youth Mental Health in Canada [Infographic]. Retrieved July 28, 2020 from https://www.cihi.ca/en/child-and-youth-mental-health-in-canada-infographic
Wood E (2018) Strategies for reducing opioid-overdose deaths — lessons from canada. N Engl J Med 378(17):1565–1567. https://doi.org/10.1056/nejmp1800216
Hawke LD, Darnay K, Relihan J, Khaleghi-Moghaddam M, Barbic S, Lachance L, Ben-David S, Brown M, Iyer S, Chaim G, Soklaridis S, Kidd SA, Halsall T, Mathias T, Henderson J (2020) Enhancing researcher capacity to engage youth in research: Researchers’ engagement experiences, barriers and capacity development priorities. Health Expect 23(3):584–592. https://doi.org/10.1111/hex.13032
Paul S (1995) Culture and its influence on occupational therapy evaluation. Can J Occup Ther 62(3):154–161. https://doi.org/10.1177/000841749506200307
Bolton P, Tang AM (2002) An alternative approach to cross-cultural function assessment. Soc Psychiatry Psychiatr Epidemiol 37(11):537–543. https://doi.org/10.1007/s00127-002-0580-5
Romaine S (2000) Language in society: an introduction to sociolinguistics, 2nd edn. Oxford University Press, Oxford, UK
Fern EF (1982) The use of focus groups for idea generation: The effects of group size, acquaintanceship, and moderator on response quantity and quality. J Mark Res 19(1):1–13. https://doi.org/10.1177/002224378201900101
Wight D (1994) Boys’ thoughts and talk about sex in a working-class locality of Glasgow. Sociol Rev 42(4):703–737. https://doi.org/10.1111/j.1467-954X.1994.tb00107.x
Kitzinger J (1995) Focus groups: method or madness? In: Boulton M (ed) Challenge and innovation: methodological advances in social research on HIV/AIDS. Taylor & Francis, Florida, pp 159–175. https://doi.org/10.4324/9781315041094
Watts M, Ebbutt D (1987) More than the sum of the parts: Research methods in group interviewing. Br Edu Res J 13(1):25–34. https://doi.org/10.1080/0141192870130103
We wish to acknowledge the support of Foundry Central Office and Foundry Vancouver Granville for project coordination efforts. We also acknowledge with gratitude that our work takes place on the unceded traditional territories of the xʷməθkʷəy̓əm (Musqueam Indian Band), Sḵwx̱wú7mesh (Squamish Nation), and səlilwətaɬ (Tsleil-Waututh Nation).
This study was funded by a grant from the Canadian Institutes of Health Research.
Ethics approval and consent to participate
Ethical approval was provided by the University of British Columbia’s Office of Research Ethics. Participants were explained the purpose of the study and signed informed consent forms prior to participating.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Barbic, S., Brooks, E., Lassak, N. et al. “It cannot be boring!”: Developing a measure of function for young adults accessing integrated youth services. J Patient Rep Outcomes 6, 92 (2022). https://doi.org/10.1186/s41687-022-00491-6