Skip to main content
Journal of Patient-Reported Outcomes
Download PDF
Download ePub

Translation and validation of the Korean version of PROMIS® pediatric and parent proxy measures for emotional distress

Journal of Patient-Reported Outcomes volume 3, Article number: 36 (2019) Cite this article

Introduction

Public awareness of mental health among adolescents is growing, as about 10–20% of adolescents suffer from mental health problems globally [1, 2]. Patient-reported outcomes (PROs) are especially important in measuring mental health symptoms such as depression, anxiety, and anger, which are the most common forms of emotional distress experienced by adolescents [3,4,5]. In addition, parent-proxy reports are also crucial to complement adolescents’ self-report in assessing such emotional distress [6].

The Patient-Reported Outcome Measurement Information System (PROMIS®) was developed by the National Institute of Health (NIH) to address the need for more valid, reliable, and generalizable measures to assess critical PROs [7, 8]. The PROMIS measures efficiently assess health status across populations based on item response theory (IRT). The PROMIS pediatric self-report and parent-proxy item banks have been developed to assess physical, mental, and social health in youth aged 8 to 17 years [9,10,11,12]. The PROMIS Pediatric and Parent Proxy scales were adapted and validated in different languages, showing good psychometric properties [13, 14].

Given the PROMIS measures’ advantages in assessing PROs within an innovative framework, we judged it important that these benefits were made available to assess emotional distress in Korean adolescents. Thus, this study aimed to develop a Korean version of the PROMIS Pediatric self-report and parent-proxy measures for emotional distress (i.e., depression, anxiety, and anger) for adolescents and to evaluate their psychometric properties and unidimensionality.

Methods

Translation

The translation of the PROMIS Pediatric and Parent Proxy measures for emotional distress was performed after obtaining approval from the institutional review boards at S. University and S. Hospital in Korea. Specifically, we translated the original PROMIS® Pediatric self-report Item Bank Version 1.1 for Depressive Symptoms (13 items), Anxiety (13 items), and Anger (5 items); Parent-Proxy Item-Bank Version 1.1 for Depressive Symptoms (13 items) and Anxiety (13 items); and Parent-Proxy Short-Form Version 1.0 for Anger (5 items) based on standard translation methodology (i.e., Functional Assessment of Chronic Illness Therapy) [15]. The process included forward-translation, reconciliation, back-translation, and expert reviews. The English version of each measure was translated into Korean by two independent bilingual translators. Subsequently, a third bilingual translator reconciled the two previous translations. A fourth independent bilingual translator then performed a back-translation of the reconciled version. Next, three experts (a linguistic expert, a mental health specialist, and an expert experienced in the translation and validation of measures) independently reviewed all the translation steps for each item and selected the most appropriate translation or suggested an alternative one. Finally, the entire translation history for each item was sent to the PROMIS Statistical Center for harmonization and quality assurance to assess their equivalence with the original versions.

Cognitive interviews

Cognitive interviews were performed to ensure cross-cultural equivalence between the original and Korean versions and identify possible sources of response errors. For the pediatric measures, a convenience sample of five adolescents aged 13 to 17 years (2 boys; 3 girls) was interviewed. In addition, five mothers of adolescents were interviewed regarding the proxy measures. During the interviews, participants first responded to all the translated items in writing and then, reviewing one item at a time, were asked about difficulties in understanding the item. The meanings of original and translated items were discussed with participants to ensure their equivalence and comprehensibility. After completing all the interviews, the research team analyzed participant comments to evaluate the comprehensibility and equivalence of the translated items. Several translated items that were reported to sound unnatural or vague, and items that used double negatives were revised. Overall, the participants reported no serious difficulties in understanding the translated items.

Psychometric testing

Participants

The psychometric testing included a non-clinical sample from one middle school and two high schools and a clinical sample from one hospital, all in Seoul. First, an information sheet stating the study purpose and procedures was provided to students aged 13 to 17 years and their parents, after getting permission from the school principals. When both students and parents expressed willingness to participate, the participants were each given individual sealed envelopes containing informed consent forms and the Korean version of the PROMIS questionnaires. A total of 160 adolescent-parent dyads completed and returned the consent forms and questionnaires between September and December 2016.

Subsequently, 114 adolescent-parent dyads recruited from the psychiatric outpatient department between December 2017 and March 2018 also completed the consent forms and questionnaires. Eligibility criteria were as follows: (1) aged 12 to 17, (2) able to speak Korean and understand the study purpose and procedures, and (3) no psychotic symptoms or developmental disorders based on the medical records.

Measures

Responses to the Korean version of the PROMIS measures were based on recollection of the previous seven days. The measures employed a 5-point Likert scale ranging from 0 (never) to 4 (almost always) with higher scores reflecting higher levels of emotional distress. The standardized mean score (T-score, with a mean value of 50 and a standard deviation of 10) for each measure was obtained from the online PROMIS Assessment Center.

Analyses

Data analyses were performed using STATA version 15 (Stata Corp LP, College Station, Texas, USA) and R version 3.5.0. Reliability was examined using Cronbach’s α. Construct validity was measured using correlations between the pediatric and proxy measures for depressive symptoms, anxiety, and anger.

The unidimensionality of the measures was examined through confirmatory factor analysis (CFA), exploratory factor analysis (EFA), omega hierarchical (ωh), and explained common variance (ECV). First, one-factor CFA was used, and the comparative fit index (CFI), the Tucker-Lewis Index (TLI), and the root mean square error of approximation (RMSEA) were evaluated to assess model fit. The generally accepted criteria for unidimensionality include CFI > 0.95, TLI > 0.95 [16, 17], and RMSEA < 0.08 [18]. Then, an EFA was performed. Unidimensionality is considered to be supported if a single factor accounts for more than 20% of the variance and the ratio of the first and second eigenvalues is higher than 4 [19]. As an additional approach for assessing unidimensionality, we employed the estimated ωh, which is the proportion of total variance attributable to a general factor, using the Psych package in R. An ωh value of 0.70 or greater indicates that an item set is sufficiently unidimensional for IRT models [20]. Additionally, explained common variance (ECV) was calculated to assess the relative strength of one general factor compared to group factors, and we considered ECV values greater than 0.60 to be acceptable [20].

Results

Of 274 adolescent-parent dyads who returned consent forms and questionnaires, we excluded 22 dyads with missing data from the analysis. Consequently, data from 252 parent-child dyads were analyzed. In this study, 52.78% of adolescent participants were girls, and 87.30% of parents were women. Mean age was 15.18 years for adolescents and 45.61 for parents (Table 1). Descriptive values of all PROMIS measures are presented in Table 2.

Table 1 Participant Characteristics
Full size table
Table 2 Descriptive statistics for PROMIS measures
Full size table

Cronbach’s alpha for the pediatric depression, anxiety, and anger measures was 0.93, 0.94, and 0.91, respectively; and for the parent proxy measures, 0.95, 0.96, and 0.90, respectively. Strong relationships were observed between pediatric depression, anxiety, and anger measures (0.70 ≤ r ≤ 0.79; p < 0.001) and the respective parent proxy measures (0.59 ≤ r ≤ 0.84; p < 0.001).

The results obtained for unidimensionality are summarized in Table 3. CFA indices outside the recommended ranges were found for pediatric Depression (CFI = 0.92; TLI = 0.91; RMSEA = 0.10), Anxiety (CFI = 0.91; TLI = 0.90; RMSEA = 0.10), and Anger (CFI = 0.96; TLI = 0.91; RMSEA = 0.18). Similarly, indices outside the ranges were observed for parent-proxy Depression (CFI = 0.93; TLI = 0.92; RMSEA = 0.10), anxiety (CFI = 0.89; TLI = 0.86; RMSEA = 0.14), and anger (CFI = 0.97; TLI = 0.95; RMSEA = 0.13). Although the CFA indices were slightly beyond the recommended ranges, the EFA results supported unidimensionality. The first factor of pediatric Depression, Anxiety, and Anger explained 94.52%, 94.23%, and 100% of the variance, respectively. Additionally, the ratios of the first and second eigenvalues for pediatric Depression, Anxiety, and Anger were greater than 4. In the case of the parent-proxy measures, the first factor of proxy-report Depression, Anxiety, and Anger explained 95.38%, 92.88%, and 100% of the variance, respectively, and the ratios of the first and second eigenvalues were greater than 4 as well. In addition, values of estimated omega hierarchical (ωh) for pediatric Depression, Anxiety, and Anger were 0.83, 0.81, and 0.84, respectively, and those for parent-proxy Depression, Anxiety, and Anger were 0.88, 0.87, and 0.87, respectively. All the omega hierarchical (ωh) values for pediatric and parent proxy measures were higher than 0.7 and thus supported unidimensionality. Furthermore, the ECV values for the pediatric and parent-proxy measures were greater than 0.6, thus indicating unidimensionality. Based on these results, the Korean PROMIS Pediatric self-report and parent-proxy measures sufficiently exhibited unidimensionality.

Table 3 Model fit indices for Depression, Anxiety, and Anger
Full size table

Discussion

This study was conducted to translate the PROMIS Pediatric and Parent-Proxy measures for emotional distress into Korean and evaluate their psychometric properties among Korean adolescents. The Korean version of the PROMIS Pediatric self-report and parent-proxy measures for emotional distress for adolescents were developed based on the standard PROMIS methodology and demonstrated satisfactory psychometric properties and unidimensionality, similar to those reported in previous studies [21,22,23,24].

The PROMIS measures for emotional distress have been found to be easy to understand and complete with minimal burden and can be used in both research and clinical settings. In addition, the use of self-report and parent-proxy measures may enable researchers and clinicians to gain a deeper understanding of adolescents’ emotional distress and assess the association between parents’ and adolescents’ perceptions of it.

As for study limitations, our convenience sample consisted of adolescents recruited from schools and a hospital in Seoul, potentially compromising the generalizability of our findings. Thus, additional studies with larger samples, more varied populations, and pre-adolescent participants are called for. Moreover, cross-cultural validation requires the use of differential item functioning for Korean and English measures. Finally, this cross-sectional study could not assess the predictive validity of the emotional distress measures.

In conclusion, the Korean version of the PROMIS Pediatric self-report and parent proxy measures for emotional distress are reliable and valid for Korean adolescents and their parents. Given the increasing prevalence and seriousness of adolescents’ mental health problems, the Korean version of PROMIS measures could advance early detection and intervention for adolescents’ mental health problems.

Abbreviations

CFA:

Confirmatory factor analysis

ECV:

Explained common variance

EFA:

Exploratory factor analysis

IRT:

Item response theory

PROMIS®:

Patient-Reported Outcome Measurement Information System

PROs:

Patient-reported outcomes

RMSEA:

Root mean square error of approximation

ωh :

Omega hierarchical

References

  1. Bor, W., Dean, A. J., Najman, J., & Hayatbakhsh, R. (2014). Are child and adolescent mental health problems increasing in the 21st century? A systematic review. The Australian and New Zealand Journal of Psychiatry, 48, 606–616. https://doi.org/10.1177/0004867414533834.

    Article  PubMed  Google Scholar 

  2. Kieling, C., Baker-Henningham, H., Belfer, M., Conti, G., Ertem, I., Omigbodun, O., Rohde, L. A., Srinath, S., Ulkuer, N., & Rahman, A. (2011). Child and adolescent mental health worldwide: Evidence for action. Lancet, 378, 1515–1525. https://doi.org/10.1016/S0140-6736(11)60827-1.

    Article  PubMed  Google Scholar 

  3. Merikangas, K. R., He, J., Burstein, M., Swanson, S. A., Avenevoli, S., Cui, L., Benjet, C., Georgiades, K., & Swendsen, J. (2010). Lifetime prevalence of mental disorders in US adolescents: Results from the National Comorbidity Survey Replication–Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry, 49, 980–989. https://doi.org/10.1016/j.jaac.2010.05.017.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Carr, A. (2015). The handbook of child and adolescent clinical psychology: A contextual approach. London: Routledge.

    Book  Google Scholar 

  5. Wolpert, M., Ford, T., Trustam, E., Law, D., Deighton, J., Flannery, H., & Fugard, R. J. B. (2012). Patient-reported outcomes in child and adolescent mental health services (CAMHS): Use of idiographic and standardized measures. J Ment Health, 21, 165–173. https://doi.org/10.3109/09638237.2012.664304.

    Article  PubMed  Google Scholar 

  6. Kuhn, C., Aebi, M., Jakobsen, H., Banaschewski, T., Poustka, L., Grimmer, Y., Goodman, R., & Steinhausen, H.-C. (2017). Effective mental health screening in adolescents: Should we collect data from youth, parents or both? Child Psychiatry Hum Dev, 48, 385–392. https://doi.org/10.1007/s10578-016-0665-0.

    Article  PubMed  Google Scholar 

  7. Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., Amtmann, D., Bode, R., Buysse, D., Choi, S., et al. (2010). The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol, 63, 1179–1194. https://doi.org/10.1016/j.jclinepi.2010.04.011.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., Ader, D., Fries, J. F., Bruce, B., & Rose, M. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH roadmap cooperative group during its first two years. Med Care, 45(Suppl 1), S3–S11. https://doi.org/10.1097/01.mlr.0000258615.42478.55.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Irwin, D. E., Stucky, B., Langer, M. M., Thissen, D., DeWitt, E. M., Lai, J.-S., Varni, J. W., Yeatt, K., & DeWalt, D. A. (2010). An item response analysis of the pediatric PROMIS anxiety and depressive symptoms scales. Qual Life Res, 19, 595–607. https://doi.org/10.1007/s11136-010-9619-3.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Irwin, D. E., Gross, H. E., Stucky, B. D., Thissen, D., DeWitt, E. M., Lai, J. S., Amtmann, D., Khastou, L., Varni, J. W., & DeWalt, D. A. (2012). Development of six PROMIS pediatrics proxy-report item banks. Health Qual Life Outcomes, 10, 22. https://doi.org/10.1186/1477-7525-10-22.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Varni, J. W., Thissen, D., Stucky, B. D., Liu, Y., Gorder, H., Irwin, D. E., DeWitt, E. M., Lai, J.-S., Amtmann, D., & DeWalt, D. A. (2012). PROMIS® parent proxy report scales: An item response theory analysis of the parent proxy report item banks. Qual Life Res, 21, 1223–1240. https://doi.org/10.1007/s11136-011-0025-2.

    Article  PubMed  Google Scholar 

  12. Irwin, D. E., Stucky, B. D., Langer, M. M., Thissen, D., DeWitt, E. M., Lai, J.-S., Yeatts, K. B., Varni, J. W., & DeWalt, D. A. (2012). PROMIS pediatric anger scale: An item response theory analysis. Qual Life Res, 21, 697–706. https://doi.org/10.1007/s11136-011-9969-5.

    Article  PubMed  Google Scholar 

  13. Haverman, L., Grootenhuis, M. A., Raat, H., van Rossum, M. A., van Dulmen-den Broeder, E., Hoppenbrouwers, K., Correia, H., Cella, D., Roorda, L. D., & Terwee, C. B. (2016). Dutch–Flemish translation of nine pediatric item banks from the patient-reported outcomes measurement information system (PROMIS)®. Qual Life Res, 25, 761–765. https://doi.org/10.1007/s11136-015-0966-y.

    Article  PubMed  Google Scholar 

  14. Westmoreland, K., Reeve, B. B., Amuquandoh, A., van der Gronde, T., Manthalu, O., Correia, H., et al. (2018). Translation, psychometric validation, and baseline results of the patient-reported outcomes measurement information system (PROMIS) pediatric measures to assess health-related quality of life of patients with pediatric lymphoma in Malawi. Pediatr Blood Cancer, 65(11), e27353. https://doi.org/10.1002/pbc.27353.

    Article  CAS  PubMed  Google Scholar 

  15. Eremenco, S. L., Cella, D., & Arnold, B. J. (2005). A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Evaluation & the Health Professions, 28, 212–232. https://doi.org/10.1177/0163278705275342.

    Article  Google Scholar 

  16. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychol Methods, 3, 424–453. https://doi.org/10.1037/1082-989X.3.4.424.

    Article  Google Scholar 

  17. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling : A Multidisciplinary Journal, 6, 1–55. https://doi.org/10.1080/10705519909540118.

    Article  Google Scholar 

  18. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models. London: Sage.

    Google Scholar 

  19. Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., Thissen, D., Revicki, D. A., Weiss, D. J., Hambleton, R. K., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcomes measurement information system (PROMIS). Med Care, 45(Suppl 1), S22–S31. https://doi.org/10.1097/01.mlr.0000250483.85507.04.

    Article  PubMed  Google Scholar 

  20. Reise, S. P., Scheines, R., Widaman, K. F., & Haviland, M. G. (2013). Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educ Psychol Meas, 73, 5–26. https://doi.org/10.1177/0013164412449831.

    Article  Google Scholar 

  21. Bertisch, H., Rivara, F. P., Kisala, P. A., Wang, J., Yeates, K. O., Durbin, D., Zonfrillo, M. R., Bell, M. J., Temkin, N., & Tulsky, D. S. (2017). Psychometric evaluation of the pediatric and parent-proxy patient-reported outcomes measurement information system and the neurology and traumatic brain injury quality of life measurement item banks in pediatric traumatic brain injury. Qual Life Res, 26, 1887–1899. https://doi.org/10.1007/s11136-017-1524-6.

    Article  PubMed  Google Scholar 

  22. Dampier, C., Barry, V., Gross, H. E., Lui, Y., Thornburg, C. D., DeWalt, D. A., & Reeve, B. B. (2016). Initial evaluation of the pediatric PROMIS® health domains in children and adolescents with sickle cell disease. Pediatr Blood Cancer, 63, 1031–1037. https://doi.org/10.1002/pbc.25944.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Devine, J., Klasen, F., Moon, J., Herdman, M., Hurtado, M., Castillo, G., et al. (2018). Translation and cross-cultural adaptation of eight pediatric PROMIS® item banks into Spanish and German. Qual Life Res, 27, 2415–2430. https://doi.org/10.1007/s11136-018-1874-8.

    Article  CAS  PubMed  Google Scholar 

  24. Varni, J. W., Magnus, B., Stucky, B. D., Liu, Y., Quinn, H., Thissen, D., Gross, H. E., Huang, I.-C., & DeWalt, D. A. (2014). Psychometric properties of the PROMIS® pediatric scales: Precision, stability, and comparison of different scoring and administration options. Qual Life Res, 23, 1233–1243. https://doi.org/10.1007/s11136-013-0544-0.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This research was supported by the Promising-Pioneering Researcher Program of Seoul National University. We would like to thank all the adolescents and their parents who participated in this study, as well as the school nurses and nurses who supported data collection. We are also grateful to Mr. Jon Mann of the University of Illinois at Chicago for his editorial support.

Funding

This study was funded by Seoul National University (Grant Number: 810–20160011).

Availability of data and materials

The datasets generated and analyzed during the current study are not publicly available because individual privacy could be compromised, but are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

  1. College of Nursing and the Research Institute of Nursing Science, Seoul National University, 103 Daehak-ro, Jongno-gu, Seoul, 03080, South Korea

    Heeseung Choi

  2. The Research Institute of Nursing Science, Seoul National University, 103 Daehak-ro, Jongno-gu, Seoul, 03080, South Korea

    Chanhee Kim

  3. Jesus University, 383, Seowon-ro, Wansan-gu, Jeonju, 54989, South Korea

    Heesung Ko

  4. College of Nursing, University of Illinois at Chicago, Chicago, IL, USA. 845 S Damen Ave, Chicago, IL, 60612, USA

    Chang Gi Park

Authors
  1. Heeseung Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chanhee Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heesung Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chang Gi Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HC developed the proposal of the study and HC, CK, and HK collected the study data. CK and HC conducted statistical analysis and wrote the manuscript. CGP provided statistical consultation. HC, CK, HK, and CGP interpreted the study data and reviewed all versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chanhee Kim.

Ethics declarations

Ethics approval and consent to participate

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This study was conducted after obtaining approval from the institutional review boards at Seoul National University and Seoul National Hospital in Korea. Informed consent was obtained from all individual participants included in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, H., Kim, C., Ko, H. et al. Translation and validation of the Korean version of PROMIS® pediatric and parent proxy measures for emotional distress. J Patient Rep Outcomes 3, 36 (2019). https://doi.org/10.1186/s41687-019-0120-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41687-019-0120-7