Development and content validation of a sunlight exposure diary in patients with erythropoietic protoporphyria
Journal of Patient-Reported Outcomes volume 7, Article number: 119 (2023)
Erythropoietic protoporphyria is a rare, inherited disorder presenting in early childhood with severe, painful phototoxicity. EPP has significant impacts on health-related quality of life, though there is variable disease severity. Accurately capturing how much time individuals with EPP can spend outdoors before they develop symptoms is critical to understanding HRQoL and measuring therapeutic response. Therefore, the goal of this study was to develop a comprehensive and content valid sun exposure diary to assess the efficacy of new therapies in individuals with EPP.
Qualitative interviews were conducted with adult and adolescent EPP participants, as well as five clinical experts, to obtain their input on the content of an existing sun exposure diary. Revisions to the diary were made based on evidence generated in cognitive debriefing interviews analyzed in eight consecutive groups of EPP participant.
Interviews were conducted with 17 adults and 6 adolescents with EPP. The average age of adults was 40 years and of adolescents was 14 years. Clinical experts thought the original diary needed clarification on the description of symptoms, how time outdoors was captured, and the distinction between direct vs. indirect sunlight. Participants with EPP also noted these items needed revision, and that the distinction between prodromal symptoms and full reaction symptoms should be clarified. In the final diary version, participants with EPP found most items to be clear and easy to complete/think about. Seventy-six percent of participants (13/17) asked thought the diary was easy to complete. The remainder thought the majority of the diary was easy to complete with the exception of select questions.
Evaluating a new treatment for EPP requires accurately capturing time in sunlight and symptoms in this unique disorder. The newly developed sun exposure diary is content valid and can be used to assess important aspects of symptoms and daily life and therefore evaluate clinically meaningful therapeutic response.
Erythropoietic protoporphyria (EPP) and X‐linked protoporphyria (XLP), collectively referred to as EPP, are rare, inherited photodermatoses that generally present in childhood with severe, painful phototoxicity [1, 2]. Symptoms after sun exposure can include a prodromal syndrome with tingling, burning, and/or itching that may progress rapidly to severe pain, erythema, and swelling. Severity is variable, but most patients experience symptoms within 30 min of sun exposure, and severe phototoxic reactions can last for several days [1,2,3]. There is only one approved treatment for adults with EPP in the US and the European Union, Scenesse (afamelanotide), a subcutaneously administered α-melanocyte stimulating hormone analogue, but no therapies are currently available for pediatric patients .
Daily life is significantly impacted for individuals with EPP [4,5,6,7,8,9,10] not only because of the symptoms of phototoxicity, but also because the disease necessitates avoiding sunlight as much as possible. Accurately tracking sunlight exposure is important for evaluating new therapies and pain-free light exposure time is the precedented, clinically meaningful endpoint used to support afamelanotide’s marketing authorization in the US and European Union. However, the sun exposure diary utilized in those trials had some limitations: iterative patient feedback was not solicited during its development, the Likert pain scale superimposed qualitative categories of mild/moderate/severe, and the diary was deployed on paper.
The goal of this study was to conduct in-depth, qualitative interviews with adults and adolescents with EPP in order to evaluate and modify the content of a sunlight exposure diary where necessary to document evidence of content validity.
Qualitative interviews were conducted from March to June 2022, and the study was approved by the WCG Institutional Review Board.
Potential participants were referred from the United Porphyrias Association (https://www.porphyria.org). Adult participants provided written consent; parents consented for pediatric participants and pediatric participants assented.
Full inclusion criteria are described elsewhere . Briefly, participants with EPP were ≥ 12 years old, had a confirmed diagnosis of EPP, and resided in the US or Canada. If participants were treated with afamelanotide, treatment was initiated no more than three months prior to enrollment. Participants completed a demographic survey and received a $150 Amazon gift card after participation was complete.
Initial interviews were conducted with 5 individuals with EPP and 5 clinical experts that treat EPP, using the same diary that was used in the afamelanotide clinical studies . In those studies, phototoxic pain for each day was recorded by answering the question, “Have you experienced any reactions to light today?” (Yes/No) and indicating the pain level on a scale from 0 to 10 (‘no pain’ to ‘worst imaginable’). Subjects also recorded the amount of time spent outdoors in either direct sunlight or shade between 10:00 to 18:00 hours . Results from those interviews were used for initial modifications to the diary.
One-on-one interviews were conducted using online video conferencing by an experienced health service researcher. Transcripts were coded and analyzed using a qualitative software, MAXQDA. The diary content was revised iteratively based on eight subsequent waves of interviews (for a total of 23 interviews [n = 17 with adults and n = 6 with adolescents]). The first 5 interviews with individuals with EPP included open-ended concept elicitation (CE) questions only. CE questions are used to better understand the symptoms and impacts experienced by individuals with EPP. Examples of CE questions for sun exposure included “How much time do you typically spend outside each day in direct sun?” and “When you do go outside, what type of clothing do you typically wear?”. All subsequent interviews included both CE and cognitive debriefing (CD) of the modified sunlight exposure diary. CD questions are used to obtain specific feedback about the content, clarity, and relevance of the draft sunlight exposure diary. Results of the CE portion are described elsewhere . The results of the CD portion of the interviews focusing on the sun exposure diary are reported here.
EPP participant characteristics
Seventeen adults and six adolescents with EPP participated. There were approximately equal numbers of males (52%) and females (48%), the majority were White (94%) and non-Hispanic (88%). Most adults had a college degree or higher (71%), worked full-time (71%), were married or living with a partner (71%), and the mean age was 40 years. Most adults were not receiving any treatment, 47% previously received afamelanotide, and 13% were currently receiving afamelanotide. On average, the adolescents were 14 years of age and had a high school education or less. Additional participant characteristics are described in Mathias et al .
Cognitive debriefing of sunlight exposure diary
Clinical expert interviews
Feedback from clinical experts on the original diary  included:
4/5 thought adding instructions to the diary would be helpful
Three experts thought individuals with EPP would understand the phrase “reactions to light,” but two did not think the wording in the pain item was clear (whether referencing pain associated with a reaction or pain associated with prodromal symptoms).
Two felt it would be helpful to know which specific prodromal symptoms were experienced.
Two indicated it might be confusing thinking about time outside if this includes exposure to sunlight while in a car or sitting by a window.
One thought increments of 15 min were too long to consider and suggested using start/ stop times instead. Another liked the 15-min intervals and felt it would be too difficult to use start/stop times. One expert would expand the range from 7:00 am until 9:00 pm because of the potential for sunlight exposure during those hours, especially during certain months of the year.
Two did not think the question about time in shade was relevant.
Most experts agreed it was more appropriate to ask about “today” versus “the past 24 h.”
EPP participant interviews
Each participant was interviewed only once, and therefore only reviewed one version of the diary. Most participants were able to accurately paraphrase the items in Version 1 of the Diary (range: 80–100%). However, the items of “any reactions to light today” and “time in direct sun and time in shade” were unclear. Some participants (40%) were confused by the use of military time and some (40%) did not know how to answer if sun exposure was less than 15 min.
Based on results from these initial interviews with clinical experts and participants with EPP, the original diary was revised (Version 2). In general, participants were able to accurately paraphrase all items in the diary (range: 80–100%). Most found the items to be clear (range: 60–100%) and could rephrase the question accurately in their own words. Of those who found it unclear, fifty percent found the question “did you have early warning symptoms or a painful reaction” to be confusing because some individuals experience pain as an early warning symptom. Sixty-seven percent were unsure how to respond to the question about time in shade.
In Versions 3 and 4b, participants were able to correctly paraphrase the instructions and all items. Most questions were easy to think about, with one exception. For the question about total time in sunlight, one participant thought it was unnecessary to include one start/stop time for each hour since she would never leave her home more than one time each day.
In Versions 5–7 minor revisions were made between each version. Most participants were able to accurately paraphrase each item (range: 67–100%). Most (67–100%) thought the questions were easy to complete/think about. With one exception, participants found the questions to be clear. One participant would add “whether you experienced a reaction or not” to the question about “whether you were exposed to sunlight.”
Version 8 of the Diary was debriefed with two adolescents. Both were able to correctly paraphrase the instructions and all items. With some exceptions, participants found the items to be clear. One adolescent did not know what “severity” meant and was not able to describe the word “moderate” but could figure out the meaning within the context of the question and other response options. For the question about exposure to sunlight, one suggested adding “sun through a window or sunlight through a car window” as examples of indirect sunlight.
During the final round of interviews, all participants were able to correctly paraphrase the instructions and all questions. Participants found most items to be clear and easy to complete/think about. Table 1 summarizes the key changes from the original to the final diary, and Table 2 includes sample final questions. A copy of the diary is available upon request from the authors.
Meaningful change to level of pain
Participants were asked how much change in the level of pain would be meaningful if they were in a clinical study for an EPP treatment, on a 10-point scale ranging from 0 (no pain) to 10 (worst pain imaginable). Responses ranged from 2 to 7 points, with most saying ~ 5 points would be meaningful.
Participants who completed Versions 1–4 were asked how easy or difficult it was to think about the past 24 h (from midnight to 11:59 pm on a specific day) when answering the questions. Eighty percent (n = 4 of 5) thought it was an easy timeframe to consider. One participant said it could be difficult. The diary was then revised to include a specific day of the week and the date (versions 5–8). All participants (n = 5) thought it was easy to recall back for a specific date.
Ease/difficulty of completing diary
Seventy-six percent of participants (n = 13 of 17) thought it was easy to complete the diary. Two participants thought it was easy except for the question about time in sunlight by hour. One participant said between easy and medium, and one said easy except for a few words (adolescent who had difficulty with “severity” and “moderate”). Ninety-four percent of participants (n = 15 of 16) thought the formatting was clear.
The goal of this qualitative research was to confirm the content validity, relevance, and clarity of a sunlight exposure diary in individuals with EPP. The diary was cognitively debriefed during a total of 23 combined CE/CD interviews with adults and adolescents.
Individuals with EPP and clinical experts, who are also researchers, were asked to review and complete (n = 5 individuals with EPP only) the original version of the sunlight exposure diary. Based on their feedback, it was determined that revisions were needed. Revisions included providing more detailed instructions, allowing respondents to record time in sunlight in exact minutes (versus 15-min increments), further differentiating early warning symptoms (prodrome) from full reactions, assessing common symptoms experienced by individuals, omitting reference to time in shade, and including reactions from indirect sunlight to capture scenarios like sun coming through a window of a car or home, reflected off of surfaces.
During nine total waves of interviews, and following several additional modifications (see Table 1), cognitive debriefing revealed that individuals with EPP were able to understand and complete the sunlight exposure diary with ease. Problematic questions/phrasing were identified, and minor revisions were made following each wave of interviews. In general, the final sunlight exposure diary was found to be clear, comprehensive, and relevant.
Data from this diary will result in a detailed documentation of sun exposure and associated symptoms, which will be used to analyze total pain-free time in light as well as frequency and severity of symptoms. Though subject to recall bias, encouraging daily completion can minimize this. In addition, data collected from this diary is one element of trial participation and the patient experience, and can be complemented with objective measures such as light dosimetry. There is also inherent individual variability in sun exposure due to geography, seasonality, and clothing that can affect the diary responses. This can be accounted for, in part, by capturing zip codes for participants, and these variables are less confounding if treatment effects on sunlight exposure are evaluated in a randomized controlled trial.
This study included a fairly heterogenous sample in terms of demographic characteristics and was geographically diverse. Limitations included a relatively small sample of adolescents and a lack of racial/ethnic diversity of participants.
This sunlight exposure diary was developed in accordance with FDA guidance and the first to be rigorously debriefed to ensure its content is relevant and readily understood with individuals with EPP. The new version of the diary can be considered content valid and will be valuable for use in clinical trials of new EPP treatments.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available due to privacy concerns but are available from the corresponding author on reasonable request.
Health related quality of life
Time to prodrome
Balwani M, Naik H, Anderson KE, Bissell DM, Bloomer J, Bonkovsky HL, Phillips JD, Overbey JR, Wang B, Singal AK et al (2017) Clinical, biochemical, and genetic characterization of north American patients with erythropoietic protoporphyria and X-linked protoporphyria. JAMA Dermatol 153:789–796. https://doi.org/10.1001/jamadermatol.2017.1557
Lecha M, Puy H, Deybach JC (2009) Erythropoietic protoporphyria. Orphanet J Rare Dis 4:19. https://doi.org/10.1186/1750-1172-4-19
Balwani M (2019) Erythropoietic protoporphyria and X-linked protoporphyria: pathophysiology, genetics, clinical manifestations, and management. Mol Genet Metab 128:298–303. https://doi.org/10.1016/j.ymgme.2019.01.020
Langendonk JG, Balwani M, Anderson KE, Bonkovsky HL, Anstey AV, Bissell DM, Bloomer J, Edwards C, Neumann NJ, Parker C et al (2015) Afamelanotide for erythropoietic protoporphyria. N Engl J Med 373:48–59. https://doi.org/10.1056/NEJMoa1411481
Naik H, Shenbagam S, Go AM, Balwani M (2019) Psychosocial issues in erythropoietic protoporphyria - the perspective of parents, children, and young adults: a qualitative study. Mol Genet Metab 128:314–319. https://doi.org/10.1016/j.ymgme.2019.01.023
Rufener EA (1992) “... and found no shade”. Coping processes of patients with erythropoietic protoporphyria. Psychother Psychosom Med Psychol 42:339–348
Naik H, Overbey JR, Desnick RJ, Anderson KE, Bissell DM, Bloomer J, Bonkovsky HL, Phillips JD, Wang B, Singal A, Balwani M (2019) Evaluating quality of life tools in north American patients with erythropoietic protoporphyria and X-linked protoporphyria. JIMD Rep 50:9–19. https://doi.org/10.1002/jmd2.12052
Holme SA, Anstey AV, Finlay AY, Elder GH, Badminton MN (2006) Erythropoietic protoporphyria in the UK: clinical features and effect on quality of life. Br J Dermatol 155:574–581. https://doi.org/10.1111/j.1365-2133.2006.07472.x
Frank J, Poblete-Gutiérrez P (2011) Delayed diagnosis and diminished quality of life in erythropoietic protoporphyria: results of a cross-sectional study in Sweden. J Intern Med 269:270–274. https://doi.org/10.1111/j.1365-2796.2010.02283.x
Jong CT, Finlay AY, Pearse AD, Kerr AC, Ferguson J, Benton EC, Hawk JL, Sarkany RP, McMullen E, Rhodes LE et al (2008) The quality of life of 790 patients with photodermatoses. Br J Dermatol 159:192–197. https://doi.org/10.1111/j.1365-2133.2008.08581.x
Mathias SD, Burke L, Colwell HH, Mensing G, Savage W, and Naik H (2023) Development and content validation of novel patient-reported outcome measures to assess disease severity and change in patients with erythropoietic protoporphyria: the EPP impact questionnaire (EPIQ). J Patient Rep Outcomes Submitted (unpublished)
We are indebted to the individuals with EPP who participated in study and would like to thank the United Porphyrias Association for assisting with recruitment.
Ethics approval and consent to participate
This study was approved by an IRB and all participants consented.
Consent for publication
All participants consented to publication with consent to participate in this study.
Financial Disclosures (related): GM and WS are employed by and have a financial interest in Disc Medicine. SDM is an employee of Health Outcomes Solutions (HOS) and HHC is a consultant to HOS, which received funding from Disc Medicine for the conduct of this study, and LB is a consultant for Disc Medicine. Financial disclosures (unrelated): In addition to funding from Disc Medicine related to this work, LB has past and ongoing research support and contracts from various non-profit organizations and for-profit companies that are unrelated to this work.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Mathias, S.D., Burke, L., Colwell, H.H. et al. Development and content validation of a sunlight exposure diary in patients with erythropoietic protoporphyria. J Patient Rep Outcomes 7, 119 (2023). https://doi.org/10.1186/s41687-023-00655-y