Skip to main content

Integrating the patient voice with clinician reports to identify a hepatocellular carcinoma-specific subset of treatment-related symptomatic adverse events



Incorporating patient reporting of symptomatic adverse events (AEs) is important in evaluating safety and tolerability in cancer clinical trials. The Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) was developed to assess the frequency, severity, and/or interference of patient-reported symptomatic AEs. The objective of this study was to identify, based on oncologist and patient interviews, a relevant subset of symptomatic AEs from the PRO-CTCAE that can be used to optimize patient reporting of symptomatic AEs in hepatocellular carcinoma (HCC) clinical trials.


Qualitative and quantitative data on HCC diagnosis, treatment, symptoms, and side effects were collected from patients. Using a numerical rating scale, medical oncologists specializing in HCC rated the importance of (1) 34 symptomatic AEs (Grade ≥2) identified in past sponsor HCC clinical trials, (2) each PRO-CTCAE symptomatic AE item to the patient, and (3) each PRO-CTCAE symptomatic AE item with respect to patient safety or tolerability. Patients completed the PRO-CTCAE items and were debriefed on the importance of each PRO-CTCAE symptomatic AE to them.


Five medical oncologists from the United States, Spain, Taiwan, Korea, and Hong Kong with 14 to 30 years of experience, and 17 patients with HCC and Child-Pugh class A or B cirrhosis status completed interviews. Medical oncologists rated the following symptomatic AEs from prior trials as being highly important to patients (mean rating of ≥7 on a scale from 0 to 10): hand-foot syndrome, diarrhea, fatigue, decreased appetite, rash, vomiting, and weight loss. PRO-CTCAE symptomatic AEs rated by medical oncologists as being highly important to patients included diarrhea, vomiting, shivering or shaking chills, hand-foot syndrome, rash, fatigue, difficulty swallowing, decreased appetite, and loss of control of bowel movements. Patients rated the following PRO-CTCAE symptomatic AEs as being highly important: loss of appetite/lack of interest in food, pain/tenderness at injection/insertion site, fatigue/lack of energy/tiredness, nausea, and hair loss.


This study identified a preliminary list of clinically relevant symptomatic AEs from interviews with both medical oncologists and patients that can be used to support assessments of treatment safety and tolerability in HCC clinical trials.


Cancer treatments are often associated with symptomatic adverse events (AEs), and a more robust assessment of these events has important clinical and scientific value [1]. In cancer clinical trials, clinicians and study researchers commonly use the Common Terminology Criteria for Adverse Events (CTCAE) to grade and report AEs; it is composed of approximately 790 AEs graded on a severity scale (0 to 5) [2]. With the CTCAE, patient signs and symptoms are interpreted and reported by clinicians [3]. Given the subjective nature of many CTCAE criteria, differentiation of Grade 2 and 3 symptomatic AEs can be ambiguous [4], and it has been demonstrated that the reliability of CTCAE grading among providers is not robust. For example, examination of interobserver agreement of CTCAE criteria showed that independent providers agreed on the grading of fewer than half of all patients (45.9%) [5]. Furthermore, clinicians often underreport symptomatic events [6,7,8].

The Patient-Reported Outcomes version of the CTCAE (PRO-CTCAE) was developed to assess the frequency, severity, and/or interference of symptomatic AEs as directly reported by patients ( It measures 78 symptomatic AEs drawn from the CTCAE and was developed to complement the CTCAE data from clinical trials [3, 9, 10]. Akin to the CTCAE, the PRO-CTCAE assesses symptomatic AEs that have relevance across therapies and disease states. In any application of the PRO-CTCAE, it is important to ensure that items assessing all relevant symptomatic AEs are included. However, the exclusion of irrelevant items is also important to minimize respondent burden and measurement noise, and would better allow symptomatic AEs that are important to patients to be emphasized. Although several recommendations have been put forward for selecting subsets of items from the PRO-CTCAE that are relevant in a specific therapeutic context, including criteria based on the patient perspective, a consensus has not been reached [11, 12].

For many patients with advanced malignancies, there is currently a high unmet need for safe and effective therapies, and many currently available treatments have problematic side effects. Patients with hepatocellular carcinoma (HCC) are often diagnosed at advanced stages and have poor outcomes [13]. Sorafenib is approved for first-line treatment of patients with advanced HCC; regorafenib and nivolumab are approved as second-line treatments (nivolumab in the United States only). All 3 treatments have notable side effects [14,15,16]. Sorafenib has been associated with diarrhea, hand-foot skin reactions, and weight loss [14], regorafenib has been associated with hand-foot skin reactions, diarrhea, and fatigue [15], and nivolumab has been associated with diarrhea, pruritus, and rash [16]. Safety assessments in clinical trials could be better focused through patient and clinician identification of key symptomatic AEs with the use of the PRO-CTCAE.

Here we report results from a mixed-methods investigation, in which input from medical oncologists who treat patients with HCC and from patients with HCC was integrated with clinical trial data to identify a subset of symptomatic AEs from the PRO-CTCAE that were clinically relevant to HCC therapies and important to patients enrolled in HCC clinical trials. The approach described in this report enabled comparisons between patient and provider perceptions of symptomatic AEs in HCC, which could be used as a template for future studies.


Study population

This study analyzed cross-sectional, mixed-methods data that were collected from both clinical experts and patients with HCC. HCC clinical experts were 5 international medical oncologists from 5 locations (United States, Spain, Taiwan, Korea, and Hong Kong) with experience ranging from 14 to 30 years. The analysis patient population included 17 male and female English-speaking adult patients from the United States with HCC. Eligible patients had Child-Pugh class A or B disease, had hepatitis B or C virus infection or were uninfected, and had an Eastern Cooperative Oncology Group performance status of 0 or 1. Patients were ineligible if they had known fibrolamellar HCC or mixed cholangiocarcinoma and HCC, active co-infection with both hepatitis B and C viruses, and viral hepatitis other than type B or C. Other exclusion criteria were brain metastases or suspected brain metastases, concomitant second malignancies, uncontrolled or significant cardiac disease, prior liver transplant, known anti–programmed death 1/programmed death ligand 1 treatment, or any other mental or physical condition that in the judgment of study staff would impair the patient’s ability to participate in cognitive interviews or complete study questionnaires. Patients were recruited from 5 third-party contracted clinical sites; the study sponsor was blinded to the name and location of clinical sites, and the sites were blinded to the study sponsor. This study was approved by the Chesapeake Institutional Review Board (protocol 00013983). Patients provided written informed consent.

Data collection

Data were collected from medical oncologists with semi-structured, one-on-one, qualitative telephone interviews of 45 to 60 min. Medical oncologists were asked open-ended questions about HCC diagnostic pathways, treatment options, symptoms, and common treatment-related symptomatic AEs. Following the interview, they used a numerical rating scale to rate the perceived importance to patients (i.e., how burdensome) of 34 symptomatic AEs, on a scale of 0 (unimportant) to 10 (important). The 34 symptomatic AEs (Table 1) were selected based on safety data from prior phase 3 clinical trials (BRISK-FL (Brivanib Versus Sorafenib as First-line Treatment in Patients With Advanced HCC; NCT00858871), BRISK-PS (Brivanib Plus Best Supportive Care Versus Placebo Plus Best Supportive Care in Patients With Advanced HCC Who Have Failed or Are Intolerant to Sorafenib; NCT00825955), and BRISK-TA (Brivanib Versus Placebo as Adjuvant Therapy to Trans-Arterial Chemo-Embolization in Patients With Unresectable HCC; NCT00908752)). From these trials, symptomatic AEs were selected for review if they had a Grade ≥2.

Table 1 List of 34 symptomatic AEs ≥2 from sponsor trials rated by oncologists in one-on-one interviews

The medical oncologists also used a 0–10 numerical scale to rate the importance of all items within the PRO-CTCAE (v1.0), first with respect to safety or tolerability (i.e., dose limiting), then based on how important (i.e., how burdensome) they perceived the item would be to patients. An average pooled oncologist rating score ≥7 was selected as the cutoff for inclusion based on 70% (i.e., approximately two-thirds) of the total scale range. The team felt that this would optimize the balance between being inclusive and selecting a practical number of items for use within clinical trials.

Data were collected from patients with semi-structured, one-on-one, qualitative telephone interviews of 60 to 90 min. Patients were asked open-ended, concept-elicitation questions about their HCC diagnosis, the process by which they were diagnosed, the HCC symptoms they experienced, the types of treatment received, any treatment side effects, and the impact of symptomatic AEs on their quality of life. Patients also completed the PRO-CTCAE and underwent cognitive debriefing about the importance and relevance of each PRO-CTCAE item in the context of their experience with HCC. Following completion of the interview, patients completed a sociodemographic form, and the clinical sites completed brief case report forms to collect information on HCC treatments received, comorbidities, and current medications.

Data analysis

Audio responses from patient interviews were transcribed by a professional transcription service. Upon receipt of interview transcripts, all protected health information was deidentified, and a quality review of each transcript was conducted. A coding dictionary was developed to assist with review and analysis of the qualitative data.

A content analysis approach was used to assess qualitative data obtained from the patient interviews using ATLAS.ti software v7.0 (or later) [17], designed for the qualitative analysis of textual, graphical, audio, and video data. ATLAS.ti databases allow researchers to create concepts, or “codes,” that are used to organize and systematically analyze and compare large amounts of qualitative data to identify underlying themes and relationships.

For the concept-elicitation phase of the study, saturation was evaluated by entering ATLAS.ti codes for HCC-related symptoms and treatment-related symptomatic AEs into “saturation grids” that tracked the content of the interviews. Saturation is said to occur when no new information is gained during several sequential interviews [18, 19]. In this study, saturation was defined when no new symptomatic AEs were mentioned in two consecutive interviews. Concepts obtained from patient interviews were compared with the content of the PRO-CTCAE to determine whether there was appropriate content coverage. For the cognitive interview portion of the study, patient responses were coded to examine comprehension, relevance, and ease or difficulty of selecting a response for each item on the PRO-CTCAE.


Figure 1 presents a schematic diagram of patient recruitment. This was a challenging population to recruit and interview. Of 44 patients initially screened, 31 (70.4%) were considered eligible to participate. Of these, 17 (54.8%) were ultimately interviewed and analyzed. Among the 13 patients who were ineligible, 8 were medically ineligible, 1 died before he or she could be interviewed, 1 was psychiatrically ineligible, and 3 were listed as ineligible by the site because they did not want to participate. Of the 31 eligible patients, 3 did not want to participate, 7 could not be reached, 2 were later judged ineligible, and 19 were interviewed; however, 2 interviews were not analyzed due to disclosure of exclusion factors during the interview.

Fig. 1
figure 1

Patient recruitment. aInterview audio files were not transcribed because patient did not self-report having received any treatment for HCC

Medical oncologist assessments

Medical oncologists who were interviewed about treating patients with HCC were asked, “What are the most common symptomatic treatment-related toxicities?” Their summarized responses are presented in Table 2. Four of the 5 medical oncologists discussed symptomatic AEs related specifically to treatment with sorafenib, including skin toxicity (hand-foot syndrome, rash), diarrhea, fatigue, hypertension, and loss of appetite. The symptomatic AEs mentioned that were related to post–transcatheter chemoembolization (TACE) included pain (abdominal pain), fever, and chills.

Table 2 Medical oncologist summarized responses

Symptomatic AEs identified from prior clinical trials that oncologists rated as being most important (average score ≥7) to patients included hand-foot syndrome (8.60), diarrhea (7.80), fatigue (7.80), rash (7.40), decreased appetite (7.20), vomiting (7.20), and weight loss (7.00). The PRO-CTCAE symptomatic AEs that medical oncologists rated as being most important to patients were hand-foot syndrome (8.40), diarrhea (8.40), rash (8.00), fatigue (7.80), loss of control of bowel movements (7.60), vomiting (7.40), difficulty swallowing (7.20), shivering/shaking chills (7.20), and decreased appetite (7.00), whereas those rated most important for assessment of patient safety and tolerability were diarrhea (8.40) and shivering/shaking chills (7.40).

Patient-based assessments

The baseline demographics and clinical characteristics of patients with HCC recruited into the study are shown in Table 3. Patients had a mean age of 69 years (range, 54–84 years), and most (88%) were male. More than half (53%) of the patients self-reported that their general health was “good” or “very good.” Patients had received one or more different HCC treatments, which included chemotherapy (11.8%), TACE (29.4%), external radiation (5.9%), ablation (47.1%), liver resection (23.5%), Y90 radioembolization (35.3%), and sorafenib (29.4%). Nine (52.9%) patients had received one treatment, five (29.4%) had received two, one (5.9%) had received three, and two (11.8%) patients had received four different HCC treatments.

Table 3 Patient characteristics

During the qualitative interviews, a total of 33 symptomatic AEs were reported spontaneously by patients. Symptomatic AEs reported by ≥4 (i.e., ~25%) of the patients were fatigue/lack of energy/tiredness (n = 10), loss of appetite/lack of interest in food (n = 6), nausea (n = 6), pain/tenderness at injection/insertion site (n = 5), and hair loss (n = 4). The sequential listing of interviewed participants in Table 4 illustrates the degree of saturation of symptomatic AEs, as well as the frequency counts. The saturation grid demonstrates that the sixteenth and seventeenth interviewed participants did not mention any symptomatic AEs unique to what had been mentioned by the prior fifteen participants. Some descriptions from patients about symptomatic AEs are provided:

“My diet was terrible at first. When I first got home, you know, I could only eat very little in the way of it…it was lack of appetite. It had nothing to do with the food. It was just the lack of appetite.”

“I felt like I had to urinate and pee and I felt like throwing up, and I felt like, I would start sweating profusely. I said, you know, if this happens again, the nurse said and I said, well, just let me try it [Y90 treatment]. I’ll go, I promise I’ll go to bed, and I did. I went straight to bed and it just knocked me out.”

“I mentioned to the doctor that I was experiencing a lot of itching. Not rash, but itching, because it tells you right on the thing that you may or may not experience a rash and a whole mess of other things. But a lot of itching, and eventually that went away during the course of the year. But then, of course, then I started losing weight…and then I experienced other things like loss of appetite.”

“My hair just started falling out by the handful, and so that went on for quite a while. You know, I always shed tons of hair and still have tons of hair, and it was, my hair was falling out and really thin and stuff. And then after, I don’t know, maybe 5 or 6 months, it quit falling out so much.”

Table 4 Saturation grid

Figure 2 depicts the overlap of the oncologist and patient feedback on thirteen symptomatic AEs. These thirteen symptomatic AEs were those with an average pooled oncologist rating score ≥7 and/or endorsed by four or more patients. Decreased appetite and fatigue were frequently reported by patients and were highly rated by oncologists as being important to patients. As stated above and shown in Fig. 2, patients frequently reported nausea, pain/tenderness at insertion/injection site, and hair loss; these symptomatic AEs were not highly rated by clinicians. Conversely, loss of control of bowel movements and difficulty swallowing were highly rated by clinicians, but mentioned by one and zero patients, respectively.

Fig. 2
figure 2

Source of information for specific symptomatic adverse events. Venn diagram showing symptomatic adverse events highly rated by clinicians and important to patients


This study highlights the value of combining available data with clinician and patient input to identify a cancer-specific subset of PRO-CTCAE items. As seen in Fig. 2, we identified seven important symptomatic AEs based on clinical expert feedback on safety data from sponsor-conducted clinical trials. Clinical expert feedback on PRO-CTCAE items identified three additional symptomatic AEs as important. Finally, interviews with patients contributed another three symptomatic AEs. Overall, a preliminary list of 13 clinically relevant symptomatic AEs in HCC were identified using information from sponsor-conducted clinical trials, qualitative interviews with medical oncologists, and patients with HCC. Each source of information contributed at least one unique symptomatic AE. Had we not used all three sources, important AEs for measurement in HCC trials could have been overlooked. The 13 symptomatic AEs identified were hand-foot syndrome, diarrhea, fatigue, decreased appetite, rash, nausea, vomiting, weight loss, shivering/shaking chills, loss of control of bowel movements, difficulty swallowing, pain/tenderness at the injection site, and hair loss (Fig. 2).

Some of the 13 symptomatic AEs identified in this study are the same treatment-related AEs reported with nivolumab or regorafenib, systemic therapies that were recently approved for patients with advanced HCC. Diarrhea, decreased appetite, fatigue, weight decrease, and nausea were among the most frequent treatment-related AEs in patients with advanced HCC who were treated with nivolumab and were among the 13 symptomatic AEs reported in this analysis [16]. Likewise, commonly reported treatment-related AEs with regorafenib included hand-foot skin reaction, diarrhea, fatigue, nausea, weight loss, and vomiting, which are also covered in this list [15]. However, pruritus and abdominal pain were commonly reported in patients treated with nivolumab and regorafenib, respectively, but were not included among the symptomatic AEs identified in this study. Although there is some overlap in the list of symptomatic AEs identified in this study (of patients in whom a majority received locoregional therapy) with treatment-related AEs reported in patients who received new immune checkpoint or multikinase inhibitors, notable differences indicate that identification of relevant symptomatic AEs that could potentially be used as PRO-CTCAE items will likely be specific to a given class of antitumor therapy.

Because the focused symptomatic AE lists are intended for use in clinical trials that evaluate new treatments, it is important to include symptomatic AEs that cover all relevant toxicities of the treatments under study. It may be of value to target lists to different treatment types, because a list based on chemotherapies may not be completely pertinent to immunotherapies since it may not include immune-related adverse events. In our study, in spite of the small sample, we observed considerable heterogeneity of patient experiences with a variety of treatment exposures: the fact that 33 symptomatic AEs were mentioned by patients, of which 19 were endorsed by only one participant, and only one (fatigue) was mentioned by more than half, demonstrates the importance of including patients with a full range of treatment exposures. Furthermore, if patients are naïve to the experimental treatment, this could bias item selection against active comparators. The PRO-CTCAE includes an open-text field option for reporting experiences with AEs not represented in the library, such as weight loss. There also exists a process by which users of the PRO-CTCAE can partner with the National Cancer Institute to develop candidate items for use in research applications.

Another consideration is that all patients interviewed for this study were from the United States, whereas the medical oncologists were an international sample. Treatments for HCC and symptomatic AE reporting might vary based on geographic location. For example, in the GIDEON (Global investigation of therapeutic decisions in hepatocellular carcinoma and of its treatment with sorafenib) study, global variations were reported in the frequency and number of TACE treatments before sorafenib [20]. This difference in patterns of HCC treatment could affect the symptomatic AEs reported by patients. Thus, it may be important for future studies to include patients from the same countries as the clinicians enrolled in the study.

It is possible that the medical oncologists could have identified additional important symptomatic AEs. The list of AEs that they reviewed from sponsor-conducted clinical trials was generated from Grade ≥2 AEs. Had AEs of Grade 1 been presented they may have provided different feedback. However, Grade 1 AEs by definition imply ‘no intervention required’ and including them could have increased the number of non-specific items, thus potentially undermining the goal of the study, which was to develop a salient subset of PRO-CTCAE items for HCC. Interestingly, the medical oncologists did not identify some events observed in prior nivolumab and regorafenib trials in HCC. This suggests one cannot assume clinicians will identify all relevant events for drugs under study/yet to be approved.

Finally, socioeconomic status (SES) is a barrier that could limit oncology patient input [21, 22]. A growing body of evidence indicates that oncology patients of low SES have worse outcomes and a survival disadvantage compared with patients of higher SES. SES may be particularly relevant for a condition such as HCC [23, 24] because chronic viral hepatitis (a major risk factor for HCC) is correlated with substance abuse behaviors, which are in turn associated with poor education and unemployment [25, 26]. Thus, it is important for future studies to investigate how applicable the 13-item subset identified in this study is to patients with HCC who are of low SES.

Combining clinician and patient input enabled us to identify a more comprehensive set of AEs, including weight loss, an event not currently represented in the PRO-CTCAE library. Some of the symptomatic AEs such as fatigue/tiredness and lack of appetite were reported by both medical oncologists and patients. However, symptomatic AEs such as skin toxicities and diarrhea were reported commonly by medical oncologists but not by patients; patients frequently reported nausea, pain/tenderness at insertion/injection site, and hair loss in addition to fatigue/tiredness and lack of appetite. These discrepancies noted between patient and clinician input in this study highlight the importance of combining their perspectives and are in agreement with previous work that has also reported differences in clinician and patient reports of symptomatic AEs [6,7,8].

Evidence for the validity of this preliminary 13-item set of symptomatic Es should be collected in future HCC clinical trials. The methodology applied herein suggests that a relatively small subset of PRO-CTCAE items can be identified for application within specific tumors, using information from clinical trials and a limited number of physician and patient interviews. Inclusion of the patient perspective, rather than reliance on clinician reporting alone or previous information from sponsor clinical trials, helps capture subjective elements of safety information that may have both clinical and scientific value [1].


In summary, this study identified clinically relevant symptomatic AEs from both medical oncologists and patients with HCC that can be used in evaluating treatment tolerability in HCC clinical trials. Through the methodology applied herein, we were able to reduce the 78 symptomatic AEs represented in the PRO-CTCAE library to a set of 13 events warranting further exploration. These results support incorporating both patient and clinician perspectives in identifying PRO-CTCAE items for use in cancer clinical trials.



Adverse event


Common Terminology Criteria for Adverse Events


Hepatocellular carcinoma


Patient-reported outcome


Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events


Socioeconomic status


Transarterial chemoembolization


  1. Basch, E. (2010). The missing voice of patients in drug-safety reporting. N Engl J Med, 362, 865–869.

    Article  CAS  Google Scholar 

  2. National Cancer Institute (2009). Common Terminology Criteria for Adverse Events (CTCAE) version 4.0. Accessed 18 July 2018.

  3. Basch, E., Reeve, B. B., Mitchell, S. A., et al. (2014). Development of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer Inst, 106(9).

  4. Kuroi, K., Shimozuma, K., Ohashi, Y., et al. (2009). Prospective assessment of chemotherapy-induced peripheral neuropathy due to weekly paclitaxel in patients with advanced or metastatic breast cancer (CSP-HOR 02 study). Support Care Cancer, 17, 1071–1080.

    Article  Google Scholar 

  5. Postma, T. J., Heimans, J. J., Muller, M. J., et al. (1998). Pitfalls in grading severity of chemotherapy-induced peripheral neuropathy. Ann Oncol, 9, 739–744.

    Article  CAS  Google Scholar 

  6. Di Maio, M., Gallo, C., Leighl, N. B., et al. (2015). Symptomatic toxicities experienced during anticancer treatment: Agreement between patient and physician reporting in three randomized trials. J Clin Oncol, 33, 910–915.

    Article  CAS  Google Scholar 

  7. Basch, E., Jia, X., Heller, G., et al. (2009). Adverse symptom event reporting by patients vs clinicians: Relationships with clinical outcomes. J Natl Cancer Inst, 101, 1624–1632.

    Article  Google Scholar 

  8. Basch, E., Iasonos, A., McDonough, T., et al. (2006). Patient versus clinician symptom reporting using the National Cancer Institute common terminology criteria for adverse events: Results of a questionnaire-based study. Lancet Oncol, 7, 903–909.

    Article  Google Scholar 

  9. Dueck, A. C., Mendoza, T. R., Mitchell, S. A., et al. (2015). Validity and reliability of the US National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). JAMA Oncol, 1, 1051–1059.

  10. Hay, J. L., Atkinson, T. M., Reeve, B. B., et al. (2014). Cognitive interviewing of the US National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). Qual Life Res, 23, 257–269.

    Article  Google Scholar 

  11. Kluetz, P. G., Chingos, D. T., Basch, E. M., et al. (2016). Patient-reported outcomes in cancer clinical trials: Measuring symptomatic adverse events with the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). Am Soc Clin Oncol Educ Book, 35, 67–73.

    Article  Google Scholar 

  12. Critical Path Institute. (2016). NCI’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE): selecting items and assessment frequency for cancer trials. Presented at: Seventh Annual Patient-Reported Outcome (PRO) Consortium Workshop; April 27–28, 2016; Silver Spring, MD.

  13. Sun, V. C., & Sarna, L. (2008). Symptom management in hepatocellular carcinoma. Clin J Oncol Nurs, 12, 759–766.

    Article  Google Scholar 

  14. Llovet, J. M., Ricci, S., Mazzaferro, V., et al. (2008). Sorafenib in advanced hepatocellular carcinoma. N Engl J Med, 359, 378–390.

    Article  CAS  Google Scholar 

  15. Bruix, J., Merle, P., Granito, A., et al. (2016). Efficacy and safety of regorafenib versus placebo in patients with hepatocellular carcinoma (HCC) progressing on sorafenib: Results of the international, randomized phase 3 RESORCE trial. Presented at: European Society for Medical Oncology 18th World Congress on Gastrointestinal Cancer; June 29–July 2, 2016; Barcelona, Spain.

    Google Scholar 

  16. El-Khoueiry, A. B., Sangro, B., Yau, T., et al. (2017). Nivolumab in patients with advanced hepatocellular carcinoma (CheckMate 040): An open-label, non-comparative, phase 1/2 dose escalation and expansion trial. Lancet, 389, 2492–2502.

    Article  CAS  Google Scholar 

  17. Friese, S., & Ringmayr, G. T. (2013). ATLAS.ti 7 user manual. Berlin: ATLAS.ti Scientific Software Development GmbH.

  18. Urquhart, C. (2013). Grounded theory for qualitative research: A practical guide (p. 194). Thousand Oaks: Sage.

    Book  Google Scholar 

  19. Given, L. M. (2016). 100 questions (and answers) about qualitative research (p. 135). Thousand Oaks: Sage.

    Google Scholar 

  20. Geschwind, J. F., Kudo, M., Marrero, J. A., et al. (2016). TACE treatment in patients with sorafenib-treated unresectable hepatocellular carcinoma in clinical practice: Final analysis of GIDEON. Radiology, 279, 630–640.

    Article  Google Scholar 

  21. El-Rayes, B. F., Jasti, P., Severson, R. K., et al. (2010). Impact of race, age, and socioeconomic status on participation in pancreatic cancer clinical trials. Pancreas, 39, 967–971.

    Article  Google Scholar 

  22. Sharrocks, K., Spicer, J., Camidge, D. R., et al. (2014). The impact of socioeconomic status on access to cancer clinical trials. Br J Cancer, 111, 1684–1687.

    Article  CAS  Google Scholar 

  23. Jembere, N., Campitelli, M. A., Sherman, M., et al. (2012). Influence of socioeconomic status on survival of hepatocellular carcinoma in the Ontario population; A population-based study, 1990-2009. PLoS One, 7, e40917.

    Article  CAS  Google Scholar 

  24. Hoehn, R. S., Hanseman, D. J., Jernigan, P. L., et al. (2015). Disparities in care for patients with curable hepatocellular carcinoma. HPB (Oxford), 17, 747–752.

    Article  Google Scholar 

  25. Compton, W. M., Gfroerer, J., Conway, K. P., & Finger, M. S. (2014). Unemployment and substance abuse outcomes in the United States 2002-2010. Drug Alcohol Depend, 142, 350–353.

    Article  Google Scholar 

  26. Zimmerman, E., & Woolf, S. H. (2014). Understanding the relationship between education and health. Institute of Medicine, Washington, DC. Accessed 3 Oct 2018.

Download references


This study was supported by Bristol-Myers Squibb (Princeton, NJ, USA). The authors thank the patients and their families and research staff at all study sites. The authors thank Caroleen Quach, MSPH, PhD, who was affiliated with Bristol-Myers Squibb at the time much of this research was conducted, and Peter Chongpinitchai, MBS, of Evidera, who conducted the majority of the patient interviews. The authors are also grateful to the National Cancer Institute for providing the PRO-CTCAE for use in this research. Editorial assistance was provided by Bridget Sackey Aboagye, PhD, of Chrysalis Medical Communications, Inc. (Hamilton, NJ, USA) and was funded by Bristol-Myers Squibb.

Author information

Authors and Affiliations



RMS and WRL were involved with study design, data collection, data analysis, and manuscript preparation. JWS was involved with study design and manuscript preparation. All authors read and approved the final manuscript.

Corresponding author

Correspondence to James W. Shaw.

Ethics declarations

Competing interests

RMS and WRL are employees of Evidera. JWS is an employee and shareholder of BMS.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Speck, R.M., Lenderking, W.R. & Shaw, J.W. Integrating the patient voice with clinician reports to identify a hepatocellular carcinoma-specific subset of treatment-related symptomatic adverse events. J Patient Rep Outcomes 2, 35 (2018).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: