Skip to main content

Results on patient-reported outcomes are underreported in summaries of product characteristics for new drugs

Abstract

Background

Summaries of product characteristics (SmPCs) are regulatory documents published upon drug approval. They should report all relevant study data and advise how to use drugs safely and effectively. Patient-reported outcomes (PROs) are increasingly used in clinical trials to incorporate the patient perspective—SmPCs should thus adequately report PROs. In Germany, new drugs undergo mandatory early benefit assessment. Pharmaceutical companies submit dossiers containing all evidence; the subsequent dossier assessments focus on patient-relevant outcomes and comprehensively report PROs.

Objective

The primary aim was to investigate to what extent PROs recorded as outcomes in clinical trials of new drugs are reported in SmPCs.

Methods

We analysed dossier assessments with randomized controlled trials (RCTs) of new drugs entering the market between 01/2014 and 07/2018 and the corresponding SmPCs, and compared PRO reporting in both document types. For this purpose, we evaluated dossier assessment characteristics (e.g. drug name, indication, disease category) and study characteristics (e.g. evaluable PROs available?). PROs were divided into symptoms and health-related quality of life (HRQoL). SmPCs were screened to identify RCTs. We conducted 3 main evaluation steps: (1) Did the RCT included in the dossier assessment contain evaluable PROs? (2) If yes, was the RCT included in the SmPC? (3) If yes, were the PROs reported in the SmPC? Results are presented descriptively.

Results

88 dossier assessments including 143 RCTs on 72 drugs were considered: 109 (76.2%) RCTs included evaluable PROs, of which 89 were included in SmPCs. 38 RCTs (42.7%) investigated oncologics, 18 (20.2%) anti-infectives, and 33 (37.1%) other drugs. The RCTs considered symptoms more often than HRQoL (82 vs. 66 RCTs). In SmPCs, PROs were reported for 41 RCTs (46.1%), with a slightly higher reporting rate for RCTs considering HRQoL (43.9%) than for RCTs considering symptoms (41.5%). In oncologic indications, PROs were reported for 36.7% of RCTs considering HRQoL and 33.3% of RCTs considering symptoms. In infectious diseases, the rates were 21.4% (symptoms) and 0% (HRQoL), and for other diseases about 60% (symptoms) to 70% (HRQoL).

Conclusion

Even though a large amount of PRO data on new drugs is available from clinical trials included in SmPCs, the corresponding results are underreported.

Background

Clinical trials used for regulatory approval provide essential information on new drugs. Traditionally, they primarily focus on objective clinical outcomes and laboratory parameters. However, the importance of considering the patient perspective to fully capture the benefits and harms of new drugs as a basis for regulatory, health policy, clinical and patient-centred decision-making is now widely accepted. Patient-reported outcomes (PROs) are therefore also increasingly being used in clinical trials to measure effectiveness [1,2,3,4,5,6].

PROs cover both single and multi-dimension measures of outcomes such as symptoms and health-related quality of life (HRQoL). They “provide a unique means of capturing the personal and social context of the disease and treatment experience, as OS (overall survival), PFS (progression-free survival), biomarker measures or adverse events may not necessarily capture the full impact of a treatment on how a patient feels or functions” [2].

In Europe, a drug’s marketing authorization includes the creation of a legal document called a summary of product characteristics (SmPC), which is published at the time of market authorization or extension of therapeutic indication. The SmPC provides healthcare professionals with information on the drug’s properties, approved conditions of use, safe and effective use, and main results of clinical trials supporting market authorization. It is updated regularly to include new data (e.g. on safety) [7,8,9].

In the United States, the equivalent of the SmPC is the prescribing information (also called product information or labelling or package insert) [10].

In Germany, an early benefit assessment according to the Act on the Reform of the Market for Medicinal Products (AMNOG) is conducted for every new drug after market entry and for previously approved patented drugs with new therapeutic indications [11]. The Federal Joint Committee (G-BA), the main decision-making body in the healthcare system, generally commissions the Institute for Quality and Efficiency in Health Care (IQWiG) for this purpose. In the early benefit assessment, symptoms and HRQoL are always considered as essential patient-relevant outcomes besides mortality and adverse effects. The target population investigated is not necessarily identical to the one defined in an SmPC, as specific aspects of the German healthcare setting may be considered (e.g. a subpopulation receiving the standard care defined by the G-BA). All data on a new drug, including the clinical study reports, must be submitted in a dossier by the manufacturer and all relevant study results are published online in a so-called dossier assessment, including those on PROs [11, 12].

In line with regulatory requirements [2, 3], only PROs recorded with valid and suitable tools are considered in dossier assessments in Germany; we would thus expect these PROs, at least the ones included in the main trials supporting market authorization, to be reported in SmPCs.

Objective

Our primary aim was to investigate to what extent PROs recorded as outcomes in clinical trials used for regulatory approval are reported in SmPCs of new drugs. We therefore compared the reporting of PROs in dossier assessments with the reporting of PROs in the corresponding SmPCs. We also investigated whether the reporting of PROs in SmPCs depended on the direction of the treatment effect.

Methods

Data sources

Eligible dossier assessments were identified via IQWiG’s internal (non-public) AMNOG database. All assessments were considered that included randomized controlled trials (RCTs) and assessed drugs entering the market or receiving an extension of therapeutic indication between 01.01.2014 and 01.07.2018. Orphan drugs were excluded: orphan drug status is automatically associated with a (fictitious) added benefit; only the extent of added benefit is assessed, primarily by the G-BA. The dossier assessments were downloaded from the G-BA website [13].

The corresponding SmPCs (including any updates) were identified via the EU Register of medicinal products [14].

Data extraction and evaluation

We extracted and evaluated the following information from the dossier assessments:

  • Characteristics of the dossier assessment project number, drug name, therapeutic indication (i.e. target disease and target population), and disease category.

  • Study characteristics:

    • name of RCT

    • features of RCT:

      • whether it investigated the target population specified in the SmPC or a different population (specified in the dossier assessment), including the reason for this deviation;

      • whether it included evaluable PROs, i.e. sufficient data recorded with valid and suitable tools.

  • From each RCT included in the dossier assessment, the PROs were evaluated separately for the outcome categories of morbidity (symptoms) and HRQoL, according to the classification in the dossier assessments.

The corresponding SmPCs were screened to identify the RCTs included in the dossier assessment. The PROs included in these RCTs and reported in the SmPCs were then evaluated as described above.

Data were extracted and evaluated by one author (LJ) according to the table in Additional file 1 and then checked by another (SH). Any discrepancies were resolved by consensus; overall, discrepancies in data evaluation between the two authors were low (< 10%).

Information synthesis and analysis

We first compiled the information extracted, followed by a 5-step procedure:

  1. 1.

    Did the RCT included in the dossier assessment contain evaluable PRO data?

    • yes/no; per outcome category

  2. 2.

    If yes, was the RCT also included in the SmPC?

    • yes/no

  3. 3.

    If yes, were the PROs that were reported in the dossier assessment also reported in the SmPC?

    • yes/no; per outcome category and disease category

The following steps only considered those RCTs where the whole study population or a subpopulation specified in the therapeutic indication of the SmPC was included in the dossier assessment investigating this target population (e.g., a subpopulation with high disease severity), as in these cases, the PROs reported in the SmPC were expected to be consistent with the PROs reported in the dossier assessment.

  1. 4.

    To what extent were the PROs reported in the SmPC?

    • completely/partly/not reported; per outcome category

  2. 5.

    To what extent were the PROs reported in the SmPC according to the direction of the treatment effect?

    • completely/partly/not reported; per outcome category and effect category (positive vs. negative/no effect—the latter was combined post hoc due to the small number of RCTs showing a negative effect, see Additional file 2).

All results are presented descriptively without further statistical testing.

Results

Document and study characteristics

88 dossier assessments including 143 RCTs on 72 drugs were considered (Fig. 1). 109 (76.2%) of these RCTs included evaluable PRO data, of which 89 were included as main trials in the SmPCs: 38 RCTs (42.7%) investigated oncologics, 18 (20.2%) anti-infectives (all for HIV or chronic hepatitis C), and 33 (37.1%) other drugs. For 50 of the 89 RCTs, the population investigated in the dossier assessments corresponded to the target population specified in the SmPC.

Fig. 1
figure 1

Flow chart of the data set included. HIV human immunodeficiency virus, N number of RCTs, PRO patient-reported outcome, RCT randomized controlled trial, SmPC summary of product characteristics

Reporting rates for PROs in SmPCs

Overall results

PRO data were reported for 41 out of 89 RCTs (46.1%) considering evaluable PROs and included in the SmPCs (Fig. 2a). In the 89 RCTs, PROs on symptoms were investigated more often than those on HRQoL, but the reporting rate for PROs from RCTs considering HRQoL was slightly higher.

Fig. 2
figure 2

Reporting rate of PROs in SmPCs: a by outcome category, b by outcome/disease category. *Proportion of RCTs with evaluable PRO data reported in the SmPCs. HRQoL health-related quality of life, N number of RCTs with evaluable PRO data included in dossier assessments, PRO patient-reported outcome, SmPC summary of product characteristics

Figure 2b shows reporting according to disease category: The reporting rate was lowest for PROs in infectious diseases (PRO data were reported for 21.4% of RCTs considering symptoms and 0% of RCTs considering HRQoL). In oncologic indications, PRO data were reported for 33.3% of RCTs considering symptoms and 36.7% of RCTs considering HRQoL. In other disease categories, PRO data were reported for about 60% (symptoms) to 70% (HRQoL) of the RCTs.

RCTs investigating the target population specified in SmPCs

The reporting rate was even lower for the PROs from the 50 RCTs covering the target population specified in the SmPCs. PRO data were fully or partly reported only for 32.6% (symptoms) and 25.8% (HRQoL) of these RCTs (Additional file 3).

Selective outcome reporting

The data investigating whether the extent of reporting in SmPCs depended on the direction of the treatment effect on the PROs are shown in Additional file 3. For RCTs considering symptoms, PRO data were reported for 50% of the RCTs showing positive effects, whereas this was the case only for 18% of RCTs showing negative or no effects. However, for RCTs considering HRQoL, there were only small differences in the corresponding reporting rates.

Discussion

Despite endeavours to incorporate the patient perspective in drug development and regulatory decision-making, a substantial amount of information on PROs considered in RCTs on new drugs is not reported in the corresponding SmPCs. In the two disease categories with sufficient data for separate analysis, oncology and infectious diseases, reporting of PROs was poor for both categories and particularly poor for the latter. An explanation could be that the assessment of treatment benefit in infectious diseases is traditionally based on biomarkers [15].

No consistent difference was shown between reporting rates for HRQoL and those for symptoms, so no conclusions can be drawn. The results on selective outcome reporting were also inconclusive.

Comparison with previous research

In our analysis, low reporting rates of PROs in SmPCs were evident in oncologic indications, even though regulatory authorities specifically recommend considering PROs [2], for instance, to inform choices between therapies with similar efficacy results, especially in palliative settings [16]. Although comparability with previous research is limited due to methodological differences, our findings of poor reporting for PROs are generally supported, both for oncologic indications and across indications. A review of PRO labelling for oncologics approved by the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA), 2012–2016, found that while no FDA PRO labelling was identified, the EMA SmPCs included PRO information for 21 (46.7%) of 45 indications. The authors explained this discrepancy by “different evidentiary standards to assess PRO data from oncology studies, with the EMA more likely to accept data from open-label studies and broad concepts such as health-related quality of life” [17].

In a review of FDA PRO labelling for 182 new drug applications across indications (2011–2015), 16.5% had PRO labelling, almost all for concepts proximal to the disease [15]. In a further review across indications, which investigated the role and extent of PRO usage within EMA European Public Assessment Reports (2008–2012), which contain an SmPC section, PROs as any outcomes were used for 82 of 180 drugs (46%) [18].

Non- or delayed reporting of PROs is also evident in journal publications. Out of 231 primary publications of oncology randomized phase III trials with HRQoL data as a secondary or exploratory outcome, HRQoL results were available in only 143 (61.9%). For trials without HRQoL results in the primary publication, probability of secondary publication with such results was 12.5% after 1 and 40.3% after 3 years [16].

With regard to selective outcome reporting, the systematic review above [16] found that the non-reporting of HRQoL data in primary publications was similar in trials with negative and positive results, indicating that the main problem seems to be underreporting, not selective reporting.

Our analysis expands on previous research by adding the HTA perspective and highlighting the value of PROs as clinically relevant outcomes for health policy decision-making.

The role of SmPCs and dossier assessments

SmPCs can support patient education (e.g. regarding rare adverse effects) and clinical decision-making, as they often contain unique and detailed information on a drug [19]. However, their user friendliness has been criticized; for instance, advice may be vague or information difficult to find [19, 20]. User testing of the effectiveness of SmPCs in communicating essential information to prescribers found that they are of low perceived value and not central to prescribing behaviour [21]. Merely including PROs as a default in the SmPC would thus not suffice; to increase usage of PROs (and of SmPCs in general), user friendliness of SmPCs would also need to be improved. In addition, regular updating should be ensured for PRO data. The lower PRO reporting rates in SmPCs compared with dossier assessments might be explained by the traditional focus of regulators on objective clinical outcomes and laboratory parameters. Thus, even though pharmaceutical companies record PROs in clinical trials, they are not necessarily reported in SmPCs, as they are not regarded to be of major relevance for the regulatory decision-making procedure [18, 22]. We would like to note that, as an HTA agency, we have a different perspective than a drug approval authority and the question arises whether it is appropriate to apply HTA standards to a regulatory outcome document such as the SmPC. However, according to the European Commission’s SmPC guideline, regarding SmPC content “It may be appropriate to provide limited information, relevant to the prescriber, such as the main results … regarding pre-specified end points or clinical outcomes in the major trials…” [9]; in our opinion, information relevant to the prescriber must include information on PROs.

Dossier assessments, which are also available in English, may represent a valuable additional PRO source. Examples where PROs informed health policy or clinical decision-making, but were not reported in SmPCs, indicating that they did not contribute much to regulatory decision-making, are presented in Additional file 4. Overall, both the amount of PRO data considered in dossier assessments and their relevance for decisions on added benefit have increased over the years since the introduction of AMNOG [23, 24], indicating that both the generation and the quality of PRO data in clinical trials have improved. However, there is still much room for improvement here and moreover, the mere availability of high-quality PRO data does not necessarily translate into better consideration in healthcare decision-making in practice. A systematic review of the impact of PROs from clinical trials showed relatively limited evidence demonstrating real world PRO-related research impact [25]. Therefore, PROs should not only be reported more frequently, but should generally be given greater importance in treatment decisions.

Limitations

Our sample size was relatively small and we did not conduct statistical tests, so we cannot draw any conclusions on the statistical significance of differences between our results. In addition, we only considered basic outcome categories for PROs.

Conclusion

Even though a large amount of PRO data on new drugs is available from RCTs included in SmPCs, the corresponding results are underreported. PROs should be reported as default in SmPCs to support informed decision-making in healthcare that ensures adequate consideration of the patient perspective.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AMNOG:

Arzneimittelmarktneuordnungsgesetz (Act on the Reform of the Market for Medicinal Products)

EMA:

European Medicines Agency

FDA:

Food and Drug Administration

G-BA:

Gemeinsamer Bundesausschuss (Federal Joint Committee)

HRQoL:

Health-related quality of life

IQWiG:

Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (Institute for Quality and Efficiency in Health Care)

PRO:

Patient-reported outcome

RCTs:

Randomized controlled trial

SmPC:

Summary of product characteristics

References

  1. Kluetz PG, O’Connor DJ, Soltys K (2018) Incorporating the patient experience into regulatory decision making in the USA, Europe, and Canada. Lancet Oncol 19:e267–e274

    Article  Google Scholar 

  2. European Medicines Agency (2016) Appendix 2 to the guideline on the evaluation of anticancer medicinal products in man; The use of patient-reported outcome (PRO) measures in oncology studies. https://www.ema.europa.eu/en/documents/other/appendix-2-guideline-evaluation-anticancer-medicinal-products-man_en.pdf. Accessed 16 Mar 2021

  3. Food and Drug Administration (2009) Guidance for industry; patient-reported outcome measures: use in medical product development to support labeling claims. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf. Accessed 18 Mar 2015

  4. Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, Brundage MD (2013) Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA 309:814–822

    CAS  Article  Google Scholar 

  5. Calvert M, Kyte D, Duffy H, Gheorghe A, Mercieca-Bebber R, Ives J et al (2014) Patient-reported outcome (PRO) assessment in clinical trials: a systematic review of guidance for trial protocol writers. PLoS ONE 9:e110216

    Article  Google Scholar 

  6. Chalasani M, Vaidya P, Mullin T (2018) Enhancing the incorporation of the patient’s voice in drug development and evaluation. Res Involv Engagem 4:10

    Article  Google Scholar 

  7. European Medicines Agency. Summary of product characteristics (SmPC); What is it and what does it contain? n.d. https://www.ema.europa.eu/documents/presentation/presentation-summary-product-characteristics-what-it-what-does-it-contain_en.pdf. Accessed 05 Nov 2020

  8. European Medicines Agency. How to prepare and review a summary of product characteristics. n.d. https://www.ema.europa.eu/en/human-regulatory/marketing-authorisation/product-information/how-prepare-review-summary-product-characteristics. Accessed 16 Mar 2021

  9. European Commission (2009) A guideline on summary of product characteristics (SmPC). http://ec.europa.eu/health/files/eudralex/vol-2/c/smpc_guideline_rev2_en.pdf. Accessed 16 Mar 2021

  10. Food and Drug Administration. Drug advertising: a glossary of terms; prescribing information. n.d. https://www.fda.gov/drugs/prescription-drug-advertising/drug-advertising-glossary-terms#prescribing_information. Accessed 16 Mar 2021

  11. Bundesministerium für Gesundheit. Gesetz zur Neuordnung des Arzneimittelmarktes in der gesetzlichen Krankenversicherung (Arzneimittelmarktneuordnungsgesetz - AMNOG). Bundesgesetzblatt. 2010:2262–77

  12. Hörn H, Nink K, McGauran N, Wieseler B (2014) Early benefit assessment of new drugs in Germany: results from 2011 to 2012. Health Policy 116:147–153

    Article  Google Scholar 

  13. Gemeinsamer Bundesausschuss. Verfahren der Nutzenbewertung nach § 35a SGB V. n.d. https://www.g-ba.de/bewertungsverfahren/nutzenbewertung/. Accessed 05 Mar 2020

  14. European Commission. Union Register of medicinal products for human use. n.d. https://ec.europa.eu/health/documents/community-register/html/reg_hum_act.htm?sort=a. Accessed 11 Aug 2020

  15. Gnanasakthy A, Mordin M, Evans E, Doward L, DeMuro C (2017) A review of patient-reported outcome labeling in the united States (2011–2015). Value Health 20:420–429

    Article  Google Scholar 

  16. Marandino L, La Salvia A, Sonetto C, De Luca E, Pignataro D, Zichi C et al (2018) Deficiencies in health-related quality-of-life assessment and reporting: a systematic review of oncology randomized phase III trials published between 2012 and 2016. Ann Oncol 29:2288–2295

    CAS  Article  Google Scholar 

  17. Gnanasakthy A, Barrett A, Evans E, D’Alessio D, Romano CD (2019) A review of patient-reported outcomes labeling for oncology drugs approved by the FDA and the EMA (2012–2016). Value Health 22:203–209

    Article  Google Scholar 

  18. Bansal D, Bhagat A, Schifano F, Gudala K (2015) Role of patient-reported outcomes and other efficacy endpoints in the drug approval process in Europe (2008–2012). J Epidemiol Glob Health 5:385–395

    Article  Google Scholar 

  19. Why look at an SPC? Drug Ther Bull 47:56–8 (2009)

  20. Albassam A, Hughes DA (2021) What should patients do if they miss a dose? A systematic review of patient information leaflets and summaries of product characteristics. Eur J Clin Pharmacol 77:251–260

    Article  Google Scholar 

  21. Raynor DK, Veene P, Bryant D (2014) The effectiveness of the summary of product characteristics (SmPC) and recommendations for improvement. Ther Innov Regul Sci 48:255–265

    Article  Google Scholar 

  22. European Medicines Agency (2018) Guideline on the evaluation of anticancer medicinal products in man. https://www.ema.europa.eu/documents/scientific-guideline/guideline-evaluation-anticancer-medicinal-products-man-revision-5_en.pdf. Accessed 05 Oct 2021

  23. Lohrberg D, Augustin M, Blome C (2016) The definition and role of quality of life in Germany’s early assessment of drug benefit: a qualitative approach. Qual Life Res 25:447–455

    Article  Google Scholar 

  24. Storm A. AMNOG-Report 2019: Nutzenbewertung von Arzneimitteln in Deutschland; Schwerpunkt: Höchstpreise für ein Mehr an Lebensqualität? 2019. https://www.dak.de/dak/download/amnog-report-2019-pdf-2099700.pdf. Accessed 28 Sep 2021

  25. Rivera SC, Kyte DG, Aiyegbusi OL, Slade AL, McMullan C, Calvert MJ (2019) The impact of patient-reported outcome (PRO) data from clinical trials: a systematic review and critical analysis. Health Qual Life Outcomes 17:156

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank Ursula Gundert-Remy, Drug Commission of the German Medical Association, Berlin, for her comments on the role of summaries of product characteristics as well as staff from the Institute for Quality and Efficiency in Health Care, Cologne, for their comments on the manuscript (Marie-Luise Brandi, Stefan Sauerland) and the formatting check of citations (Verena Wekemann).

Funding

Open Access funding enabled and organized by Projekt DEAL. This work was supported by non-financial support from the Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG). No external funding was received.

Author information

Authors and Affiliations

Authors

Contributions

TK conceived the idea for the analysis. TK, RP, SH, and LJ prepared the project outline. LJ performed data extraction and evaluation. SH checked data extraction and evaluation. FL analysed the data. SH, NM, MP and TK drafted the manuscript. All authors contributed to data interpretation and to critical revision of the manuscript and have reviewed and approved the final version.

Corresponding author

Correspondence to Susanne Haag.

Ethics declarations

Ethics approvals and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table.

Information evaluated in our analysis and coding options.

Additional file 2: Figure.

Completeness of reporting of PROs in SmPCs grouped by direction of treatment effects (positive effects / no difference / negative effects)

Additional file 3: Figure.

Completeness of reporting of PROs in SmPCs: (a) grouped by outcome category, (b) grouped by outcome category and direction of treatment effects (positive effects / negative or no effects)

Additional file 4: Table.

Examples of dossier assessments with relevant PROs missing in the SmPC

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Haag, S., Junge, L., Lotz, F. et al. Results on patient-reported outcomes are underreported in summaries of product characteristics for new drugs. J Patient Rep Outcomes 5, 127 (2021). https://doi.org/10.1186/s41687-021-00402-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41687-021-00402-1