PROMs and PREMs in routine perinatal care: mixed methods evaluation of their implementation into integrated obstetric care networks

Abstract

Background

In the transition towards value-based healthcare, patient-reported outcome and experience measures (PROM and PREM) are recommended by international collaborations and government programs to guide clinical practice and quality improvement. For many conditions, using PROM/PREM over the complete continuum of care requires implementation across care organizations and disciplines. Along PROM/PREM implementation in obstetric care networks (OCN), we aimed to evaluate implementation outcomes and the processes influencing these outcomes in the complex context of care networks across the continuum of perinatal care.

Methods

Three OCN in the Netherlands implemented PROM/PREM in routine practice, using an internationally developed outcomes set with care professionals and patient advocates. Their aim was to use PROM/PREM results individually to guide patient-specific care decisions and at group-level to improve quality of care. The implementation process was designed following the principles of action research: iteratively planning implementation, action, data generation and reflection to refine subsequent actions, involving both researchers and care professionals. During the one-year implementation period in each OCN, implementation outcomes and processes were evaluated in this mixed-methods study. Data generation (including observation, surveys and focus groups) and analysis were guided by two theoretical implementation frameworks: the Normalization Process Theory and Proctor’s taxonomy for implementation outcomes. Qualitative findings were supplemented with survey data to solidify findings in a broader group of care professionals.

Results

Care professionals in OCN found the use of PROM/PREM acceptable and appropriate, recognized their benefits and felt facilitated in their patient-centered goals and vision. However, feasibility for daily practice was low, mainly due to IT issues and time constraints. Hence PROM/PREM implementation did not sustain, but strategies for future PROM/PREM implementation were formulated in all OCN. Processes contributing positively to implementation outcomes were internalization (understand the value) and initiation (driven by key-participants), whereas challenges in relational integration (maintain confidence) and reconfiguration (refine activities) affected implementation negatively.

Conclusion

Although implementation did not sustain, network-broad PROM/PREM use in clinic and quality improvement matched professionals’ motivation. This study provides recommendations to implement PROM/PREM meaningfully in practice in ways that support professionals in their drive towards patient-centered care. In order for PROM/PREM to fulfill their potential for value-based healthcare, our work highlights the need for sustainable IT infrastructures, as well as an iterative approach to refine their complex implementation into local contexts.

Background

In the past decade, the discourse of value-based healthcare (VBHC) has had an immense uptake in healthcare [1]. At system level, healthcare systems strive to use patients’ well-being to evaluate care performance for full treatment cycles for a condition [2]. At patient level, professionals aim to organize integrated care around a health condition and make personal values prescriptive to guide treatment decisions [3]. In the development towards VBHC, patient-reported outcomes and experiences measures (PROM and PREM) have been embraced to generate data about what matters to patients and drive patient-centered quality improvement (QI) [4]. Therefore, the capture and use of PROM/PREM has been encouraged in many healthcare settings by international collaborations and government programs [5, 6]. Nevertheless, PROM/PREM implementation remains challenging, especially in network settings like perinatal care [7, 8].

PROM/PREM implementation has been considerably studied with an implementation science approach, identifying common influencing factors such as technology and clinical leadership [9, 10]. Different challenges have been described dependent on the purpose of PROM/PREM implementation. For example, a challenge for individual-level use includes fitting PROM collection to appointment schedules, while at group-level motivating care professionals for (external) QI appears more challenging [8, 9]. Yet most implementation factors have been explored in single organization settings or primary care predominantly [11, 12], whereas the majority of health conditions require interdisciplinary and interorganizational collaboration across healthcare tiers to provide the full continuum of care [13, 14]. As for pregnancy and childbirth, where care professionals collaborate both interdisciplinary (e.g., obstetrics, neonatology) and interorganizational (e.g., hospitals, midwife practices, youth care) to provide acute and long-term care with in-hospital, outpatient and community-based care and support. Thus, to contribute to patient- and family-centered care, PROM/PREM in perinatal care would ideally be implemented across care networks, to cover patients’ whole care trajectory in individual-level use and involve all stakeholders in group-level use for QI. Yet, implementation in network context prompts other challenges, like engaging diverse stakeholders, aligning incentives and resources, and building common infrastructures [4, 15]. Evaluations of individual-level PROM/PREM implementation in network context are scarce, but needed to advance our understanding of practice challenges, contextual factors, and mechanisms through which implementation strategies work across organizations [10, 16].

For perinatal care, until recently, no consensus on PROM/PREM had been formed to evaluate its patient outcomes [17]. Yet, in 2017, a set of standardized patient-centered outcomes measures for pregnancy and childbirth (PCB set) was developed internationally with perinatal care professionals and patient advocates [17, 18]. This set includes PROM/PREM from beginning of pregnancy until six months postpartum. Over the last years, the PCB set has been adopted internationally and implementation efforts have been started worldwide, of which most are in research context [19,20,21]. Potential factors influencing PCB set adoption in practice have been explored in pre-implementation analyses, indicating all stakeholders recognized the relevance and potential benefits of PROM/PREM [8, 22]. At the same time, stakeholders acknowledged important efforts yet to be made, e.g., embedding PROM/PREM into service processes or informing care professionals and patients about their purpose.

Recently, the patient-reported measures of the PCB set were implemented in three obstetric care networks (OCN) in the Netherlands, that aimed to use these PROM/PREM for two levels of VHBC: individual scores to guide patient-specific care decisions and group-level results in to improve quality of care. This implementation process was designed following the principles of action research to enhance practice change and, concurrently, gain knowledge about PROM/PREM implementation in the context of care networks. Guided by theoretical frameworks for implementation, this study aimed to evaluate (1) the outcomes of PROM/PREM implementation in obstetric care networks and (2) the implementation processes that influence these outcomes to increase our understanding of this complex implementation, its practice challenges, and underlying change mechanisms.

Methods

Design

This mixed-methods study was conducted between December 2019 and June 2022 as part of an action research project aimed at PROM/PREM implementation in clinical practice and QI processes of OCN. Action research aims to both change practice and develop knowledge about that change via a cyclic design of action, data generation and reflection, while involving all stakeholders in research and practice change [23]. Action research is particularly useful to implement a complex intervention that needs adjustment to the local context, as detailed data are generated on both the implementation activities (what it involved) and change mechanisms (how it worked). This way, the outcomes achieved can be explained for, increasing the transferability of findings [24]. To understand the change mechanisms underlying the complex implementation of PROM/PREM, the use of multilevel implementation frameworks and theories has been recommended by scoping literature [10, 25]. To evaluate PROM/PREM implementation in the context of care networks, this study combinedly used Proctor’s taxonomy for implementation outcomes [26] and the Normalization Process Theory (NPT) [27]. Proctor’s taxonomy describes the outcomes of different stages in implementation, whereas the NPT describes implementation processes in terms of what care professionals (don’t) do to embed a new way of working in routine practice and is distinct in proposing mechanisms for sustained uptake. Proctor and NPT guided the collection and analysis of both qualitative and quantitative data within the mixed-methods design, increasing both the depth and transferability of our findings.

Setting and participants

Dutch perinatal care is provided interdisciplinary from two healthcare tiers: primary care by community midwives and maternity care organizations; and secondary/tertiary care by hospital employed care professionals. Hospitals, regional community midwife practices and maternity care organizations increasingly cooperate in OCN to provide continuity of care across pregnancy, childbirth and puerperium. In 2019, PROM/PREM implementation was initiated from a regional collaborative between ten OCN in the middle of the Netherlands, of which three OCN participated. In each OCN, the hospital and 2–4 midwifery practices implemented individual-level PROM/PREM in clinic. All other professionals working in the OCN (e.g., from other midwifery practices, maternity care organizations, youth care) could join network-broad QI with group-level outcomes. Each OCN had an interdisciplinary team in charge of implementation (including, at least one obstetrician, clinical midwife, and community midwife from each participating midwifery practice), of which one was appointed project leader. In this study, participants were defined as (1) professionals directly involved in implementation: project team members (key participants) or obstetricians/midwives using individual-level PROM/PREM, and (2) indirectly involved professionals: from other OCN-organizations or discipline, such as nurses. Patients were involved in implementation as they completed PROM/PREM for routine care but did not actively participate in this evaluation study. As patients had participated in our pre-implementation analysis and feasibility pilot [8, 28], their needs were incorporated in the initial implementation strategy.

Action research project

The PROM/PREM implemented in this project were those proposed in the PCB set: questionnaires at two moments during pregnancy (T1: first trimester, T2: early third trimester) and three postpartum (T3: maternity week, T4: 6 weeks postpartum, T5: 6 months postpartum). The PCB set was developed internationally and subsequently translated to the Dutch setting, both phases involving all stakeholders, including care professionals and patients [18, 29]. An overview of the PCB set’s patient-reported domains and timeline for completion is provided in Additional file 1: Fig. S1. The set’s PROM/PREM were implemented for two purposes. First, individual-level PROM/PREM were implemented in clinic: reviewing N = 1 scores with patients during a regular care contact after completing a questionnaire. The timeline of collection, workflow, and follow-up services (including scoring and alert values) were organized as described in the national pilot project [30]. Second, the same PROM/PREM outcomes would be used at group-level in network-broad QI sessions. Despite the complexity of combining these purposes, findings in our pre-implementation research amongst care professionals, patients and other stakeholders in perinatal care suggested both goals could also reinforce each other [8]. Direct usability in clinical practice could, for instance, motivate care professionals and patients to comply, thereby generating data for group-level use (and vice-versa). Likewise, other previous findings from our pre-implementation analysis and feasibility pilot [8, 28], were used to design the initial implementation strategy. Important elements for individual-level use included visual alerts to support care professionals in interpreting the answers and offering patients a choice whether their care professional had insight in their individual PREM answers. During the action research project, this initial implementation strategy (Fig. 1) was continuously refined guided by action research principles in iterative cycles of planning and executing implementation activities, data generation, and reflection on these data to refine subsequent activities. These cycles were conducted jointly by researchers and care professionals. The researchers developed the baseline strategy for project organization and education (e.g. identified possible IT-systems, developed an e-learning and kick-off meeting), provided materials and support for its execution (e.g. patient information folder, for working protocol for care professionals), and facilitated data generation for its refinement (e.g. organized focus groups, sent out the survey). The project teams designed and coordinated local implementation (e.g. adapt instruction material to local workflow, chose the IT system that best fitted local needs and resources) and participated in data generation and reflections (e.g. survey results were discussed in project team meetings, participation in focus groups). Three OCN started implementation sequentially to be able to learn from previous experiences, exchanged via the researchers and directly between care professionals from different OCN. After the one-year implementation period, project teams reported their experiences to their OCN and advised future steps in an end-evaluation.

Fig. 1

Timeline of implementation and data generation activities. PROM, patient-reported outcome measure. PREM, patient-reported experience measure. QI, quality improvement. OCN, obstetric care network. CP, care professional. VHBC, value-based healthcare

Outcome measures

First, implementation outcomes were assessed using Proctor’s taxonomy of implementation outcomes. Inspired by the translation to PROM/PREM specific implementation outcomes by Stover et al. [10], implementation outcomes and the indicators to assess them were defined for this study’s context (Table 1). These indicators were evaluated with survey items of the Measurement Instrument for Determinants of Innovations (MIDI), via administrative data and embedded in qualitative methods such as observation checklists. The MIDI was developed to identify factors influencing the use of an implemented intervention by measuring determinants in innovation, user, organization and socio-political context [31]. As recommended by its developers, a selection of items was made based on relevance for our context. Second, implementation processes were evaluated along the NPT, which describes four core mechanisms towards normalization. These mechanisms and their subconstructs were measured trough the validated Normalization Measurement Development (NoMAD) instrument [32, 33], and were included in the survey and qualitative methods (Additional file 1: Table S1). The complete survey administered to care professionals consisted of validated NoMAD and MIDI items, completed with three extra questions (about education used, knowledge level, needs in implementation) based on our feasibility pilot and PROM/PREM specific implementation literature [28, 34]. All survey questions and details about scoring are provided in Additional file 1: Table S2.

Table 1 Implementation outcomes and their assessment

Data generation

A timeline of data generation is provided in Fig. 1. For quantitative data, the survey was sent to all OCN care professionals at the start and end of implementation by e-mail. Care professionals indirectly involved in implementation were led to a short version. Demographics were collected on profession and working experience. This way, the survey explored implementation processes in a broad group of care professionals, which was used to solidify qualitative findings and to guide reflection on the implementation process and needs with participants during qualitative methods. Qualitative data were generated through focus group discussions, observations, reflections and naturally occurring data. At each kick-off session, group discussion was organized using photovoice (i.e. a method to empower all participants to share their perspectives [35]), of which notes were taken for the observation report. Along the QI sessions, traditional focus group discussions were led by two researchers (AD, AK) along statements about implementation based on outcome indicators and NPT subconstructs (Additional file 1: Table S3). For each focus group, a selection of these statements was made to address specific gaps in data generation emerging from collective iterative reflections and quantitative results from the survey. After informed consent, focus groups were recorded and transcribed ad verbatim. During the whole implementation, two researchers (AD, ML) conducted participative observations in all meetings and kept a reflection logbook, both structured along the theoretical frameworks. Considered as naturally occurring data [36], all documents emerging during the implementation process were gathered (e.g., meeting reports), containing administrative data too (e.g., IT system data on costs, professionals with account).

Data analysis

Quantitative survey data were analyzed in R version 4.0.2 [37]. Mean scores were calculated for items consisting of multiple statements and multiple items measuring a subconstruct. Frequencies of responses to items were visualized in stacked-bar diagrams to gain insights in the diversity of opinions. All qualitative data (i.e., open-ended survey answers, transcriptions, observation reports, reflection logbook, documents) were thematically analyzed in Microsoft Excel version 16.61 conform QUAGOL guidelines, combining a deductive and inductive approach [38]. The researchers assigned codes from the conceptual frameworks (Proctor and NPT) as well as open codes describing themes within their concepts. At start, three researchers (AD, BP, ML) coded three documents independently, and discussed the resulting codes to develop a mature coding scheme. Data were then analyzed by AD until saturation was reached, after which four researchers (AD, ML, BP, MB) reviewed and discussed the codes to establish final interpretations. Quantitative and qualitative data were then triangulated by exploring (dis)agreements and silences between both datasets. This was conducted by a single researcher (AD) identifying items and subconstructs in the quantitative dataset demonstrating particularly high or low survey scores, to compare these against qualitative themes and discuss that among the research team. In this process, quantitative data were used to solidify quantitative findings in a broader group of professionals and over time.

Results

Overall, 159 surveys were returned, of which 63 (39%) in phase 1 and 97 (61%) in phase 3. Five focus groups were held with, in total, 78 care professionals attending QI sessions. Other data (from observations, reflections, documents) were generated along 39 project team meetings, 3 kick-off sessions, 5 QI sessions, and the logbook. Participants’ characteristics for the survey and focus groups are presented in Table 2. Of survey respondents, 62% (99/159) was directly involved in implementation (i.e., project team member or using individual-level PROM/PREM). Mean survey scores were largely in agreement with qualitative themes, thus strengthening each other, and are together presented per theoretical framework below. Full response frequencies per survey item are provided in Additional file 1: Fig. S2.

Table 2 Baseline characteristics survey and focus group participants

Implementation outcomes

Below, Proctor’s outcomes as defined in Table 1 are provided along our most important findings.

Motivations and objectives

At the start, potential benefits of PROM/PREM were recognized by most care professionals, contributing to acceptability and adoption. Care professionals expected that individual-level PROM/PREM would assist them in recognizing symptoms and identifying topics important to their patient and empower patients to prepare visits and raise issues. Moreover, care professionals expressed enthusiasm for using group-level PROM/PREM for patient-centered quality improvement. Patients’ opinions were care professionals’ main motivation to comply and 54% (46/85) of survey respondents expected their cooperation, whereas 11% (9/85) did not.

Experienced benefits

According to care professionals, system-wide PROM/PREM capture and use facilitated their patient-centered goals and vision, expressing good appropriateness. In consultations, several care professionals felt supported by PROM/PREM results to identify and discuss patients’ issues, sometimes leading to richer conversations and/or appropriate referrals. From the group-level PROM/PREM data in QI sessions, care professionals gained valuable insights and directions for improvement in their patients’ wellbeing and experiences, which contributed to their work pleasure. In practice, the PROM/PREM content was considered appropriate for most of their patients, except for non-Dutch speaking women and those with low health literacy, who care professionals hesitated to invite for that reason. Also, some adaptations to PROM/PREM content were suggested, such as open answer options to enable personalized care even more.

Experienced barriers

Whilst most care professionals strongly favored integration in their electronic health record (EHR), the locally explored IT options either could not function across different EHRs, or their costs to realize that were too high. Hence, in each OCN, an affordable start-up IT system without EHR integration was chosen, that promised automated data capture, visualization for care professionals, network communication and privacy. This IT system enabled PROM/PREM adoption in all participating practices per OCN but became the main barrier for acceptability, feasibility, and further implementation. Care professionals did not consider it to be user-friendly (complicated access, frequent issues and bugs, poor overview, not visible whether responses had been discussed and unable to connect PROM/PREM measurements to visits) and felt increasingly frustrated by the IT supplier’s slow pace, and sometimes inability, to solve issues. Although their patients often appeared willing to complete PROM/PREM, IT was considered a major barrier for patients too, due to poor accessibility and bugs, leading to privacy concerns too. Other patient barriers mentioned were a lack of motivation or time (especially postpartum) and misunderstanding of the purpose.

Additionally, the high time investment for care professionals negatively influenced acceptability (44% of care professionals (60/135) expected it would take too much time), appropriateness (for their high current workload) and feasibility (of workflow integration). Factors contributing to a high time investment in practice included the administrative burden of the non-integrated IT system, instructing patients, reviewing PROM/PREM results, and learning a new skill.

Costs

The IT systems’ costs and care professionals’ time investment (i.e., project team efforts and using PROM/PREM in practice) were the main drivers for implementation costs. In two of three OCN, these costs demanded external funding (used for the IT system and project leader allocation); the third OCN could finance them from a joint reimbursement structure.

Fidelity and penetration

The process of creating an account for the external IT system, inviting patients, and discussing individual PROM/PREM responses required continuous support from project teams and action researchers to reach fidelity and penetration in participating practices. At start, project teams decided to begin with a selection of patient groups, measurement moments (all selected T1–T4) and care professionals. Eventually, most targeted care professionals created an IT system account, but only few actively invited patients: others often missed eligible patients due to time constraints and low exposure resulting from the patient group selection. Half of the invited patients created an account and completed PROM/PREM; postpartum response rates were lower. Based on experiences shared in project team meetings, almost all completed PROM/PREM were discussed in the next visit, except in case of IT bugs or care transitions in the maternity week (T3). Regarding group-level PROM/PREM use, five QI sessions were carried out during the pilot periods. Reflecting good fidelity, local care professionals actively participated in preparation, presentation, and elaboration of these sessions, which were attended by an average of 17 (range 11–25) care professionals representing all participating disciplines. Unlike in-clinic PROM/PREM use, QI sessions extended penetration to care professionals without direct involvement in implementation.

Sustainability

Except for one community midwifery practice that sustainably integrated PROM/PROM, routine PROM/PREM administration was stopped in all OCN after the one-year implementation period. However, all OCN intended to continue the QI sessions with data available in the OCN and, after EHR integration, reinitiate PROM/PREM capture and use. After the decision to stop, the second QI session in one OCN was not conducted, because the project team expected it would be of more benefit to a future restart.

Implementation process

The complete NPT framework analysis is listed in Table 3 with supportive qualitative and quantitative data (mean survey scores on a 5-point Likert scale) per subconstruct. Per core mechanism, subconstructs contributing most to (un)successful implementation outcomes are elaborated on below. Overall, main processes contributing to implementation positively were internalization (understand value) and initiation (drive by key-participants), whereas relational integration (maintain confidence) and reconfiguration (attempts to redefine) affected implementation negatively.

Table 3 Framework analysis for the NPT subconstructs with supportive data

Coherence: sense-making

As terminology like PROM/PREM and VBHC often appeared abstract at the start, hearing experiences directly from participants of earlier regions helped to gain understanding of practical aspects. This was arranged both across and within OCN enhancing differentiation and individual specification. Care professionals early recognizing the potential benefits of PROM/PREM contributed to internalization and willingness for implementation. The ability to incorporate patients’ voice in QI appeared their main driver, so they were enthused by the QI sessions. Although some experienced that individual-level PROM/PREM supported time-efficiency and personalized care by discussing important rather than all topics, care professionals felt they needed more exposure to these benefits for sustained internalization.

Cognitive participation: relational work

Formally appointed local project leaders mainly drove initiation, particularly if this was a clinician from a participating practice with OCN management support (both in resources and vision). Project team members representing each participating practice and discipline could engage colleagues, reflect on practical challenges, and establish possible solutions. Initiation by key-participants was facilitated by action researchers’ activities (e.g., share experiences and materials, participate in identifying and solving issues, practical support) and by the one-year implementation period, making them feel able to try PROM/PREM without being ‘stuck’ to them. Whether local key-participants drove initiation or relied on the action researcher, depended on the level of ownership felt by local project teams. In-clinic support from key-participants and action researchers was most important for enrolment of other care professionals, since training reached a minority: 22% (22/99) of survey respondents had used support or training. Enrolment was harder in large practices, as care professionals felt less influence on the decision (or had little knowledge of the reasons) to participate. Care professionals differed in their feeling of PROM/PREM being a legitimate part of their role, which could be supported by positive practice experiences or those of colleagues. Additionally, enrolment and legitimation appeared to improve by the QI sessions, where valuable interprofessional conversations led to concrete improvement actions. However, care professionals’ positive expectations and involvement decreased over time by enduring IT issues and low exposure to benefits. At the end, (key) participants kept support for the potentials of PROM/PREM for VBHC and formulated future strategies for sustainable activation.

Collective action: operational work

Discussed in 92% of project team meetings (36/39), feasibility issues dominated the implementation process and impaired workflow integration (i.e., interactional workability). Key-participants’ and action researchers’ time and efforts mainly went into getting the IT system working and supporting users (care professionals and patients) in operational work. Project teams experienced a vicious circle of poor-usable IT and not building up workflow routine: their attempts to increase routine, like expanding patient groups, were withheld by IT issues and concurrent time investment. The IT system affected participants’ confidence in the innovation (i.e., relational integration), especially the inability to improve or solve issues in time. Also, reliability of PROM/PREM results was questioned, because care professionals experienced varying clinical relevance of alerts, inappropriate timing, unsuitable answer options and, at group-level, numbers were too small. Most care professionals expressed confidence about discussing PROM/PREM, but the challenging part of skill set workability was allocating all tasks appropriately, for example ensuring that individual-level PROM/PREM were discussed across participating practices. To solve this, allocating a principal care provider to discuss PROM/PREM was opted by care professionals, both to keep overview of which responses had been discussed, as to gain most value from that conversation in a trusted relationship.

Reflexive monitoring: appraisal work

Facilitated by action researchers, project teams continuously reflected on (systemization) and tried to refine (reconfiguration) processes to improve implementation, like standard phrases to report PROM/PREM conversations to decrease administration burden. Reconfiguration was easier for smaller practices, such as temporarily collect T3 (maternity week) on paper to increase response rates. However, limited adaptability was experienced for several reasons: IT suppliers’ inability to improve, time constraints and the PCB set’s international origin. Key-participants’ evaluation reports stated reconfigurations needed for future restart and sustained implementation. For individual use, PROM/PREM should be easily accessible for patients and professionals, with EHR-integration across the network. For QI with group-level data, essential aspects were data analysis and visualization (provided by the researchers during the action research project) and linking PROM/PREM to clinical outcomes.

Discussion

In this mixed-method evaluation of PROM/PREM implementation in the context of care networks, the use of PROM/PREM was found to be acceptable and appropriate but not feasible in daily practice, mainly due to IT issues and time constraints. Hence PROM/PREM implementation did not sustain, but their potentials for VBHC fitted professionals’ motivation and strategies for their future adoption were formulated in all OCN. In line with previous evidence [10, 11], our findings affirm the value of individual-level PROM/PREM for clinical care perceived by professionals and emphasize the need for workflow integration. Based on participants’ and researchers’ reflections on the re-adjusted, co-created implementation strategy, recommendations for PROM/PREM implementation across care networks were formulated in end-evaluations and summarized in Table 4. To embed these recommendations, an iterative approach is key to adjust to local context.

Table 4 Recommendations for PROM/PREM implementation across care networks

Despite tailoring the strategy to our pre-implementation analysis amongst patients and care professionals and further adaption of implementation activities during each action research cycle [8, 28], the feasibility of integrating PROM in practice was lower than expected, largely explainable by poor usability of the IT system chosen at start. Of the numerous PROM/PREM capture systems developed in the past years, most were designed for single center settings or group-level, anonymous use only [11, 39, 40]. Besides healthcare systems with a shared EHR [41], successful system-wide PROM collection with direct visualization for individual-level use in clinic has proven challenging to realize and was only recently described and developed in a Welsh national program [42]. To support PROM/PREM implementation and network collaboration for patient-centered care, there is a need for PROM/PREM integration into EHRs and, moreover, infrastructures for cross-EHR data exchange [43]. Structural financial support for their development and governance should be explored, as most network collaborations are temporarily funded which undermines adoption, feasibility, and sustainability [4, 11, 44].

Previous PROM/PREM implementation strategies, both at the individual and group level, often emphasize the selection of PROMs and the challenge of involving care professionals [5, 11, 45]. Although we acknowledge their importance, most care professionals in our study already demonstrated a positive attitude towards PROM/PREM at start, reflected in good coherence and cognitive participation and consistent with previous findings [22, 41]. They were keen to learn from previous experiences and motivated by the prospective of patient-centered QI with group-level PROM/PREM, which fueled their efforts for individual-level implementation as well. In the current healthcare landscape with professional shortage and high turnover, care professionals’ work pleasure might be one of the most valuable benefits of PROM/PREM [46, 47]. Despite feasibility challenges and IT issues, key participants’ threshold to adopt such complex implementation was lowered by the iterative approach that gave space to ‘try out’ and adapt to local context, which enabled them to get acquainted with PROM/PREM and their potential for VBHC. Concurrently, other care professionals felt demotivated and overruled by management when unaware of the reasons to participate in such implementation and driving their workload even higher. So new initiatives should be carefully selected and coordinated across care networks, where an iterative and participative approach to implementation can provide space for early adopters’ energy, sharing practice experiences to engage others, and fine-tuning to local context.

The integrated care context affected implementation not only by challenges in IT infrastructure, fragmented leadership and allocation of costs, but also in consistency of discussing individual-level PROM/PREM results across care transitions. To ensure that individual-level results were discussed, care professionals opted to allocate a principal care provider, arguing that a conversation about the topics would gain most value in a trusted relationship, similar to a solution to improve continuity of perinatal care in general [48]. However, the issues arising from network-broad implementation are lacking in current PROM/PREM implementation frameworks and strategies [9, 10]. Further research within real-life projects should identify and address barriers and enablers for innovation across organizational boundaries. That way, innovations can improve value of care for individuals and overall care performance from patients’ perspective.

Reflecting on the action researchers’ role, many similarities were seen with the facilitator role described by Roberts in the iPHARIS framework [49]. Similar to their findings, our action researcher was a crucial enabler for implementation, providing an external view with expert knowledge to identify and solve emerging issues in practice, especially in collaboration with the local project leader. Additionally, participating in all regions resulted in overview, expertise and sharing previous lessons in new regions. However, the tension between guidance in problem solving and doing the work to fit local workflow was present in our projects as well: in some regions, the PROM/PREM workflow never became completely independent of the action researcher. Across OCN, the level of implementation ownership of the project teams varied, which could partly be explained by existing collaboration mechanisms and integrated reimbursement in some OCN.

As called for in recent literature [10, 16], this study substantially contributes to the understanding of care professionals’ real-life experiences and challenges for PROM/PREM implementation, specifically addressing the integrated care context in a realistic range of collaborating organizations. In the mixed-methods design, consistency in data from different sources and methods strengthened our findings. Also, our data collection and analyses were supported by widely used implementation science theories and their validated instruments. The iterative, participatory action research approach enabled in-depth understanding of implementation activities, processes and outcomes, which contributes to the transferability of findings. An important limitation of our study was that we did not invite patients to the evaluation of the implementation process and outcomes, except indirectly via care professionals. We did explore patients’ experiences with individual-level use in another study along a national pilot with the PCB set [50], while the current project focused on the (organizational challenges of) implementation. In next action cycles, patients should be certainly involved. Here, special attention should go to women with low health-literacy and language barriers, who are prone to be neglected by PROM/PREM, to prevent existing health inequities becoming even larger [51]. Besides providing digital support and translating questionnaires, solutions to involve these women should be sought outside the idea of questionnaire completion. In thinking of solutions, research methods should be embraced that centralize patients and local opportunities (e.g. linkage to primary care, community-based solutions) [52, 53]. Another limitation of our study is that the IT-system used appeared such a major barrier to implementation, that other factors might have been undervalued. Selection bias of both early adopter OCN and professionals is likely to have enhanced a positive attitude towards PROM/PREM. We attempted to reach professionals broader by inviting the whole OCN for QI sessions and the survey, which had a short version for indirectly involved professionals. Lastly, the COVID-19 outbreak has probably influenced care professionals’ willingness and ability to adopt a new way of working, affected implementation planning (e.g., paused, postponed) and restricted study activities to online contacts with minimal field work.

Conclusion

Although implementation did not sustain, network-broad PROM/PREM use in clinic and for QI matched professionals’ motivation for patient-centered care. This study provides recommendations to implement PROM/PREM meaningfully in practice, in ways that support professionals in their drive towards patient-centered care by efficient, person-centered assessment of patients’ wellbeing. For PROM/PREM to fulfill their potential for VBHC, our work highlights the need for sustainably funded technology infrastructures that communicate across healthcare tiers, as well as an iterative and participative approach to refine their complex implementation to local contexts.