Key concepts and background

PRO measures are generally scored—whether manually or by an automated system—using a calculation based on validated algorithms. However, simply reporting the patient’s score to the provider is not often sufficient to fully support the patient care process (e.g., shared decision-making, ongoing condition monitoring).

This section provides an in-depth look at how ePRO reports can be constructed leveraging various types of ePRO content, automation, and visual presentation; this approach can increase the clinical usefulness of ePROs by examining the ways in which they can be transformed, appended, aggregated, and distilled. This section also recognizes that reporting consists of both report construction as well as meaningful, effective use.

Need for guidelines

Robust literature exists to guide the design of reports and visualizations in other domains, but available models and guidelines do not adequately address considerations specific to the unique nature of electronic patient-reported outcome (ePRO) reporting in clinical practice. Stakeholder-informed guidelines regarding critical design considerations for ePRO reports and the use of these visualizations can support successful clinical integration of ePRO data. Such guidelines can assist both developers and implementation teams in key decisions regarding the content, visualizations, and functions associated with ePRO reports and their use across a healthcare system.

While a moderate degree of organizational customization may be needed for a system-wide deployment of ePROs in order to meet clinician expectations of utility and ease of use, some general considerations for ePRO reporting exist. The guidelines in this section provide these general considerations.

ePRO reporting to providers

As the collection of ePROs increases across healthcare settings, so does the need to develop reports that incorporate effective mechanisms (e.g., visualizations) for bringing ePRO data to providers. Many providers perceive an ePRO system as a potentially valuable addition to augment their patient care process if it can meet their preferences for report design and if the ePRO workflow is streamlined.

Given the complex requirements for ePRO capture and use, ePRO systems must consider how to analyze and deliver actionable insights in the clinical context for a variety of audiences and purposes, including individual patient care and decision-making, clinical quality improvement, and population health.

This section takes the perspective of human-centered design of ePRO reports and their use, particularly use by front line providers for clinical care purposes. The section includes design considerations for meeting requirements across diverse user groups of providers (e.g., different clinical specialties) and considerations for integrating ePRO data with electronic health record (EHR) data to provide meaningful reports that support the patient care process and quality of care. While we focus on providers as front-line users of ePROs adopted by health systems, we encourage readers to review the design guidelines with an eye towards what may be useful and applicable to their efforts towards a human-centered approach to providing reports directly to patients.

What is ePRO reporting?

ePRO reporting refers to the way in which PRO data is packaged electronically and used by providers. In an ePRO system, the patient enters the information into the system by providing answers to items from one or more PRO measures. In order to make this patient-entered data clinically actionable, it must then be reported to the patient’s provider(s) in a clear and relevant way for providers to efficiently and accurately understand the reports.

But, the goals of ePRO reporting do not stop there. The ultimate goals of integrating ePRO reports into clinical care (i.e., into a patient encounter) involve enhancing existing knowledge and understanding with ePROs to move toward the wisdom. Wisdom in the use of ePROs includes appropriate integration of ePROs into assessments of patient status, communications with patients in a shared decision-making process, and care coordination efforts.

Therefore, an ideal reporting process will progress from the collection of PRO data to wisdom gained from using ePRO reports effectively in decision-making and improving the care process, as outlined in Figure 4A and further described here:

• Data enters the system as patient scores of patient-submitted PRO measures.
• As data accrues, for example, by repeated collection of ePROs over time, a broader depth of information is available for interpretation. In an ePRO system, these steps may be accomplished through the system itself or entered in the system following manual collection of PRO measures from patients.
• The information can then be transformed and synthesized in various formats supported by the ePRO system in order to glean knowledge about patient condition. For example, a functional status measure collected over time following a procedure can be reported on a longitudinal graph that visually depicts each ePRO (see the Longitudinal Information guideline for details about longitudinal information on ePRO reports). The graph can also be annotated with other salient data (e.g., patient comorbidities and medications) that adds further relevant clinical context to PRO information (see Contextualize Data guideline to learn about adding contextual information to ePRO reports).
• Once the ePRO report is available for review, the provider can interpret the findings of the report (i.e., knowledge) that are most salient to patient care, thereby gaining intelligence.
• Finally, intelligence gained can be put to action in the care process as wisdom through application in medical decision-making and communication with the patient.

Enactment of this flow in patient encounters embodies the ePRO value proposition in the clinical care process.

A User-Centered Design approach to reporting

The provider is an important lead and primary ePRO report user in enacting the described ePRO reporting flow. Effective design for ePRO reporting considers report content and presentation along with the users’ mental model (i.e., a person’s thought process about how they may use the report in the real world) affecting their interaction with the electronic reports and situations of report use. A user-centered design (UCD) approach to designing ePRO reports to support effective human computer interaction acknowledges content, situation, and the users’ mental model by:

• placing the user at the center of the design
• focusing on users and their tasks early in the design process
• measuring usability empirically
• supporting an iterative approach, whereby a product is designed, evaluated, and modified with real users repeatedly in quick successive iterations

Health systems that employ a UCD approach to ePRO reporting facilitate the likelihood of well-designed reports, engagement from providers in the implementation process, and successful adoption and use of ePROs. The methods used to inform our guidelines and the guidelines themselves reflect a user centered focus on the provider using ePROs to serve and engage patients in the care process. For additional insight on introducing user-centered design best practices (including developing user personas) into ePRO reports, see the Tools and Resources section related to UCD.

Methods informing this section

As part of the larger ePRO action research effort described in the introduction to the design guidelines, this section was particularly informed by literature review (published literature on ePRO implementations) and in-depth interviews to arrive at insights regarding ePRO reporting. Twenty (20) in-depth interviews were conducted with providers (representing 11 medical and surgical specialties) at a large academic medical center with experience in using PROMs. Drawing on a UCD approach, semi-structured interview guides explored the providers’ design needs and preferences for ePRO reporting and the use of PRO data, as well as for issues related to reporting and visualizations identified through the literature review.

Our analysis of both the literature and the provider interviews identified recurring themes regarding issues and considerations critical to the usefulness and usability of ePRO tools by providers.

The Reporting Guidelines: What to Expect

Our analysis, as well as insights gathered from more generalized human-computer interaction and visualization domains, contributed to the development of a set of guidelines which highlight key commonalities where standardization is possible for ePRO reporting, as well as situational nuances for which flexibility may be needed in the design of ePRO reports. As with much user-centered design work, these guidelines recognize that reporting is both a noun (report) and verb (use of the reports) as represented in Figure 4B.

Our analysis of the literature and interviews resulted in identification of 14 guidelines which span three key domains directly related to ePRO report design (Figure 4C). These are data and information (i.e., what type of data is provided), system functions and interactions (i.e., which capabilities are provided), and presentation (i.e., how data is visually represented). We provide a final guideline in this section related to ePRO report use that specifically focuses on effective access and interaction with the report in the context of a care encounter. Regarding the efforts needed to ready the provider and context to support appropriate and effective use of ePRO reports (e.g., training), we refer the reader to the Integration section.

We provide multiple tables and graphs in this section to help the reader conceptualize the application of the ePRO Reporting Guidelines in ePRO reports. These graphs and tables convey the general nature of ePRO visualizations. The graphs and tables are not actual screen captures from ePRO reporting tools, nor do they represent actual PRO data.

The following tools (found in the Tools and Resources section) can help guide the application of the Reporting guidelines contained in this section. These tools can facilitate the discussion, direction and design of ePRO reports:

• ePRO functional requirements assessment for system design
• ePRO Clinic report design checklist
• Visualization library of graphs

In addition to the references cited throughout this section, we have provided an additional Supplemental Bibliography of resources that informed this work and may be useful to readers.