Project Background

Toolkit methods

This toolkit was developed through work funded by the Agency for Healthcare Research and Quality (AHRQ) as part of its Digital Healthcare Research Program. The toolkit builds on over 10 years of experience at the University of Washington (UW) implementing ePROs for research, clinical care, and quality improvement. In 2015, the UW Medicine system, our team’s primary setting, launched a large-scale practice transformation initiative that prioritized better capture and use of the patient voice in clinical care. This initiative provided an environment in which to observe and learn about the processes of integrating ePROs into clinical practice.

The activities and methods that led to the development of this toolkit were grounded in action research theory (Austin et al, 2020) and marked by iterative, participatory forms of data collection and knowledge generation, as described in Table 1A. The information and recommendations made in this toolkit are meant to blend both peer-reviewed and real world evidence, providing tangible lessons that can be applied in complex healthcare environments.

Table 1A: Description of project methods for design guideline development
Phase Goal Sample Activities
Planning Understand current evidence; engage stakeholders in inquiry & problem identification
  • Ongoing analysis of peer-reviewed literature
  • Qualitative synthesis of PRO literature (2010–2015)
  • Community-building activities
  • Development of ePRO workshops and education
Acting Gain real world experience with ePRO governance, integration, and reporting practices
  • Participation and leadership on ePRO governance committees
  • Involvement in ePRO project design, workflow modeling, and implementation
Observing Gather qualitative and quantitative data to evaluate ePRO processes and outcomes
  • Participant observation
  • Interviews / Focus groups
  • ePRO implementation data (e.g. usage metrics, outcome data, etc.)
Reflecting Triangulate learnings across stakeholders and contexts
  • Stakeholder engagement activities
  • National conference presentations (e.g. AcademyHealth, American Medical Informatics Association, International Society for Quality of Life Research)
  • Coordination of educational workshops (local, national)