Recommended resources

Before implementing any intervention or workflow into clinical practice, it is critical to clarify the goals and approach and to assess the resources available.

This guide, developed by ISOQOL, provides different options for how to select PROM measures, as well as guidance on data collection and reporting for clinical practice. The purpose of this user's guide is to help clinicians who are interested in using PROMs in their clinical practice as a tool in patient management. A companion resource summarizes use cases gathered through stakeholder interviews into the dimensions outlined in the main user's guide.

Governance

• International Society for Quality of Life Research (prepared by Aaronson N, Elliott T, Greenhalgh J, Halyard M, Hess R, Miller D, Reeve B, Santana M, Snyder C). User’s Guide to Implementing Patient-Reported Outcomes Assessment in Clinical Practice, Version: January 2015.
• International Society for Quality of Life Research (prepared by Chan E, Edwards T, Haywood K, Mikles S, Newton L). Companion Guide to Implementing Patient Reported Outcomes Assessment in Clinical Practice, Version: February 2018.

ePROs can be reported and used in the context of a patient’s other health data (e.g., laboratory reports, imaging studies, clinic notes) to promote patient-centered care. This guide provides recommendations for integrating ePROs into electronic health records, thus enabling use of PRO data for multiple clinical, research, and administrative applications, and thereby promoting patient-centered care. A multidisciplinary workgroup group led by the Patient-Centered Outcomes Research Institute (PCORI) developed this guide, which builds on the ISOQOL user’s guide.

Governance, Integration

• Snyder C, and Wu, A.W., eds. Users’ Guide to Integrating Patient-Reported Outcomes in Electronic Health Records. Baltimore, MD: Johns Hopkins University. 2017. Funded by Patient-Centered Outcomes Research Institute (PCORI); JHU Contract No. 10.01.14 TO2 08.01.15.

The ability to scale PROs across healthcare systems has been limited by knowledge gaps around how to manage the diversity of PRO uses and leverage health information technology. This paper reports learnings and practice insights from UW Medicine’s practice transformation efforts to incorporate the patient voice into multiple areas of care. This paper explores three PRO use cases (preventive care, chronic care management, and intervention assessment).

Governance, Integration

Seeking a range of perspectives and expertise in each phase of the ePRO implementation process helps ensure that stakeholder needs are identified and addressed. Many methodologies exist for planning and carrying out stakeholder engagement strategies. These resources provide information on stakeholder engagement, along with tools to build a stakeholder engagement plan.

Stakeholders, Governance, Reporting, Integration

• Institute for Patient- and Family-Centered Care - A Toolbox for Creating Sustainable Partnerships with Patients and Families in Research
• AHRQ Learning Modules: Engaging Stakeholders in the Effective Health Care Program
• Stover, A. M., Tompkins Stricker, C., Hammelef, K., Henson, S., Carr, P., Jansen, J., Deal, A. M., Bennett, A. V., & Basch, E. M. (2019). Using stakeholder engagement to overcome barriers to implementing patient-reported outcomes (Pros) in cancer care delivery: Approaches from 3 prospective studies. Medical Care, 57, S92. 

Introducing the integration of new technology into a healthcare setting is difficult and requires significant workflow planning and redesign to be successful. These resources provide basic information on workflow concepts, experiences of other organizations involved in health IT implementations, and practical tools for workflow assessment.

Integration

• HIMSS Toolkit - Workflow Redesign in Support of the Use of Information Technology within Healthcare
• AHRQ Workflow Assessment for Health IT Toolkit
• SADT: Marca, D., & McGowan, Clement L. (1988). SADT: Structured analysis and design technique. New York: McGraw-Hill.

There are many approaches to using technology for ePRO data collection and application. Once your health system decides on an approach, these resources can provide further information on technical specifications and considerations for technical integration within electronic health records or other technical platforms.

Integration, Governance

• Electronic Health Record Access to Seamless Integration of Patient-Reported Outcomes (EASIPRO)
• Patient Reported Outcomes FHIR Implementation Guide

Implementation is complex, and various models exist that support the applied implementation process, as well as formal evaluation. Relevant to ePROs are learning health system models that support real-world, complex interventions. The following resources are commonly used in healthcare delivery and research and may be dependent on preferences.

Integration, Governance

• Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and policy in mental health, 38(2), 65–76.
• Damschroder, L.J., Aron, D.C., Keith, R.E., Kirsh, S.R., Alexander, J.A. Lowery, J.C. (2009). Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science, 4: 50.
• Stover, A. M., Haverman, L., van Oers, H. A., Greenhalgh, J., Potter, C. M., Ahmed, S., Greenhalgh, J., Gibbons, E., Haverman, L., Manalili, K., Potter, C., Roberts, N., Santana, M., Stover, A. M., van Oers, H., & On behalf of the ISOQOL PROMs/PREMs in Clinical Practice Implementation Science Work Group. (2020). Using an implementation science approach to implement and evaluate patient-reported outcome measures (Prom) initiatives in routine care settings. Quality of Life Research.
• Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, Hinder S, Fahy N, Procter R, Shaw S. Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies. J Med Internet Res 2017;19(11):e367.
• Cusack CM, Byrne C, Hook JM, McGowan J, Poon EG, Zafar A. Health Information Technology Evaluation Toolkit: 2009 Update (Prepared for the AHRQ National Resource Center for Health Information Technology under Contract No. 290-04-0016.) AHRQ Publication No. 09-0083-EF. Rockville, MD: Agency for Healthcare Research and Quality. June 2009.
• AHRQ Digital Healthcare Research, evaluation resources.

Change management is a necessary component of any HIT initiative. Change management theory applies a critical view on existing processes in health systems to understand organizational culture and climate and resistance to change. These resources describe the core concepts of change management, as well as tools and templates to help manage change.

Integration

• Canada Health Infoway - A Framework and Toolkit for Managing eHealth Change: People and Processes
• The Heart of Change, John P. Kotter and Dan S. Cohen, Harvard Business School Press, Boston, 2002.
• Lorenzi, N., & Riley, Robert T. (1995). Organizational aspects of health informatics: Managing technological change (Computers in health care (New York, N.Y.)). New York: Springer-Verlag.

Underlying this toolkit is advocacy for a user-centered design approach. These resources describe the concepts and importance of a user-centered design approach to the design of artifacts, as well as methods and tools used to execute this approach.

Reporting

• Still, B., & Crane, K. (2017). Fundamentals of user-centered design: A practical approach. CRC Press.
• Mulder, S., & Yaar, Z. (2006). The user is always right: A practical guide to creating and using personas for the web. New Riders.
• Tool used to create persona in the Model Report Use guideline

Multiple ePRO visualizations (i.e., tables and graphs) can be presented on one screen via dashboards. These resources describe dashboard concepts, as well as best practices and methods in designing dashboards.

Reporting

• Few, S. (2013). Information Dashboard Design: Displaying data for at-a-glance monitoring (Vol. 81). Burlingame, CA: Analytics Press.
• LeRouge, C., Hasselquist, M.B., Kellogg, L., Austin, E., Fey, B.C., Hartzler, A.L., Flum, D.R., and Lavallee, D. (2017). Using heuristic evaluation to enhance the visual display of a provider dashboard for patient-reported outcomes. AcademyHealth eGEMs (Generating Evidence & Methods to improve patient outcomes): 5(2), article 6.
• Dowding, D., Randell, R., Gardner, P., et al. (2015). Dashboards for improving patient care: review of the literature. International Journal of Medical Informatics, 84(2), 87-100.

Data visuals are included on most ePRO reports. These resources describe the data visualization concepts and theories, as well as best practices and methods in designing dashboards.

Reporting

• Kirk, A. (2019). Data visualisation: A handbook for data driven design. Second Edition (Revised Edition) edition, Sage.
• Cairo, A. (2012). The functional art: An introduction to information graphics and visualization. New Riders.
• Grossman, L. V., Feiner, S. K., Mitchell, E. G., & Creber, R. M. M. (2018). Leveraging patient-reported outcomes using data visualization. Applied clinical informatics, 9(03), 565-575.
• Arcia, A., Woollen, J., & Bakken, S. (2018). A systematic method for exploring data attributes in preparation for designing tailored infographics of patient reported outcomes. eGEMs (Generating Evidence & Methods to improve patient outcomes), 6(1), 2.
• Snyder, C., Smith, K., Holzner, B., Rivera, Y. M., Bantug, E., Brundage, M., Weber, D., Basch, E., Aaronson, N., Reeve, B., Velikova, G., Heckert, A., Stotsky-Himelfarb, E., Chauhan, C., Hoffman, V., Ganz, P., Barbera, L., Frank, E., Smith, M. L., … PRO Data Presentation Delphi Panel. (2019a). Making a picture worth a thousand numbers: Recommendations for graphically displaying patient-reported outcomes data. Quality of Life Research, 28(2), 345–356. 
• Lor, M., Koleck, T. A., & Bakken, S. (2019). Information visualizations of symptom information for patients and providers: a systematic review. Journal of the American Medical Informatics Association, 26(2), 162-171.

Quantitative PROMs are ultimately presented as scores and other forms of statistics on ePRO reports. These resources review key considerations in statistical representation of PROMs.

Reporting

• Davidson, M., & Keating, J. (2014). Patient-reported outcome measures (PROMs): how should I interpret reports of measurement properties? A practical guide for clinicians and researchers who are not biostatisticians. Br J Sports Med, 48(9), 792-796.
• Chad E Cook (2008) Clinimetrics Corner: The Minimal Clinically Important Change Score (MCID): A Necessary Pretense, Journal of Manual & Manipulative Therapy, 16:4, 82E-83E.