As part of the implementation process, it is important to monitor and iteratively evaluate implementation progress. This may occur via activities such as real-time reporting and stakeholder feedback. The initial launch phase of an ePRO tool should maintain engagement of the full multidisciplinary project team, including dedicated time from IT and/or informatics team members, as well as time for iterative cycles of reflection and refinement of functionality or workflows (e.g., Plan-Do-Study-Act cycles).
Sample implementation monitoring metrics
- % of eligible patients who receive ePRO notification
- % of eligible patients who submit ePRO responses
- % of missing data (i.e. incomplete ePRO responses)
- % of ePRO responses appropriately documented by clinical team
Project teams should identify key implementation process and outcome metrics that can support ongoing assessment of implementation progress, including feasibility, fidelity, acceptability, sustainability, adoption, and cost. The selection of implementation process and outcome metrics should align with your health system’s internal goals or initiatives (see Governance Activities guideline, Strategy C), and can be guided by formal evaluation frameworks (see Integration section Overview). Project teams should also ensure that clinical champions and key staff roles have the ability to easily access and view data reports in real time throughout implementation.
While quantitative data will provide critical insight into implementation progress, feedback from users (patients, healthcare teams) through formal or informal routes (e.g., interviews, conversations, observing practice) can better contextualize issues with workflow or tool alignment and inform adaptations that improve ePRO use.
The Tools and Resources section provides a sample ePRO implementation monitoring plan for reference.