Resource Title

Web-based Comparative Patient-reported Outcome Feedback to Support Quality Improvement and Comparative Effectiveness Research in Total Joint Replacement


Introduction: Patient-reported outcomes (PROs) are rarely included in quality monitoring systems, surgeon comparative feedback reports, or registries. We present the design and implementation of a secure website in a federally funded research program—Function and Outcomes Research for Comparative Effectiveness in Total Joint Replacement (FORCE-TJR)—to return comparative PRO reports to participating surgeons, in addition to including traditional quality measures, in order to monitor and improve quality and health outcomes.

Methods: The surgeon-specific comparative PRO reports were designed and structured based on user input for content, data elements, integration, and display. Three questions are addressed regarding the knee and hip joint symptom profiles of patients before TJR, as well as outcomes of surgery. The website is organized with a hierarchical structure to display data at national, practice, and individual surgeon levels, and provides a comprehensive site-level executive summary and surgeon-level data reports that can be downloaded.

Early Results: As of September 2014, over 22,000 patients were enrolled from more than 130 surgeons in 22 states. The reporting website was launched in September 2012 and has been updated quarterly for all surgeons to review their site- and individual-specific outcomes data compared to national benchmarks.

Discussion: In this novel system, quarterly comparative surgeon feedback extends beyond traditional measures of complication rates to include PROs of pain relief and functional gain. We anticipate that this enhanced data will facilitate patient-centered quality improvement (QI) and outcomes research from the registry. As the Centers for Medicare & Medicaid Services (CMS) and other insurers consider future implementation of PROs, surgeons will increasingly need comparative data by which to self-monitor their practice outcomes.

Use Case

Quality Improvement

Implementation Considerations

Scoring, interpretation and display of results

Resource Types


Data Collection Tools/Healthcare Settings

In FORCE-TJR research registry, validated PRO surveys are completed by patients via scannable paper forms or secure web-based surveys that can be done in the office during the visit or at home. Site staff enroll patients and monitor baseline survey completion for their settings. A centralized Data Coordinating Center (DCC) disseminates post-operative PROs for all sites by sending electronic or paper PRO surveys directly to the patient’s home and managing follow-up survey return rates. The DCC maintains the centralized secure database that integrates data from patients, surgeons, and hospitals. Risk-adjusted, comparative data are provided quarterly through web reporting. More than 200 surgeons from 28 states are able to review their site- and individual-specific pre-operative patient risk profiles as well as post-operative outcomes compared to national benchmarks.

Generalizability to other settings and best practices

The feasibility of integrating PROs in clinical settings is supported by years of successful implementation of the FORCE-TJR research registry across more than 200 practicing orthopedic surgeons. The PRO collection process minimizes modifications in office flow and uses a dedicated clerk to guide the patient through the PRO survey. Post-operative PRO data collection is managed by web-based monitoring software that generates weekly, updated “to do” lists to remind the staff to email, mail, or make telephone reminder calls to assure complete PRO data. Data feedback to surgeons and hospitals extends beyond traditional measures of complication rates to include profiles of patient risk factors and post-operative PROs with comparison to peers and national benchmarks. The data capture and reporting methods can be used as a general-purpose template for PRO collection and use in other settings to monitor and improve quality and health outcomes.

Acknowledgement of Funders

This work was supported by Agency for Healthcare Research and Quality (AHRQ) Grant Number 1 P50HS018910-04. The authors thank the study team: Jeroan Allison MD MS, Bruce Barton PhD, Norm Weissman PhD, Courtland Lewis MD, Philip Noble PhD, Regis O’Keefe MD PhD, Vincent Pellegrini MD, and the data analytic team: Celeste Lemay, Wenyun Yang, Michael Kenny, Betsy Costello, Dane Netherton, and Peter Lazar, for their contributions to this work. This submission is based on work presented at the 2014 EDM Forum Symposium.