Domain

Clinical Informatics

Type

Protocol

Theme

quality

Start Date

7-6-2014 2:55 PM

End Date

7-6-2014 4:15 PM

Structured Abstract

Introduction Collecting patient-reported outcome measures (PROMs) during clinical care is becoming an increasingly important part of healthcare. The Cleveland Clinic has developed a web-based information technology platform, the Knowledge Program (KP), to enhance collection of discrete clinical data for patient management, quality, and research. These data are stored within the electronic health record (EHR) and both sources are consolidated into a joint data warehouse. Assessing data quality in such a warehouse poses many challenges, including: the complexity and variety of data feeds; the number of available EHR locations for a single data element; and the distinction between dynamic data linked to patient records and static data tied to specific clinical encounters. These challenges render quality assessment methods which treat a single data source as a complete reference inapplicable.

Methods Two reviewers—one each for the commercially-available EHR and the KP data warehouse—assessed completeness and concordance of a pre-specified selection of the following clinical data elements in 250 patient charts: Demographic information, laboratory results, vital signs, medications, and diagnoses. Up to seven areas were reviewed, depending on the source (past medical history, problem list, encounter diagnosis, progress notes, vital signs, medications and scanned documents). Completeness was considered only as a relative measure and concordance was assessed in cases where the data element was available in both sources.

Findings Demographic data were present in more cases in the EHR compared to the KP database (98.4-99.6%), with the exception of death date, which was present more often in the KP database. Labs were recorded in 2-58% of cases. Relative to the KP database, labs in the EHR were more complete (5-6% difference). Vital signs were recorded in 25-78% of cases; the KP was 3-9% less complete relative to the EHR. When either lab results or vital signs were present in both the EHR and the KP database, we observed 100% concordance. For diagnoses, prevalence range was wide (2.8-36.4%). The prevalence of open medication orders was higher in the KP database than the EHR, ranging from 3.6-6.8% difference.

Lessons learned The EHR is conventionally considered the reference source for clinical data, when compared to external joint data repositories. While this paradigm may be applicable for static, encounter-based data, such as laboratory values or vital signs, the complete reference for other data elements may not be straightforward. When undertaking data quality assessments, investigators should pay careful attention to the timing of review, the specific locations of data elements subject to review, and consider methods which assess relative rather than absolute accuracy. From a quality improvement perspective, fewer locations for data entry and storage result in less variability.

Conclusion Managing electronic collection, storage and retrieval of data is a process that requires constant oversight. Key considerations are: understanding how data are collected and stored, sources for the various data feeds, and the timing of review. Close collaboration between programmers setting up databases and providers entering and utilizing patient data is integral to success. Understanding these issues can drive ongoing quality improvement in learning health systems.

Acknowledgements

N/A

Creative Commons License

Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.

Share

COinS
 
Jun 7th, 2:55 PM Jun 7th, 4:15 PM

The Challenges of Data Quality Evaluation in an EHR-based Data Registry

Introduction Collecting patient-reported outcome measures (PROMs) during clinical care is becoming an increasingly important part of healthcare. The Cleveland Clinic has developed a web-based information technology platform, the Knowledge Program (KP), to enhance collection of discrete clinical data for patient management, quality, and research. These data are stored within the electronic health record (EHR) and both sources are consolidated into a joint data warehouse. Assessing data quality in such a warehouse poses many challenges, including: the complexity and variety of data feeds; the number of available EHR locations for a single data element; and the distinction between dynamic data linked to patient records and static data tied to specific clinical encounters. These challenges render quality assessment methods which treat a single data source as a complete reference inapplicable.

Methods Two reviewers—one each for the commercially-available EHR and the KP data warehouse—assessed completeness and concordance of a pre-specified selection of the following clinical data elements in 250 patient charts: Demographic information, laboratory results, vital signs, medications, and diagnoses. Up to seven areas were reviewed, depending on the source (past medical history, problem list, encounter diagnosis, progress notes, vital signs, medications and scanned documents). Completeness was considered only as a relative measure and concordance was assessed in cases where the data element was available in both sources.

Findings Demographic data were present in more cases in the EHR compared to the KP database (98.4-99.6%), with the exception of death date, which was present more often in the KP database. Labs were recorded in 2-58% of cases. Relative to the KP database, labs in the EHR were more complete (5-6% difference). Vital signs were recorded in 25-78% of cases; the KP was 3-9% less complete relative to the EHR. When either lab results or vital signs were present in both the EHR and the KP database, we observed 100% concordance. For diagnoses, prevalence range was wide (2.8-36.4%). The prevalence of open medication orders was higher in the KP database than the EHR, ranging from 3.6-6.8% difference.

Lessons learned The EHR is conventionally considered the reference source for clinical data, when compared to external joint data repositories. While this paradigm may be applicable for static, encounter-based data, such as laboratory values or vital signs, the complete reference for other data elements may not be straightforward. When undertaking data quality assessments, investigators should pay careful attention to the timing of review, the specific locations of data elements subject to review, and consider methods which assess relative rather than absolute accuracy. From a quality improvement perspective, fewer locations for data entry and storage result in less variability.

Conclusion Managing electronic collection, storage and retrieval of data is a process that requires constant oversight. Key considerations are: understanding how data are collected and stored, sources for the various data feeds, and the timing of review. Close collaboration between programmers setting up databases and providers entering and utilizing patient data is integral to success. Understanding these issues can drive ongoing quality improvement in learning health systems.