Use of post-operative blood clot rate as measure of hospital quality may be flawed

(Press-News.org) A new study published by JAMA questions using the rate of postoperative blood clots as a hospital quality measure. The study is being released early online to coincide with the American College of Surgeons 2013 Annual Clinical Congress.

The study examined whether surveillance bias (i.e., the greater the intensity of a search for a condition the greater likelihood it will be found) influences the reported rate of venous thromboembolism (VTE; blood clot). Venous thromboembolism, which includes deep vein thrombosis (DVT) and pulmonary embolism (PE), is a common postoperative complication that remains a leading potentially preventable cause of postoperative illness and death. The Agency for Healthcare Research and Quality developed a risk-adjusted postoperative VTE rate measure, Patient Safety Indicator 12 (PSI-12), that has been incorporated into numerous quality improvement programs and public reporting initiatives.

"However, measuring VTE rates may be flawed because of surveillance bias, in which variation in outcomes reflects variation in screening and detection, or 'the more you look, the more you find' phenomenon. This can occur in a number of ways: hospitals may use screening protocols, in which asymptomatic patients routinely undergo VTE imaging studies on a certain postoperative day, or clinicians have a lower threshold to order a VTE imaging study for patients with minimal or equivocal signs or symptoms (e.g., any leg swelling prompts a venous duplex [an imaging procedure]). Hospitals that are more vigilant and perform more imaging studies for VTE may identify more VTE events, thus resulting in paradoxically worse performance on the VTE outcome measure," according to background information in the article.

To examine the effect of surveillance bias on the validity of VTE as a quality measure, Karl Y. Bilimoria, M.D., M.S., of Northwestern University and Northwestern Memorial Hospital, Chicago, and colleagues conducted a study that merged 2010 Hospital Compare and American Hospital Association data from 2,838 hospitals. Next, 2009-2010 Medicare claims data for 954,926 surgical patient discharges from 2,786 hospitals that were undergoing 1 of 11 major operations were used to calculate VTE imaging and VTE event rates.

Instead of finding the expected relationship between adherence to VTE prevention and lower VTE rates, the researchers found that VTE prevention rates were positively correlated with VTE event rates. "A paradoxical relationship was also found between a measure of hospital structural characteristics reflecting quality [hospital characteristics that examine health care quality and which reflect a hospital's resources and focus on programs intended to provide higher-quality care] and VTE event rates: hospitals with higher structural quality scores had better VTE [prevention] adherence rates, but they had unexpectedly higher risk-adjusted VTE rates. Most important, hospital VTE rates were associated with the intensity of detecting VTE with imaging studies."

Hospitals in the lowest imaging rate quartile diagnosed 5.0 VTEs per 1,000 discharges, whereas the highest imaging rate quartile hospitals found 13.5 VTEs per 1,000 discharges.

"Our study calls into question the merit of the PSI-12 VTE outcome measure as a quality measure and its use in public reporting and performance-based payments. Hospitals reported to have the highest risk-adjusted VTE rates may in fact be providing vigilant care by ordering imaging studies to ensure that VTE events are not missed. Patients selecting hospitals according to publicly available metrics may be misled by currently reported VTE performance. The measure could be counterproductive if a hospital performs poorly on the VTE outcome metric, expends efforts to improve VTE prophylaxis resulting in increased awareness and vigilance in looking for VTE, and then finds more VTEs and becomes an even worse performer on the VTE measure," the authors write. (doi:10.l001/jama.2013.280048; Available pre-embargo to the media at http://media.jamanetwork.com)

Editor's Note: This study was supported by AHRQ and a Center Development Award from Northwestern University and Northwestern Memorial Hospital to Dr. Bilimoria. Dr. Ju is supported by a grant from the National Institutes of Health. Dr. Haut is supported by a career development award from AHRQ. Please see the article for additional information, including other authors, author contributions and affiliations, financial disclosures, etc.

There will also be a digital news release available for this study, including the JAMA Report video, embedded and downloadable video, audio files, text, documents, and related links. This content will be available at 10 a.m. CT Monday, October 7 at this link.

Editorial: Postoperative Venous Thromboembolic Disease - Prevention, Public Reporting, and Patient Protection

"The study by Bilimoria et al demonstrates that VTE rates appearing on the Hospital Compare website reflect how aggressively clinicians look for VTE but probably are not directly related to quality of care," writes Edward H. Livingston, M.D., Deputy Editor, JAMA, Chicago, in an accompanying editorial.

"In fact, because some physicians more aggressively look for complications, they find more and appear to have worse outcomes on the Hospital Compare website. Less obvious in the data from Bilimoria et al is that the very high compliance rate with VTE prophylaxis might result from many patients receiving treatments from which they are not likely to benefit. This is because current process measures were based on older guidelines that overestimated the benefits of VTE prophylaxis."

"Public reporting of VTE rates should be reconsidered or curtailed because few hospitals have sufficient numbers of patients to show statistically significant effects of prophylactic measures on VTE rates. Improving the quality of care and safety for surgical patients will require more than simply public reporting of various process and outcome measures. The surgical and medical communities must make concerted efforts to follow the best available evidence and to conduct rigorous clinical trials to find the best ways to protect patients against the 'unpredictable, treacherous and dramatically tragic' occurrence of postoperative VTE." (doi:10.l001/jama.2013.280049; Available pre-embargo to the media at http://media.jamanetwork.com)

Editor's Note: The author has completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

### END