Work

Assessing and Improving Productivity in Primary Care: Proof of Concept Results for a Novel Value-Based Metric

Year: 2024

Type: article

Abstract: Our work used standard VA healthcare system administrative data that are continuously monitored for adherence to ethical and privacy regulations. Our project was conducted as quality improvement under the purview of VA’s national Office of Primary Care. The initial sample included 754 stand-alone VA community-based outpatient clinics. We excluded 19 (3%) clinics for which data were incomplete or unavailable due to small sample sizes, 16 (2%) clinics for which provider workload data were extreme or not plausible (< 250 patients served; < 150 patients or > 4000 patients per provider), and an additional 16 (2%) clinics located outside the contiguous US that operate in unique environments (e.g., availability of non-VA care or transportation). The remaining 703 (93%) clinics were included in analyses. All data used were from fiscal year (FY) 2019. For clinical quality of care , we used Healthcare Effectiveness Data and Information Set data reported on VA’s Electronic Quality Measurement platform. For access and patient experiences of care , we used VA’s Survey of Healthcare Experience of Patients (SHEP). SHEP surveys are based on Consumer Assessment of Healthcare Providers and Systems Surveys and are administered to randomly selected Veterans within each clinic after a primary care visit. For primary care interprofessional team clinical time , we used VA Primary Care Management Module (PCMM) tables from the VA Corporate Data Warehouse (CDW). PCMM tables identify staff assigned to each primary care team and report staff fulltime equivalent (FTE) hours for direct patient care by primary care team role. For total number of patients served , we accessed active patient-provider assignments in nationally maintained PCMM data files to assess provider panel sizes. For patient age and risk scores , we used administrative data from CDW. All measures contribute to a summary technical efficiency score for each included community-based outpatient clinic. Technical efficiency scores measure primary care clinics’ abilities to maximize outputs given their inputs. The score is calculated based on component measures—inputs and outputs—that comprise data aggregated across all of a clinic’s enrolled patients and/or all of its assigned providers. To measure interprofessional team clinical time , we summed FTEs for all providers (physicians, nurse practitioners, and physician assistants), registered nurses or care managers, medical assistants or licensed practical nurses, clerks, and pharmacists assigned to a primary care team. We included the clinical time of team members whose assignments to a primary care team were active within the study year. The composite measure for clinical quality applied a previously validated method to rank health systems and included 12 widely used measures addressing prevention of cancer, and detection or management of chronic medical diseases or mental health conditions (Supplement Appendix 1 ). Composite scores with a reliability of 0.7 and above were retained for analyses and transformed to a 0–100 scale with higher scores indicating better clinical quality. For access , we used the existing validated access composite score (0–100) from the SHEP patient survey. For patient experiences of care , we developed a composite measure (0–100; alpha: 0.8) based on the SHEP score domains of communication (four questions), comprehensiveness (three questions), and coordination (three questions) (Supplement Appendix 2 ). For total number of patients served , we calculated the average number of unique Veterans assigned to the clinic (across individual provider panels) over the fiscal year. In VA, provider panels represent active patients who have seen their providers within the past 2 fiscal years. We then rescaled the measure to have a maximum of 100. We adjusted for each clinic’s proportion of patients over age 70, and proportion of patients with a Nosos risk score > 1. Nosos is based on the Centers for Medicare and Medicaid Hierarchical Condition Categories risk adjustment model, and a score above 1 represents a VA patient with higher-than-average expected costs relative to the national average for VA patients. Based on an evidence review and expert panel, we determined that Data Envelopment Analysis (DEA) was the optimal method among available choices for developing our prototype measure. DEA, introduced by Charnes, Cooper, and Rhodes, is a computational method used to compare the performance of “decision-making units” or clinics on maximizing outputs given their quantity of inputs, or conversely, producing outputs with the minimum number of inputs. While DEA has been used to assess relative efficiency across primary care settings, such as Federally Qualified Health Centers, to our knowledge, this method has not been used to support primary care clinic–level improvement. DEA requires users to set parameters. First, we used an output-oriented analysis to encourage users to focus on output improvement. Second, we controlled for patient risk (Nosos) and proportion of patients over age 70 as two non-discretionary measures of patient health by including them directly into the DEA calculation. Third, we assumed a variable returns to scale in which increasing inputs can lead to disproportionate increases in outputs. This decision allows for larger clinics to disproportionately serve more patients, for example. Fourth, we constrained the weighting of outputs in the DEA calculation to (1) ensure that all outputs contributed to the efficiency score and to (2) reduce the possibility that a clinic that performed very poorly on any one of the outputs (e.g., clinical quality of care) could be judged as efficient. We used the “Benchmarking” and “Boot” packages in RStudio 2023.06.1 to implement the DEA model and calculate technical efficiency scores with 2000 bootstrap replicates. These efficiency scores measure the degree to which clinics, given the numbers of team FTEs dedicated to care delivery, maximized primary care outputs (see Supplemental Appendix 3 for efficiency score calculation information). To examine whether results had face validity and could show paths for clinic improvement, we summarized the distributions of efficiency scores, examined associations between efficiency scores and outputs, and reviewed the efficiency scores of clinics with similar input levels. These analyses were completed using Stata 18.0. Table 1 shows the ranges, means, and medians for each of the components of our DEA model, including outputs, inputs, contextual adjustments, and technical efficiency scores. The median scores for each output were as follows: clinical quality = 57 (range 0–100), access = 55.3 (range 27–85.4), patient experience = 69.4 (range 49.6–84.1), and number of patients served = 3249 (range 287–30,621). Table 1 also illustrates the wide variation across clinics in terms of staffing. The median FTEs for each team member category were as follows: primary care provider = 3.1 (range 0.2–26.3), nurse = 3 (range 0–28.2), medical assistant = 3 (range 0–28.6), administrative staff = 2.9 (range 0–23.1), and pharmacist = 0.5 (range 0–10.8). The median number (not shown) of Veterans served per primary care provider (PCP) FTE was 996 (range 304–2326) and was higher among large clinics with 16.4 or more team FTEs (1049 Veterans) compared to smaller clinics with fewer than 9 team FTEs (936 Veterans). In terms of context, the proportion of patients aged 70 and over served by each clinic ranged between 5.4 and 80.5% (median 44%) and the percent of patients expected to have higher-than-average costs ranged between 9.3 and 49.5% (median 20.6%). Primary care clinics were considered “efficient” when, relative to all other clinics in the sample, they produced maximum outputs (i.e., the sum of weighted scores for access, clinical quality, patient experience, and number of patients cared for) given the number of team hours (FTEs) used to deliver those output results. Of the 703 clinics in the analytic sample, 10 (1.4%) received an efficiency score of 1 and were classified as efficient. This indicates that few community-based outpatient clinics in our sample achieved “maximum” efficiency (represented by the minimum efficiency score of 1). The remaining clinics within the sample did not achieve efficiency and had scores above 1 up to a maximum of 1.8. The median efficiency score was 1.3 (95% CI 1.21–1.27), which suggests that half of clinics in the sample, given their staffing levels, could increase their outputs by at least 30%. Figure 1 displays efficiency scores of 703 clinics in relationship to one primary care output (clinical quality). Efficiency scores are on the Y -axis and clinical quality scores are on the X -axis. The scatterplot shows that, as expected, scoring higher on clinical quality was strongly associated ( r = − 0.8) with scoring lower (better) on efficiency score. We observed similar relationships between efficiency scores and the remaining outputs including access ( r = − 0.5), patient experience ( r = − 0.5), and total patients served ( r = − 0.2). Serving greater proportions of older patients ( r = 0.1) was correlated with greater inefficiency, although the relationship was weak. Scatterplot of the relationship between each primary care clinic’s efficiency score and clinical quality score, FY2019 ( n = 703)*. In Fig. 2 , we assessed clinic efficiency score results for all sample clinics relative to our four outputs. Each of the four charts in Fig. 2 displays the relationship between a clinic’s efficiency score ( Y -axis) and its score for one of the outputs ( X -axis). The distribution of values within each chart is depicted as a gray shadow. Figure 2 also highlights six example clinics that had similar staffing levels (Table 2 ). These clinics are marked across the four charts in differently colored symbols. Table 2 displays the input and output values for each of the six clinics. Although clinics had similar inputs (investment in staffing by team role), their performance across each healthcare domain varied significantly. Clinics that scored among the highest in outputs and performed better across multiple outputs were scored as more efficient. Based on total patients served alone, the Blue clinic would be rated below its peers. However, this clinic delivered exceptional clinical quality, while exceeding the sample median score for access, and achieved a nearly maximum overall efficiency score of 1.04. The Pink clinic scored below the median for both patient experience and access and was scored as less efficient. Green and Red clinics also scored low for access but scored higher in other domains and were scored as more efficient than the Pink clinic. The Orange and Purple clinics had comparable efficiency scores but different patterns of output scores. Display of primary care outputs ( X -axis) versus technical efficiency scores ( Y -axis)* for six example primary care clinics with similar staffing levels**, FY2019. *Efficiency scores are based on DEA calculations. Each of the four charts displays efficiency scores ( Y -axis) versus scores for one of four outputs ( X -axis). The grayed background shapes in each chart represent the distributions of efficiency scores versus each output score across all 703 primary care clinics. Vertical lines represent sample medians per output. **Each colored shape is assigned to a single clinic such that the clinic’s results can be visually tracked across each outputt. Efficiency scores are thus the same for each clinic across all four output charts, while output scores indicate where each “inefficient” clinic has room for improvement. Exact values for the six clinics are shown in Table 2 .

Source: Journal of General Internal Medicine

DOI: 10.1007/s11606-024-08710-0