Unequal Measurement: A Call for a National Framework for Measuring Health Equity

Abstract

• Issue: Without national, validated standards for measuring disparities in health care, rating, ranking, and regulatory organizations have adopted different metrics and methodologies for evaluating progress. Inconsistent measurement tools can draw incomplete or even conflicting accounts of an organization’s progress toward health equity and risk curbing the advancement of efforts to achieve equity.
• Goal: To investigate the current state of health equity metrics across rating and regulatory entities and propose a set of guiding principles for evaluating institutional progress.
• Methods: Systematic evaluation of the methodologies of current and proposed health equity metrics across rating and regulatory organizations.
• Key Findings and Conclusions: The lack of standardization across rating and regulatory health equity metrics may undermine efforts to achieve equity. Metrics and methodologies must be aligned to accurately assess health equity progress and drive institutional change and improvements for patient populations.

Introduction

Structural racism, poverty, and inequities in housing, education, and employment have created significant gaps in health outcomes.¹ Unequal rates of disease, treatment, and mortality disproportionately affect marginalized communities. During the COVID-19 pandemic, for instance, Black, Latina and Latino, and Native American communities suffered disproportionately high death rates, underscoring the longstanding effects of these systemic inequities.² This reality catalyzed urgent calls to address systemic drivers of health disparities and to elevate equity as a cornerstone of health care.

The Robert Wood Johnson Foundation defines health equity as “everyone [having] a fair and just opportunity to be as healthy as possible. This requires removing obstacles to health such as poverty, discrimination, and their consequences, including powerlessness and lack of access to good jobs with fair pay, quality education and housing, safe environments, and health care.”³ Some health care organizations have started incorporating health equity into their strategies through community-based initiatives, local procurement and hiring, equity-focused research, and political advocacy. However, while various national organizations have proposed different approaches to measuring progress toward equity goals in the health care sector, there is not yet an agreed upon, nationally validated standard of measurement. Inconsistent methodologies and tools for evaluating progress can produce incomplete or even conflicting accounts of an organization’s progress toward health equity. Hastily applied metrics also could cause more harm or confusion than good by inaccurately measuring success.⁴ Aligning around health equity metrics is an urgent task in addressing longstanding racial, ethnic, and economic inequities in health care.

This brief assesses frameworks and metrics developed by eight of the biggest drivers of health care quality and equity, including Centers for Medicare and Medicaid Services, The Joint Commission, and prominent hospital rating and ranking organizations like U.S. News & World Report (USNWR). A side-by-side comparison of these organizations’ measures reveal a lack of consensus around health equity progress. Based on this analysis, we developed guiding principles for measuring and validating institutional progress toward addressing health disparities and achieving health equity.

Existing and Proposed Health Equity Metrics

Current health equity measures and frameworks do not lend themselves to simple categorization or comparison because of the wide variation between them. Frameworks can rely on qualitative or quantitative measurements, and they can focus on only health care delivery and quality or look at broader domains such as community, organizational leadership, patient care, and employees. Some frameworks utilize publicly available, population-level data, which is influenced by a variety of factors — such as pollution, housing conditions, transportation access, socioeconomic status, education, cultural beliefs, discrimination, and social support networks — that may not always be accounted for in the analysis. Others require detailed assessment of processes and performance at the organizational level.

Our sample of the most well-recognized, hospital-based health equity frameworks and assessments are divided into four distinct categories based on data type and scope: 1) those that use self-reported qualitative and quantitative data at the organizational level across multiple domains like community, strategy, and policy; 2) those that use regulatory frameworks across multiple domains; 3) those that use administrative data across multiple domains; and 4) those that use quantitative data over a single domain (see Exhibit 1 for an overview).

Organizations Using Self-Reported Qualitative and Quantitative Data Across Multiple Domains

The 2023 Illinois Health and Hospital Association’s Racial Health Equity Progress Report (IHA Progress Report)⁵ comprises 10 composite metrics across four domains:

• community
• organization
• people (employees)
• patients.

Measures include comparison of organization board and leadership demographics, community and patient demographics, diversity and inclusion in the workforce, leadership engagement in health equity, patient assessments regarding experience and health outcomes, patient supports for drivers of health screening, access to free and discounted care, community investment, and community partnerships. The assessment includes 32 questions, 15 of which include open-ended follow-up questions. Open-ended questions are not scored; they are intended to prompt organizations to reflect on ways to improve their processes. Each health care organization receives a composite score. The assessment is widely available nationally⁶ for all health care organizations through a Commonwealth Fund–supported project.

The 2023 American Hospital Association Health Equity Transformational Assessment (HETA)⁷ includes six measure domains:

• equitable and inclusive organizational policies
• data collection and use
• diverse representation in leadership
• community collaboration
• accountability
• culturally appropriate care.

The assessment includes 102 questions, many with multiple answer options. Participating organizations receive access to a dashboard with hospital positions along each domain of health equity, but no scores or ranks are assigned.

Leapfrog’s 2023 Hospital Survey⁸ includes an unscored health equity section comprising 13 questions. These questions are qualitative and quantitative, falling under the following domains:

• patient demographic data collection, validation, and staff training
• quality measure stratification and improvement
• transparency in health care disparities and equity initiatives
• board engagement
• unconscious and implicit bias training for staff.

Organizations Using Regulatory Frameworks Across Multiple Domains

In 2023, the Centers for Medicare and Medicaid Services (CMS) included the Hospital Commitment to Health Equity performance measure in their Inpatient Quality Reporting (IQR) Program and structured a Health Equity Framework⁹ for hospitals to guide their equity initiatives. The IQR performance measures include five structural attestations and one social needs measure. Hospitals need to attest that:

• equity is a strategic priority
• data are being collected to measure health equity
• data analysis is being performed to measure gaps in equity
• equity outcomes are included in the quality improvement initiatives
• there is demonstrable leadership engagement in health equity.

The social needs measure includes screening for social drivers of health among adult patients over age 18 hospitalized within acute care hospitals in the United States. Hospitals report on the screening and positivity rates.¹⁰

The Joint Commission’s standards for health equity¹¹ were included within the National Patient Safety Goals, effective July 1, 2023. These standards will be evaluated as part of leadership criteria during triennial accreditation surveys. These requirements aim to ensure that:

• health equity is a quality and safety priority
• patient medical records capture social needs data
• hospitals are accountable and transparent in addressing disparities in care
• hospitals can demonstrate progress toward achieving equity goals.

Organizations Using Administrative Data Across Multiple Domains

The 2024–2025 USNWR health equity metrics¹² incorporate three domains related to health care access, including social representation in access to care, racial disparities and time spent at home, and charity care provision for uninsured patients.

USNWR examines social representation in access to care by evaluating how Medicaid patients from the surrounding community are represented in the hospital’s patient population after accounting for specific community and hospital-level factors, the percentage of patients served who live in vulnerable neighborhoods, and the representation of non-white populations from the surrounding community in the hospital’s elective procedure patient population. In determining racial disparities and time spent at home, USNWR uses claims-based data to evaluate the difference in number of days that a patient spent at home within 30 days of discharge, for Black patients compared to white patients for orthopedic and cardiac procedures. In evaluating charity care, USNWR looks at provisions for the uninsured compared to the hospital service area uninsured population by reviewing reporting on hospital 990 IRS submissions. Lastly, the ratios of Medicaid discharges and racial and ethnic patient percentages were compared to those proportions with Medicaid within the hospital service area.

The 2022 Lown Institute Hospitals Index¹³ evaluates health equity across five domains:

• pay ratio of executives to staff without advanced degrees
• time spent on charity care
• community benefit as a proportion of total expenses
• Medicaid revenue as a share of total revenue
• hospital inclusivity, including a comparison of hospital patient income, education, and racial distribution to the community area.

Organizations receive an equity grade derived from a composite score along with a state and national rank.

Organizations Using Clinical Quantitative Data in a Single Domain

Vizient’s 2022 Quality and Accountability Scorecard¹⁴ narrows in on clinical performance and includes inpatient clinical process and outcome metrics for health systems and timeliness for new patients to be seen for ambulatory settings. Inpatient metrics include sepsis and shock lactate timing and observed over expected mortality, non-ST-Elevation myocardial infarction (NSTEMI) troponin timing and observed over expected mortality, maternal hemoglobin change and transfusion rate, and heart failure brain natriuretic peptide improvement and observed over expected mortality.

The scoring methods examined equality between sex, race, and payer groups for each metric, with maternal measures stratified by only race and payer. The Vizient ambulatory metric is a 10-day access metric which evaluates the time to be seen for new Medicaid and commercial patients by specialty.

Key Findings

Current frameworks and measurements of health equity across hospitals and health systems vary widely and have little overlap. There are multiple dimensions of health equity without established gold standards for validated measures, creating a potential great risk to the field of equity measurement if proposed approaches misclassify organizations because of flawed measures.

Variability of Equity Metric Dimensions

The standards for the Centers for Medicare and Medicaid Services and The Joint Commission require hospitals to adopt a health equity framework and to screen patients for health-related social needs. They do not rely on externally available data to measure equity but rather take an organizational learning approach. The advantage of this approach is that it is applicable across many health care organizations. The disadvantage is that the standards will likely be variably interpreted and enforced.

The USNWR’s proposed metric and the Lown Institute metric attempt to measure disparities through the interrogation of publicly available data and to presume equity rankings from these data. Vizient utilizes all-payer patient outcome data only for inpatient and outpatient areas and does not cover important domains of community, employee, and organizational policies. Leapfrog captures a subset of qualitative hospital data.

The advantage of the USNWR, Lown, and Vizient approaches is that they can be widely applied across hospitals. The disadvantage is that their methodologies are not validated and can easily lead to false classification of performance. The IHA Racial Health Equity Progress Report and the AHA Health Equity Framework provide self-report qualitative and quantitative approaches for hospitals to assess and score performance among many domains from community, organizational leadership and representation, and employee and patient outcomes. While the application of these tools lends itself to organizational equity quality improvement, they are voluntary and therefore limited in reach.

Unvalidated Methodologies

Within metric domains, data sources and methodologies differ significantly. For example, in the 2024 USNWR equity measures, charity care was defined as the provision of care for uninsured patients compared to the proportion of uninsured in the hospital service area. Lown Institute, on the other hand, measures charity care spending as free care as a proportion of total hospital expenses. Since charity care measurements are highly variably reported by hospitals and with regional variation, neither USNWR nor Lown Institute approaches have been validated to assess health equity performance at a hospital level. At the other extreme, the IHA’s Racial Health Equity Progress Report charity care assessment is a self-report qualitative and quantitative survey that asks about hospital policies for the uninsured and patients insured by Medicaid and not an actual measure of charity care.

Importantly, the USNWR and the Lown Institute’s dependence on Medicare inpatient data limits the scope of evaluation to this subset of the patient population and does not comprehensively measure equity performance across all payers and all points of access.

Vizient measures disparities using an all-payer approach, but their data are only available to members and cannot be used for public reporting. On the ambulatory side, Vizient could score a hospital as providing inequitable care if Medicaid patients receive outpatient specialty care appointments sooner than commercially insured patients. Thus, a hospital can receive a lower equity ranking for providing more equitable access to ambulatory specialty care for the uninsured and Medicaid populations.

The AHA Health Equity Transformational Assessment (HETA) is limited because the results are not scored, it does not center on racial health equity, it’s only open to AHA members, and findings are not publicly reported. Because there are no comparative data, hospitals cannot see where they are on a health equity journey compared to others.

The IHA’s Racial Health Equity Progress Report includes self-reported race and ethnicity data to define the patient population across all payers. The IHA Progress Report incorporates open-ended questions such as asking hospitals to explain their social needs referral process and to score their maturity on a Likert scale, which allows organizations to track their progress toward providing equitable health care. However, despite its breadth, the data are currently self-reported and lack validation. In addition, while health care organizations receive a summative performance score and can see where they stand in comparison to each other, the findings are not publicly reported.

Discussion

Similar to the early days of the quality and patient safety movements, which called for a technical panel for validated quality improvement metrics and led to the establishment of The Joint Commission, consensus is needed for national standardized approaches to health equity measurement and validation. Unlike clinical metrics, health equity lacks a “gold standard” for comparison, making validation particularly challenging. The concept of equity encompasses structural, social, and institutional factors that are difficult to quantify consistently. Additionally, measures that may be appropriate in one setting, such as an academic medical center, may not be applicable in others, such as a rural clinic, complicating standardization efforts. The current array of equity frameworks and metrics is highly variable and could misrepresent organizations’ progress on health disparities and create confusion for the public.

The following guiding principles for a coordinated approach to health equity measurement are derived through assessing the gaps in existing health equity tools:

• Health equity metrics should include an all-payer approach, which would allow for analysis of population disparities.
• Disparities in care should be measured at the patient, community, and institutional level rather than looking at wider, more generalized factors that might not fully capture how inequalities play out in real-world settings.
• Measurement should be across the domains of community, organization, employees, and patients.
• For health equity, both inpatient and outpatient performance should be assessed.
• Process and outcome measures should be defined, specifying both the numerator and denominator for each metric.
• Methodologies must be transparent, validated, and subject to peer review.
• Identification of racial and other inequities in access to care and care outcomes reported to Board to encourage leadership to act on disparities.

Exhibit 3 applies the guiding principles listed above and provides examples of potential qualitative and quantitative measures across equity domains.