First Into the Breach: ONC Final Rule Addressing AI Transparency in Health Care

The rule adopts strategies from a toolbox of options to enhance transparency in AI development and deployment.

Transparency in AI development: The final rule requires developers of certified health IT to conduct risk management for all predictive DSIs in their health IT modules. Developers must describe their data governance procedures—including how data are acquired, managed, and used—and evaluate and mitigate risks related to accuracy, bias, and safety.

In addition, developers of certified health IT must submit summary information on their intervention risk management practices to a publicly accessible ONC site. Many commenters on the proposed rule expressed concerns about exposing trade secrets or proprietary information. In response, ONC emphasized the importance of transparency and stated they do not see new risks related to proprietary information.

Transparency in AI deployment: ONC requires certified health IT modules with predictive DSIs to disclose comprehensive performance information—conceptualized as a “model card” or “nutrition label”—to end users. These labels aim to provide clear information about the design, development, training, and evaluation of predictive DSIs, and to help health care professionals understand potential limitations or biases programed into a model. As part of these requirements, health IT developers must disclose whether their AI systems have undergone external testing and validations, which might include third-party audits. ONC’s goal is to enable providers to determine if a predictive DSI meets the ONC-developed ‘FAVES’ criterion (fair, appropriate, valid, effective, and safe).

The final rule does not directly specify the model cards, nutrition labels, datasheets, data cards, or algorithmic audits required for compliance. Rather, the developer must choose which mechanism is most appropriate to communicate information about their AI tools and can choose a process that avoids sharing their AI models with an outside entity.

Limitations

The HTI-1 final rule specifically applies to predictive DSIs integrated into certified health IT systems, such as electronic health records.^[1] The risk management and transparency requirements apply to predictive DSIs “supplied by the developer” and generally do not extend to third-party tools. They also do not apply to predictive DSIs developed internally by health care providers for their own use. Moreover, the rule does not directly regulate the payer sector (i.e., health insurers), leaving a notable gap in oversight.

Additionally, the rule puts the onus on health care providers to interpret the information presented in model cards and assess the trustworthiness of AI tools. It underscores the growing need for technical literacy among health care professionals.

Moving Forward

The ONC rule represents a significant step for AI transparency in health care, but it also highlights the challenges in regulating AI technologies. The rule is narrow in scope and relies upon technical literacy among health care professionals.

The rule has also sparked discussion about how ONC and the FDA will align efforts to regulate AI and machine learning in medical devices. Some industry stakeholders would like a separate, easier pathway for AI-enabled medical devices that have already gone through FDA review.

Regulators, developers, and health care providers will need to collaborate to navigate these complexities and ensure the safe, effective, and equitable use of AI technologies.

On February 6th, ONC announced it will delay the effective date of the final rule to provide for a 60-day delay from the date of publication in the Federal Register. Vendors of certified health IT must comply with the rule by March 11, 2024. 

---

[1] As of 2021, 78% of office-based physicians and 96% of non-federal acute care hospitals used certified EHR. See: https://www.healthit.gov/data/quickstats/national-trends-hospital-and-physician-adoption-electronic-health-records