Understanding connections between HIPAA, healthcare data, and AI

Artificial intelligence is reshaping healthcare faster than the legal frameworks governing it, and that gap is creating real compliance risk. HIPAA does not contain AI-specific provisions, however that does not mean AI exists outside the rules. The same Privacy, Security, and Breach Notification Rules that have always governed protected health information apply regardless of whether a person, a traditional software system, or a large language model handles PHI. What has changed is the scale, speed, and opacity of how AI touches that data, and what healthcare organizations are now required to do about it.

Understanding AI in healthcare

Artificial intelligence in healthcare is not a single category. A 2025 report published in JAMA by more than 60 researchers from the 2024 JAMA Summit on AI found that AI-related technologies are already being applied across clinical tools, consumer-facing mobile health apps, and business operations, and that evaluation of their safety and efficacy remains inconsistent. The report identified AI adoption at a remarkable speed while warning that the systems for evaluating and regulating it have not kept pace.

Broadly, AI in healthcare can be grouped into four categories: autonomous AI that performs tasks without continuous human involvement, augmented intelligence that supports clinical decision-making, automation software with AI capabilities, and generative AI that creates new content from patterns in large datasets. Each category carries distinct compliance implications under HIPAA. A 2025 NEJM analysis of generative AI in medicine noted the technology's promise for reducing administrative burden while flagging growing concerns about data governance and oversight as deployment accelerates into real clinical environments.

How HIPAA governs AI use

HHS's summary of the HIPAA Security Rule makes the foundational principle clear: the Security Rule is designed to be flexible, scalable, and technology-neutral. The technology-neutral design is important and HIPAA does not treat AI differently from any other system. If an AI tool creates, receives, maintains, or transmits ePHI on behalf of a covered entity or business associate, the full framework applies: minimum necessary access, access controls, encryption, audit logging, risk analysis, and breach notification.

PHI may be shared with an AI system without patient authorization for treatment, payment, and healthcare operations. An AI tool analyzing a chart to support a clinical decision is a treatment activity. An AI agent verifying insurance eligibility is a payment activity. Both are permissible, however both require a signed Business Associate Agreement with the AI vendor and strict adherence to the minimum necessary standard.

A peer-reviewed paper published in the Journal of the American College of Emergency Physicians in January 2026, with authors from George Washington University, Yale School of Medicine, NYU, and the University of Wisconsin, found that while 66% of US physicians now report actively using AI tools, only about 23% of health systems have BAAs in place with third-party AI vendors. The paper describes this gap as a daily occurrence across US hospitals and clinics, noting that even well-intentioned use of an AI tool, such as generating discharge instructions using a chatbot, may result in an unauthorized disclosure of PHI if no BAA exists.

The impact of AI on PHI exposure

The compliance gap between AI adoption and legal safeguards is not theoretical. The Netskope Threat Labs Healthcare 2025 report found that 88% of healthcare organizations have integrated cloud-based generative AI tools into their operations, 98% use apps incorporating generative AI features, and 96% use tools that use user data for training. Despite this, 71% of healthcare workers still use personal AI accounts for work purposes. Since most public AI tools, such as ChatGPT and Gemini, do not sign BAAs or meet HIPAA security standards, using them with PHI constitutes a potential violation. Netskope found that 81% of all data policy violations in healthcare involved HIPAA-regulated data, the highest rate across all industries.

The Netskope 2026 Cloud and Threat Report added further context, finding that data policy violations associated with generative AI usage more than doubled year over year, with organizations detecting an average of 223 monthly attempts from employees to include sensitive data in AI prompts or uploads.

According to Paubox's Shadow AI Report, this is also an acute problem in healthcare email specifically. The report found that 95% of healthcare organizations report staff already using AI tools, yet 25% have not formally approved any staff AI use in email, and only 42% have signed a BAA covering any AI assistant used for email communication. A further 62% had observed staff experimenting with tools like ChatGPT without organizational approval.

The email channel is where much of this risk concentrates. Healthcare depends heavily on email for care coordination, lab results, billing, referrals, and patient communication. As AI tools become embedded in email platforms such as Microsoft 365 Copilot and Google Gemini, the boundary between approved communication and AI-assisted processing of PHI becomes harder to identify. According to Paubox's 2026 Healthcare Email Security Report, 2025 saw 170 email-related healthcare breaches affecting more than 2.5 million individuals, alongside a 47% increase in attacks evading native email defenses and a 17% increase in overall phishing volume.

The proposed HIPAA Security Rule update and AI

HHS reported that from 2018 to 2023, large healthcare breach reports increased by 102%, and the number of individuals affected rose by 1,002%, primarily from hacking and ransomware. In 2023 alone, over 167 million individuals were affected by large breaches, a new record. Against that backdrop, HHS published its first proposed update to the HIPAA Security Rule in over two decades.

The NPRM, published in the Federal Register on January 6, 2025, proposes changes that carry direct implications for AI. The rule would require covered entities to maintain a written technology asset inventory and network map that explicitly includes AI software interacting with ePHI, updated at least annually. It would remove the long-standing distinction between "required" and "addressable" implementation specifications, making encryption of ePHI at rest and in transit, multi-factor authentication, network segmentation, and annual compliance audits all mandatory. It would also require vulnerability scanning every six months, penetration testing annually, and the ability to restore critical systems within 72 hours of an incident.

The HHS fact sheet on the NPRM confirms that the HHS expects AI software used to create, receive, maintain, transmit, or interact with ePHI to be included in the technology asset inventory. The public comment period closed in March 2025 with nearly 5,000 submissions. As InfoSecurity Media Group reported in December 2025, the rule's fate under the current administration remains uncertain, with privacy attorney Adam Greene of Davis Wright Tremaine describing the May 2026 HHS regulatory deadline as "more aspirational than a deadline."

Types of AI risk under HIPAA

Not all AI risk in healthcare looks the same. Compliance teams need to account for several distinct categories:

• Training data exposure: The NPRM explicitly warns that generative AI tools have produced in their outputs the names and personal information of individuals included in their training data. HHS states that training AI models on patient data without appropriate safeguards could result in impermissible disclosures under the Privacy Rule. HIPAA recognizes two de-identification methods: Safe Harbor, stripping all 18 specified identifiers, and Expert Determination, which requires a qualified statistician to certify sufficiently low re-identification risk. A PMC paper on HIPAA liability and generative AI warns of an additional risk specific to large models: re-identification of Safe Harbor-processed datasets through data triangulation, particularly as dominant tech companies hold cross-referencing data at scale.
• Shadow AI: According to Paubox's Shadow AI Report, 69% of healthcare IT leaders felt pressured to adopt AI faster than their organizations could secure it, while 16% of compliance leaders were not consulted before AI features were activated in Gmail or Outlook. The Netskope Healthcare 2025 report found that 43% of healthcare workers still use personal generative AI accounts at work, creating monitoring blind spots that traditional security controls, such as email DLP or encryption gateways, often fail to catch.
• Third-party and business associate exposure: The American Hospital Association, in its February 2026 formal comments to HHS, urged that third-party AI vendors handling PHI be held to the same accountability standards as covered entities and business associates noting that current HIPAA rules do not automatically extend to all AI vendors operating outside the traditional covered entity and business associate framework.
• Clinical AI liability: The 2026 emergency medicine paper published in PMC outlines how individual clinicians, and institutions, may face liability for inadvertent HIPAA violations caused by using public AI tools in clinical workflows. The paper draws distinctions between incidental, accidental, and unauthorized disclosures of PHI when AI is involved, and provides guidance on post-breach mitigation steps for emergency department providers.

Recognizing HIPAA risk in AI workflows

Warning signs that an AI deployment may create HIPAA exposure include: use of consumer AI tools without a BAA for any tasks involving patient information; AI vendors unwilling to sign BAAs; staff-level AI adoption that bypassed compliance review; AI systems absent from the organization's technology asset inventory; and generative AI tools producing outputs that reference or reproduce identifiable patient details.

The scale of unmanaged AI use in healthcare is documented. According to the Netskope Healthcare Threat Labs Report, regulated data, primarily PHI, accounts for the dominant share of data policy violations occurring in generative AI contexts, with adoption of DLP controls to monitor generative AI apps increasing from 31% to 54% of healthcare organizations over the past year. That growth in controls shows recognition of the problem, however also confirms that nearly half of healthcare organizations still lack active AI-specific data loss prevention measures.

Best practices for HIPAA compliant AI use

Reducing compliance risk from AI requires the same layered approach that governs other aspects of HIPAA. No single control is sufficient.

Organizations should start with a technology asset inventory that includes every AI system interacting with ePHI. The step is explicitly required under the proposed Security Rule update and is already a recurring theme in OCR enforcement actions, which have repeatedly cited failure to identify where ePHI resides as a foundational deficiency. Any AI vendor touching PHI should have a signed BAA on file before use begins. As the emergency medicine paper in PMC notes, even a well-intentioned act such as using a chatbot to draft discharge instructions constitutes a HIPAA violation when no BAA exists.

Risk analysis must be updated to show AI-specific data flows. The NPRM requires covered entities to incorporate AI tools into their risk analysis and management activities, assessing the volume and categories of ePHI accessed, the purposes for which it is used, and the controls limiting exposure. The NIST AI Risk Management Framework provides a complementary structure for evaluating AI-specific risks, including validity, reliability, safety, and explainability, alongside HIPAA's baseline requirements.

For email specifically, encryption should apply automatically to every outbound message rather than depending on keyword detection or manual staff choices. According to Paubox's 2026 Healthcare Email Security Report, 86% of healthcare IT leaders reported that current tools introduce workflow friction that leads staff to bypass controls. The same principle applies to AI governance: protections that depend on individual employees making the right call every time they use a new tool will fail at scale. Paubox's Inbound Email Security is designed to detect AI-generated phishing attempts and anomalous email behavior before they reach inboxes a critical layer as AI-generated attacks become harder to distinguish from legitimate communication.

Learn more: Paubox Inbound Email Security | Paubox Email Suite | Paubox Shadow AI Report

In the news

In March 2025, Yale New Haven Health disclosed that an unauthorized third party accessed its network and exfiltrated data affecting 5,556,702 individuals, the largest healthcare breach reported to HHS in 2025. The organization's own breach notice confirmed that electronic medical records were not accessed, however that demographic information, Social Security numbers, medical record numbers, and patient type data were among the data stolen. Healthcare Dive reported the incident came after a record-breaking year of cyberattacks in healthcare, and that Yale New Haven subsequently agreed to an $18 million class action settlement requiring meaningful data security improvements. The breach followed the sector-wide pattern documented in HHS breach portal data, showing that as of December 31, 2025, at least 642 large healthcare data breaches occurred throughout the year, affecting more than 57 million individuals.

These incidents reinforce the compliance stakes around AI adoption. Every new AI tool embedded in clinical or administrative workflows, whether for documentation, scheduling, or communication, creates an additional pathway through which PHI can move, and each pathway requires the same HIPAA safeguards as every other.

FAQs

Does HIPAA apply to AI tools used in healthcare?

Yes. HIPAA's Privacy, Security, and Breach Notification Rules apply to any system that creates, receives, maintains, or transmits ePHI, including AI tools. HHS confirms that the Security Rule's technology-neutral design means AI systems interacting with ePHI are subject to the same protections as any other electronic system. The proposed NPRM explicitly names AI software as something that must be included in technology asset inventories.

Does a healthcare organization need a BAA with an AI vendor?

Yes, if that vendor processes PHI on behalf of the covered entity. A BAA is required regardless of whether the vendor calls its product an AI assistant, a productivity tool, or a clinical support system. A 2026 peer-reviewed paper from George Washington University and Yale School of Medicine found that only 23% of health systems have BAAs in place, with AI vendors a figure the authors describe as a critical institutional protection gap, given that 66% of physicians now use AI tools.

Can healthcare organizations use patient data to train AI models?

Only under specific conditions. PHI may be used for treatment, payment, or healthcare operations purposes with appropriate safeguards and a valid BAA, or with explicit patient authorization. Data must first meet HIPAA's de-identification standards. The HHS NPRM explicitly warns that using patient data to train generative AI models without proper controls can expose patient information to bad actors.

What is shadow AI, and why does it matter for HIPAA?

Shadow AI refers to staff use of AI tools without IT or compliance approval. According to Paubox's Shadow AI Report, 62% of healthcare organizations observed staff experimenting with unsanctioned tools. The Netskope Healthcare 2025 report found that 81% of healthcare data policy violations involved HIPAA-regulated data predominantly through personal AI accounts and cloud apps used outside sanctioned workflows. When PHI reaches a system without a BAA, the disclosure may constitute a HIPAA violation regardless of the employee's intent.

What should healthcare organizations do now to prepare for AI-related HIPAA requirements?

Build a technology asset inventory that includes all AI systems, require BAAs before any AI vendor accesses PHI, incorporate AI into formal risk analysis, enforce default encryption across all outbound email, and verify that logging and monitoring controls cover AI-assisted workflows. These steps align with current OCR enforcement priorities and are explicitly required under the proposed HIPAA Security Rule update. Organizations that integrate AI into their compliance programs from the start face much less exposure than those responding after the fact.