Five major consumer AI health products launched in the first three months of 2026. When patients connect their medical records to them, that data falls outside the protections of the Health Insurance Portability and Accountability Act (HIPAA).

That is the warning at the center of an analysis from the International Association of Privacy Professionals. In "The health AI agent rush", the IAPP tracked the launches of ChatGPT Health, Claude for Healthcare, Amazon Health AI, Copilot Health, and Perplexity Health between January and March 2026. The IAPP's point is direct: these consumer products do not operate as HIPAA covered entities, so the protected health information (PHI) a user feeds them is governed only by a privacy policy and state consumer-protection law, not by HIPAA.

Healthcare adoption of these tools is running ahead of the rules meant to govern them, and patient data is what sits in the gap.

What is actually happening

Autonomous, large language model (LLM)-driven AI agents are being handed access to the systems and inboxes where PHI lives. Some of that access is sanctioned by healthcare organizations themselves. Some of it comes from patients connecting their own records to consumer tools.

Adoption inside healthcare is already broad. According to Fierce Healthcare, 75% of U.S. healthcare systems use or plan to use an AI platform in 2026. At the clinician level the number is higher still. The American Medical Association's 2026 physician AI survey found that 81% of physicians used AI in their practice, up sharply from prior years. When an agent can read a mailbox, query a record system, or draft a patient message, it is touching PHI whether or not anyone wrote that into a policy.

The consumer side is where HIPAA stops applying. A hospital that signs a business associate agreement (BAA) with its vendors operates inside HIPAA's framework. A patient who pastes their lab results into a consumer chatbot does not. The IAPP analysis is clear that the same medical record gets a different level of legal protection depending on which door it walks through.

The compliance reality

HIPAA only follows the data when a covered entity or its business associates handle it. Once PHI moves to a consumer product that has not signed a BAA and does not operate as a covered entity, HIPAA's protections do not travel with it.

That gap is what regulators and standards bodies are now trying to address. As reported by the HIPAA Journal on June 8, 2026, the Health Sector Coordinating Council (HSCC) released an 87-page AI Cyber Governance Framework Implementation Guide. The guide asks healthcare organizations to build cybersecurity into the full AI lifecycle, from assessment and design through development, deployment, and decommissioning. It includes a five-level AI autonomy framework, an AI-specific incident-response playbook, and practical tools for vendor contractual language and inventory management.

Under HIPAA, the covered entity decides which vendors get access to PHI, and a business associate earns that access by signing a BAA and meeting the security bar the covered entity sets. Paubox is one of those vendors. We sign a BAA, we are HITRUST certified, and we operate inside HIPAA's framework. The risk the IAPP flags is not about vetted business associates. It is about consumer AI tools and agents that never enter that vetting process at all.

The threat backdrop makes the timing matter. The Verizon 2026 Data Breach Investigations Report (19th edition) found that AI-assisted text in malicious emails has roughly doubled, and that vulnerability exploitation is now the number one access vector for breaches. The KnowBe4 Phishing Threat Trends Report Vol. 7 (April 2026) documents the same escalation in AI-generated phishing. Attackers are using AI to get in. Healthcare organizations are using AI to operate. Both trends are pulling agents closer to PHI.

How covered entities should respond

Treat agent access to PHI as a vendor decision, not a feature toggle. Before an AI agent or platform touches a system that holds patient data, the covered entity should confirm three things.

First, confirm whether the vendor will sign a BAA. If a tool cannot or will not sign one, it should not have access to PHI. The IAPP analysis shows what happens to data once it crosses into a product that operates outside that agreement.

Second, ask where the data goes and how it is secured in transit and at rest. The HSCC guide's vendor contractual language and inventory tools exist for exactly this question. An organization cannot govern what it has not inventoried.

Third, favor infrastructure that is built for healthcare compliance from the start. HITRUST certified systems have been independently assessed against a recognized security framework, which gives the covered entity a documented basis for the access it grants.

For more on why agent behavior needs to be explainable before it acts on email, read our look at why explainability matters when AI blocks business email. For the foundational requirements behind all of this, see the HIPAA compliant email definitive guide.

 

Frequently asked questions

Does HIPAA apply when a patient uses ChatGPT Health?

No. When a patient enters their own medical information into a consumer product like ChatGPT Health, HIPAA does not cover that data. HIPAA regulates covered entities (providers, health plans, and clearinghouses) and the business associates they hire. A consumer AI tool a patient uses on their own is neither, so the medical record the patient pastes in is governed by the product's privacy policy and applicable state consumer-protection law, not by HIPAA. The IAPP's analysis of the health AI agent rush puts it directly: the same lab result that is protected inside a hospital loses HIPAA's protection the moment a patient feeds it to a consumer chatbot.

Is an AI agent a business associate under HIPAA?

An AI agent is a business associate only if a covered entity uses it (or its vendor) to create, receive, maintain, or transmit PHI on the covered entity's behalf, and a BAA is in place. The HIPAA definition turns on function and contract, not on what the technology is called. A vendor that supplies an AI agent to a hospital and handles PHI in the course of that work meets the definition and must sign a BAA. A consumer AI product a patient uses independently does not, because no covered entity engaged it. Paubox is a business associate in the first sense: we sign a BAA, we are HITRUST certified, and we operate inside HIPAA's framework as a vendor that covered entities vet and approve.

Can a covered entity let an AI agent access PHI?

Yes, a covered entity can grant an AI agent access to PHI, provided the agent's vendor signs a BAA and meets the security requirements the covered entity sets. HIPAA puts the covered entity in the role of vetter. It decides which vendors earn access to patient data, and a business associate earns that access by signing the agreement and clearing the security bar. The Health Sector Coordinating Council's June 2026 AI Cyber Governance Framework Implementation Guide, summarized by the HIPAA Journal, gives organizations the contractual language and inventory tools to make that decision deliberately rather than by default.

What should we check before giving an AI tool access to patient data?

Confirm three things before any AI tool or agent touches a system that holds PHI: whether the vendor will sign a BAA, where the data goes and how it is secured in transit and at rest, and whether the infrastructure was built for healthcare compliance to begin with. A tool that cannot or will not sign a BAA should not have access to PHI. The HSCC guide's inventory tools help here, because an organization cannot govern access it has never catalogued. HITRUST certified systems give the covered entity a documented, independently assessed basis for the access it grants.

Build on infrastructure that signs a BAA

When developers need an agent or application to send patient communications, the question is whether the email layer underneath it operates inside HIPAA's framework. The Paubox Email API is built for that. It lets developers send transactional, HIPAA compliant email at scale, can personalize emails with PHI, and includes a BAA. It uses TLS 1.2 or higher, is HITRUST certified, and is trusted by more than 8,000 healthcare organizations.

AI agents will keep reaching into patient data. The organizations that stay compliant will be the ones that decide, before they grant access, which tools they have vetted.