“With large language models (LLMs) being rapidly integrated into the healthcare industry, an increasing number of hospitals, healthcare professionals, and even patients are relying on AI chatbots for various purposes, including workflow optimization,” explains a research article on AI Chatbots and Challenges of HIPAA Compliance for AI Developers and Vendors.

The use of these technologies creates new challenges for healthcare organizations trying to protect sensitive information. As the article explains, “The deployment of AI chatbots in the healthcare industry can be accompanied by certain privacy risks both for data subjects and the developers and vendors of these AI-driven tools.”

That’s why healthcare organizations must understand the risks involved when using AI tools to handle protected health information (PHI).

 

How AI chatbots are being used in healthcare

According to the research article, hospitals and physicians may enter patient health data into AI chat tools to answer routine medical questions, create medical documentation, generate patient letters and medical summaries, improve patients’ understanding of procedures and side effects, and generate clinical and discharge notes.

Patients can also interact directly with AI systems. The article notes that “patients engage in a customized conversation and share their own PHI with an AI chat tool” when looking up medical questions and recommendations. The article explains that when an AI chatbot interacts with a user, it “initially collects data which is then processed and transformed into a mathematical representation.” The chatbot subsequently uses training data to identify patterns and generate predictions about the most likely response. For example, a doctor may use an AI chatbot to assist in diagnosing a patient's symptoms based on the data collected during the conversation. However, in this case, the doctor must understand what happens to the patient’s information after it is entered into an AI tool.

 

When AI vendors become business associates

HIPAA applies differently depending on the relationship between a healthcare organization and the technology provider. The article describes a situation in which a HIPAA-covered entity enters a business associate agreement with an AI developer or vendor to disclose patients’ electronic medical records. In that situation, “The AI developer/vendor will be a business associate of the covered entity under HIPAA.”

As a result, business associates have different obligations under HIPAA when they handle PHI on behalf of covered entities. The author describes HIPAA as one of the leading federal health privacy laws in the United States, with the Privacy Rule focused on protecting “individually identifiable health information,” or PHI.

 

A deeper look into HIPAA’s limitations

The research article argues that HIPAA does not necessarily cover every situation involving health data and AI. The author states that there are healthcare AI scenarios in which “HIPAA lacks sufficient protection for patients and clarity regarding the responsibilities of AI developers and vendors.”

The article explains that when a patient provides PHI to an AI chatbot for medical advice, the AI developer or vendor may be “neither a covered entity nor a business associate.”

The same issue can arise when a hospital or physician provides PHI to an AI chatbot for workflow optimization. According to the article, if the AI developer or vendor is not a HIPAA-covered entity, business associate, or subcontractor, “that PHI is no longer regulated under HIPAA.”

This shows us the limitation of relying on HIPAA alone. As the author points out, “a considerable number of AI developers and vendors are technology companies that operate outside the traditional scope of HIPAA’s covered entities and business associates framework.”

Therefore, simply asking whether a technology is being used in healthcare may not be enough to determine what protections apply.

 

Health information can exist outside HIPAA

Another challenge is that sensitive health information can exist in places that do not meet HIPAA's definition of PHI. The article explains that personal information can fall outside HIPAA even when it can be used to draw conclusions about an individual's health. The author also points to user-generated health information, including health information posted on social media, as information that falls outside HIPAA's scope.

An individual's health information does not necessarily become less sensitive when it’s stored outside a HIPAA-covered system. The article also raises concerns about re-identification, noting a “significant risk of privacy violation through re-identification of health datasets that are de-identified through the Safe Harbor mechanism,” referring to this process as “data triangulation.”

For example, an individual's health data that is shared on social media platforms can be pieced together with other publicly available information to re-identify the person, potentially compromising their privacy.

 

The FTC is expanding the health-data privacy conversation

In addition to HIPAA compliance, the article examines Federal Trade Commission (FTC) enforcement actions regarding consumer health data, including cases against Flo Health, Easy Healthcare, GoodRx, BetterHelp, and 1Health.

More specifically, “The FTC has currently taken a proactive stance in protecting health data,” increasing privacy considerations for companies handling health information. Several of the cases involved allegations concerning the disclosure or sharing of health information with third parties. For example, the FTC alleged that Flo Health shared consumers’ personal health information with third parties including Google and Facebook, despite its privacy promises.

The GoodRx case similarly involved allegations concerning unauthorized disclosures of consumers’ personal health information to companies including Facebook and Google.

BetterHelp faced allegations concerning disclosure of consumers’ health information to third parties, deceptive privacy representations, and failures to take reasonable measures to safeguard collected health information.

 

Why organizations must use a risk-based approach

The author writes that organizations should treat health data in a way that is compliant with “the letter of HIPAA” and “its spirit and purpose.” More specifically, organizations should look at how health information moves through their entire workflow.

The article specifically recommends that organizations “minimize their data collection to what is strictly necessary.” It also recommends monitoring tracking technologies to prevent “unintended and unlawful collection or sharing” of consumers’ health information.

Furthermore, the research article mentions the National Institute of Standards and Technology's AI Risk Management Framework and explains that its goal is “to offer a resource to organizations designing, developing, deploying, or using AI systems.” The framework also helps manage AI risks while promoting trustworthy and responsible development and use. For example, organizations can use the framework to assess potential risks associated with their AI systems and implement appropriate safeguards to protect sensitive health data. This approach can help organizations mitigate privacy concerns and maintain HIPAA compliance.

 

Where HIPAA compliant email fits

Even if organizations use all the right AI chatbots and other cloud-based technologies, organizations must “clearly identify where failures happen within their systems to best protect themselves from potential legal actions.”

Moreover, given the ubiquity of emails in healthcare communication, email should be a part of the risk management strategy when protecting health information. Healthcare organizations must use a HIPAA compliant email solution, like Paubox, to eliminate potential points of failure that could lead to a data breach.

Rather than requiring healthcare employees to move sensitive conversations to unfamiliar platforms, a secure email solution can protect PHI as it allows staff to continue using email as part of their existing workflows. For example, Paubox provides automatic outbound email encryption designed for healthcare organizations. It allows recipients to read encrypted messages directly in their normal inbox rather than requiring them to log into a separate portal.

These secure emails are also more intuitive, as patient portals are inconvenient, requiring separate login details and remembering an additional set of passwords.

 

Improving the organizational workflow

In addition, the article suggests “to foster trust in AI and reinforcing the company’s commitment to safeguarding consumer privacy in AI applications, AI developers and vendors need to adopt a proactive approach in AI audits and periodically communicate with data subjects about how their data is being handled.”

When healthcare organizations use AI chatbots, this communication could include informing patients about when an AI chatbot is being used, what health information may be processed, and how their information will be handled. HIPAA compliant email platforms allow organizations to communicate this information to patients, including sharing relevant privacy notices, policies, or guidance about the use of AI-enabled healthcare services.

 

FAQs

Can AI chatbots process protected health information (PHI)?

Yes. AI chatbots can process PHI when they are designed or configured to handle health information on behalf of a healthcare organization. For example, a healthcare provider may use an AI chatbot to answer patient questions, assist with appointment scheduling, summarize patient communications, or support administrative and clinical workflows. If the chatbot handles PHI in these circumstances, the organization must check that its use complies with HIPAA and that appropriate safeguards are in place.

 

Are AI developers subject to HIPAA?

AI developers can be subject to HIPAA when they handle PHI on behalf of a HIPAA-covered entity or business associate. In these circumstances, the developer may qualify as a business associate and must comply with the applicable HIPAA requirements governing its use and disclosure of PHI.

For example, if a hospital contracts with an AI developer to provide a chatbot that accesses patient records, the developer may be acting as the hospital's business associate. The specific HIPAA obligations will depend on how the AI system is used and what services the developer provides.

 

When is an AI vendor considered a business associate?

An AI vendor may be considered a business associate when it creates, receives, maintains, or transmits PHI on behalf of a HIPAA-covered entity or another business associate. This could include an AI company providing a chatbot, transcription service, clinical documentation tool, or other system that processes PHI as part of a healthcare organization's operations.

In these situations, the healthcare organization generally needs a business associate agreement (BAA) with the vendor. The BAA establishes how the vendor may use and disclose PHI and requires the vendor to implement appropriate safeguards to protect that information.