Why You Shouldn't Use ChatGPT for Therapy Notes (And What to Use Instead)

Under the HIPAA Privacy and Security Rules, any vendor that creates, receives, maintains, or transmits protected health information on behalf of a covered entity (a licensed clinician in private practice qualifies) is a Business Associate. Business Associates must sign a Business Associate Agreement (BAA) that contractually binds them to specific safeguards: encryption at rest and in transit, access controls, audit logging, breach notification within 60 days, and a prohibition on using PHI for any purpose outside the agreed-upon services.

Three things follow from this:

1. If you send any identifiable client information to a vendor without a BAA in place, you are in violation of the Privacy Rule the moment the data leaves your device.

1. De-identification is harder than it looks. Removing the name is not enough. The HIPAA Safe Harbor method requires removal of 18 specific identifier categories, including dates more precise than year, geographic data smaller than state, device identifiers, and any code that could be used to re-link to the patient.

1. The penalty structure is tiered. Civil monetary penalties for unintentional violations start at around $137 per record and scale to $2 million per category per year. State attorneys general have parallel enforcement authority.

Each major LLM provider has a different default posture. The summary that follows reflects the consumer products, not enterprise tiers.

OpenAI ChatGPT (Free, Plus, Pro): Conversations are retained and may be used to train future models unless the user explicitly opts out in settings. ChatGPT Team and Enterprise plans do not train on customer data by default, but a BAA for HIPAA coverage is available only at the Enterprise tier and requires custom contracting.

Anthropic Claude (Free, Pro, Team): Anthropic states it does not train on customer conversations by default in the consumer products, but data is retained for trust and safety review. Claude Enterprise offers a BAA on request.

Google Gemini (Free, Advanced): Conversations may be reviewed by human annotators and used for product improvement. Google Workspace customers can enable Gemini with administrative controls, but the consumer Gemini app is not appropriate for PHI.

Note what is missing from all three by default: a signed BAA, contractual data segregation guarantees, audit logs you can request as the data owner, and the right to demand deletion of every copy of every piece of PHI ever sent.

Even enabling the opt-out flag does not retroactively cover historical sessions, does not produce a BAA, and does not give you the documentation a state board investigator will ask for after a complaint.

Privacy is only half the problem. The other half is what these models put into the note.

General-purpose LLMs are trained to produce plausible text, not accurate clinical documentation. When asked to summarize a session, they will invent symptoms the client did not report, attribute statements to the client that the clinician said, soften suicidal ideation into "low mood," and add diagnostic language that is unsupported by the source material.

If an inaccurate note is later cited in court, used by another provider to make treatment decisions, or reviewed by a licensing board, the clinician is responsible for every word in it. "The AI wrote it" is not a defense. The signature on the chart is the clinician's.

This is why purpose-built clinical AI tools include three safeguards that consumer LLMs do not:

• Source attribution that links every claim in the generated note back to the specific transcript segment or session entry it came from.
• A mandatory review-and-edit step before any AI output enters the medical record.
• Confidence indicators that flag low-certainty inferences for the clinician's explicit review.

1. Clinical AI scribes with BAA coverage.
These are designed for healthcare workflows from day one. They sign a BAA, segregate PHI from training data, log all access, and produce notes in clinical formats (SOAP, DAP, BIRP, GIRP). Examples in the therapy-specific space include CBT Assistant Pro, Mentalyc, and Upheal.

2. Self-hosted or on-device models.
Running a model locally (via Ollama, LM Studio, or a self-hosted instance) keeps PHI on your hardware. This avoids the BAA question because no third party receives the data. The tradeoff is operational complexity and weaker reasoning compared to frontier models. Best suited to clinicians with technical comfort and modest documentation volume.

3. De-identified consumer LLM use.
If you must use ChatGPT or Claude for clinical thinking support, strip every identifier first. Replace names with placeholders, generalize dates to "last week," remove geographic specifics, and use the LLM only for generic clinical reasoning, not for drafting notes that will enter a chart. Even then, document your de-identification process in case you need to defend it later.

Before the first use:
- Is there a signed BAA on file with this vendor? Get a PDF copy.
- Does the vendor publish a Trust Center with SOC 2 Type II, encryption details, and access logging?
- Can you, the clinician, request deletion of all data associated with a specific client?
- Where is the data physically stored, and does that jurisdiction satisfy your state law (some states impose additional residency requirements)?

For each session:
- Is the information you are about to send actually PHI under HIPAA?
- If yes, is the tool covered by a BAA?
- Is the output going into the medical record, or only into your own private thinking notes?
- Will you review every sentence before signing the note?

Periodically:
- Audit your AI tool's logs once per quarter.
- Confirm BAAs are still in effect at contract renewal.
- Update your Notice of Privacy Practices to reflect AI vendors in use.

CBT Assistant Pro was built specifically for licensed mental health clinicians, which shapes every design decision.

• A BAA is signed with every paying account before any client data is processed.
• All data is encrypted in transit (TLS 1.3) and at rest (AES-256), with key rotation handled by AWS KMS.
• Audit logs capture every access, edit, and export, and are retained for six years to satisfy state record-keeping requirements.
• AI-generated content is always presented as a draft for clinician review. Nothing enters the chart without explicit approval.
• Source attribution links every line of generated formulation or note back to the underlying session data.
• We do not use client data to train AI models. Ever. This is contractual.
• Clinicians can export and delete all of a client's data at any time.

The goal is not to remove the clinician from the loop. It is to take the administrative weight off the clinician so the clinical work gets more attention, not less.