Launching Health in ChatGPT to US Users

I like it when patients research things. Consult is more engaging for me when people care about and try to understand stuff.

It's hard for patients to interpret medical research and evidence though... ChatGPT recommended treatment for some of my family members based on studies with 60 patients. It wasn't exactly established medical practice.

Radiology is probably an edge case, due to needing the right inductive bias and training data that general purpose multimodal language models may lack. At least until someone creates RadiologyBench and provides an incentive to the labs to make that part of their data pipeline.

I have witnessed a non-radiology case (a STD on the face) where the LLM got it right and the doctor didn't (he said it was diabetes related). It may have saved that person's life by causing them to go to a different doctor with the LLM's diagnosis and getting the proper treatment.

What I don't understand is why don't doctors use LLMs as a tool for a quick second opinion before arriving at a determination? They have my data. They can put it into GPT 5.6 Pro or Fable, read the output, and choose to disregard it if they really want to, but if there's a 20% chance that it spots something they missed, why not?

I know of someone who had a red bump on their arm and asked ChatGPT if that was a problem and if they should see a doctor. It said no so they left it. Turns out it was a tick and they now have lyme disease.

One of the worst things a new mother can do is use google, instant panic attacks. I imagine using an LLM would do the same.

When my first kiddo was an infant any google result for information related to even a sneeze said it was an indication of some super tragic situation. Any advice about a behavior or trait ended with basically "your child is destined to be a serial killer".

For any new parents out there, just call your doctor. Stay far away from the internet and AI when it comes to your baby's health.

I don’t have the reference on my fingertips, but patients who are involved in their own healthcare have better outcomes. Moreover, for those of us (and there are many of us) who have gotten increasingly frustrated with US healthcare (e.g., long waits, short visits where you never really get a chance to discuss the health issue, difficulty getting appropriate tests), AI is looking like an increasingly attractive option. Not as a replacement for physicians, but at least as a path forward to possibly better understand what is going on.

Its already happening, I am not a doctor but work with them closely. One patient used ChatGPT for his problems because he waited so long for an appointment. He was a bit neurotic and talked the whole time how one doesnt need a doctor anymore as ChatGPT is already good enough.

The real main problem is that you need to wait to get help, may it be a therapist or an eye doctor. But ChatGPT is instant.

This part worries me the most.

It's not a problem on it's on, it's a symptom of a systemic issue. Access to healthcare can be very limited even in developed countries. Like, you need to sometimes wait half a year to get a specialist appointment in Europe (unless you can afford private healthcare). In this case, people will be using AI to get faster answers.

Theres a good use case in AI for patients to chat to tho, if i could transfer all of the context of a conversation I had with a AI bot to my Dr before a appointment it would most likely improve outcomes because I have been much more verbose and descriptive to the AI bot. You basically want an AI chat bot for patient history...

It's way better than the Google MD problem that previously existed.

I had Claude analyze a blood panel and provide potential diagnoses, medications, dosing and contraindications for a fairly esoteric condition. It gave me the questions to ask a doctor (being completely forthright that I'm yet another layman regurgitating AI output) who agreed with the approach and even admitted that he hadn't seen a case study it referenced but that he wish he knew about sooner.

Will doctors still be assmad that years of their study is now democratized? Probably. I am (or was) a software engineer, welcome to the club.

That study cites that 45% of vendors had hallucinations, likely vendors using smaller or cheaper models. So 55% of vendors had no known hallucinations.

It’s a patient privacy law that applies to hippos.

[dead]

HIPAA only applies to data originating from a covered entity.

It doesn't apply to information the patient supplies directly.

As a patient, I can take out a street sign displaying my health records if I want. It's my data.

If I want to upload it to a strange AI or any other app, that's completely fine. HIPAA isn't involved, for either me or the company I upload to.

In fact, quite the opposite. CMS has been going to great pains over the past decade or so to create standards to help patients get access to their records and own them themselves.

Different states may have additional protections in this case, and I know there has been some discussion at the federal level about regulating these "wellness" apps.

As an interesting technical detail not many people know, HIPAA only applies to Covered Entities and Business Associates engaging in the electronic exchange of information using "standard transactions", and was intended to regulate insurance and billing transactions.

A doctor that keeps paper charts and bills cash directly to the patient wouldn't fall under HIPAA at all.

I'm just guessing here, but I find that the models are often quite bad at ignoring information.

So if the harness appended a timestamp to each message the model sees, I wouldn't be surprised if the model often responded with "given that it's 12:30 pm on a Sunday..." even in cases where that's absolutely not relevant.

Maybe you could instead give the model a tool it could call that would return timestamps?

I find this weird too! When you say something that prompts them that you’re talking about the current time, they’ll go and fetch it and use that to understand time frames which makes sense. But if you’ve been talking about something tomorrow, and then tomorrow comes and you go back to that chat, they don’t understand that now it’s today/right now.

I too have been wondering about the missing timestamps, and sometimes feel like it would be really useful. My best guess is fear of context pollution, but no idea...

> I have very little use for the current medical establishment.

What do you think the LLMs are trained on?

I agree the current state of US healthcare is a mess. But that doesn’t mean anything is better. The current state of LLMs are so agreeable I worry what they’ll suggest to users who provide even the slightest pushback.

“Are you sure I need this test?”

“You’re right to question that. It’s unlikely to surface anything you don’t already know”

…while a chronic condition develops under the surface

Agreed on the problems, but how will an LLM fix any of that? Even if it does somehow give you the right answer, it cannot perform procedures or prescribe medicine. You still need to get an appointment for anything beyond information. Your insurance isn't going to say, "Oh, sure, if chatGPT said you need this, we'll approve it."

It sounds like you need a better patient advocate or your area is very underserved in provider choice. Many people have success with the health system.

NYS is trying to make it illegal for LLMs to give medical, legal or engineering advice. Because of course its a state built upon exploitation so.

https://www.nysenate.gov/legislation/bills/2025/S7263/amendm...

The engineering aspect is quite hilarious. I have a PE license, but don't use it. But the PE definition in NY is wide open. Technically, if you do the right work, even writing software can fall under a PE license, though its incredibly rare. So this could theoretically ban LLM Code assistants.

Do you have a health insurance?

"We'll see!"

-USA

It's a boon for low income people in low income countries. Some random facts:

- They avoid doctors because it's expensive and they don't want to burden their families.

- There aren't enough doctors, regardless of social spending.

- There isn't enough social spending.

Even if this only increases bandwidth of existing doctors, or allows lesser-trained nurses to cosplay as a doctor during triage to increase bandwidth, it's a great thing.

I bet people out there are already uploading all sorts of private or confidential stuff to ChatGPT. I don't think that will make much of a difference.

This isn't so bad vs the alternative. My healthcare data is spread over a bunch of small systems built with poor security practices at the clinics and hospitals I've been to. It's probably out there in some pirate database already, all cleaned up and consolidated.

With ChatGPT at least I control what I enter in the prompt to minimize my exposure.