Thread turns to risks of relying on LLMs for medical guidance
4 Sep 27 8:07 AM · 1d ago · 6 comments · 1 source · development 4 of 4
Commenters dispute how much to trust chatbots' self-caveats and disclaimers when used for symptom triage, with one citing research showing LLMs often recommend the wrong course of action.
“LLMs may be good at identifying a condition based on a description of the symptoms, but they are much worse at recommending the correct course of action (getting it wrong half of the time).”
AnduiaPaper authors Researchers behind the arXiv studyAnthropic AI lab cited in discussionHacker News commentariat Discussion participants
The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump
What people said 9 voices · best of 10 · verbatim
-
'alignment' (in ai corporation speak) is a set of revealed political beliefs. i also have political beliefs. i am an absolutist about alignment.A i am in favor of policies 'aligned' with the following:freedom to live as the person i want to be without fear, shame, surveillance or interference. freedom to make my own decisions. to be empowered as…
-
Be careful there. LLMs may be good at identifying a condition based on a description of the symptoms, but they are much worse at recommending the correct course of action (getting it wrong half of the time).[0]
-
Though I don't wish the world was filled with people like you, remembering that it's not, and it's filled with people that have very little discernment when it comes to higher learning makes it's pretty obvious companies do not want the liability of it's users thinking the technobabble passes for wisdom or experience or intelligence.
-
This statement should be restricted to answers for provocative questions. If an LLM is being asked a question that goes against the guidelines, then "As a Language Model..." is a valid starting point. Rest, obviously we're aware that a software doesn't have the judgement that a human has.
-
Neither of your wants are realistic or sensible, at least in the way I think you're presenting them?The first is equivalent to "I don't want my operating system to be used to program viruses."The second is "I don't want vendors to include marketing in their product."
-
Maybe I'm missing something deeper here, but isn't it clear that this is driven by post-training and system prompt? Anthropic's constitutional reinforcement (soul document,etc), for example, is very clear about "who" (not so much what) Claude is supposed to be.
-
Always makes me think of that Bill Bailey "as a mother" joke. Similar cringe to those UX "As a user, I want to blah blah" things too. Just say "Users want to be able to blah blah", or better yet make a freaking table.
-
Oh, I had similar case. First, I used qwen without any ChatML-like syntax and it continued speaking and speaking, then I used <|im_start|>/<|im_end|> to control it somehow
-
So bizarre to see the article refer to outputs as the models referencing "themselves". Computers are not a "them".
All 4 developments of Study: Chat Templates, Not Self-Awareness, Drive LLMs' "As… →
MastodonNewswiresHacker News