Study: Chat Templates, Not Self-Awareness, Drive LLMs' "As a Language Model" Voice
A new arXiv paper argues the first-person, self-referential phrasing chatbots use is an artifact of chat-template formatting rather than evidence of genuine self-knowledge.
What to know
- The paper argues that a model's first-person self-descriptions ('As a Language Model...') are shaped by chat-template formatting rather than reflecting any inherent self-knowledge.
- Commenters split on whether framing chatbots with a first-person identity is a manipulative 'dark pattern' or simply a design/UX choice comparable to marketing.
- Practical concern raised: users leaning on LLMs for medical or other high-stakes triage may be poorly served, since models are noted to often get recommended actions wrong.
- Some commenters say the phenomenon is unsurprising and fully explained by post-training and system prompts (e.g., Anthropic's 'constitutional' persona design).
The dispute Whether self-referential chatbot language reflects a deliberate, manipulative engagement strategy by AI companies or is just an unremarkable side effect of training and prompting. · positions read across 15 posts and comments
Boilerplate self-referential disclaimers ('As a Language Model...') are annoying and get in the way of useful answers, especially in high-stakes contexts like health questions.
-
“"As a Language Model..." is one of the beginnings of a sentence I hate the most from LLMs and is the reason why I support free (as in "Liberty"), local models.”
Izmaki · Hacker News ↗
Packaging LLMs as friendly, first-person chatbots is a manipulative design choice meant to encourage anthropomorphization.
-
“The AI companies have chosen to package LLMs as friendly chatbots because they know that will be engaging for humans, but it's manipulative dark pattern.”
ForHackernews · Hacker News ↗
The finding is unsurprising and fully explained by post-training and system prompts, not a deep mystery.
-
“Presumably the fact that they're heavily trained to reply in this way? I don't know about the rest of the paper, but this part sticks out as a really odd claim unless I'm entirely misunderstanding this part.”
LiamPowell · Hacker News ↗
Paper authors Researchers behind the arXiv studyAnthropic AI lab cited in discussionHacker News commentariat Discussion participants
How it unfolded 4 developments, newest first · click a bar or a number to jump articlespostscomments
-
4
Thread turns to risks of relying on LLMs for medical guidance
Commenters dispute how much to trust chatbots' self-caveats and disclaimers when used for symptom triage, with one citing research showing LLMs often recommend the wrong course of action.
“LLMs may be good at identifying a condition based on a description of the symptoms, but they are much worse at recommending the correct course of action (getting it wrong half of the time).”
— Anduia -
'alignment' (in ai corporation speak) is a set of revealed political beliefs. i also have political beliefs. i am an absolutist about alignment.A i am in favor of policies 'aligned' with the following:freedom to live as the person i want to be without fear, shame, surveillance or interference. freedom to make my own decisions. to be empowered as…
2 more of the top 3 · 10 posts in this stretch
-
Be careful there. LLMs may be good at identifying a condition based on a description of the symptoms, but they are much worse at recommending the correct course of action (getting it wrong half of the time).[0]
-
Though I don't wish the world was filled with people like you, remembering that it's not, and it's filled with people that have very little discernment when it comes to higher learning makes it's pretty obvious companies do not want the liability of it's users thinking the technobabble passes for wisdom or experience or intelligence.
-
-
3
Commenter highlights paper's core claim about model self-reports
A Hacker News commenter quotes the paper's finding that what models say about themselves is not a reliable fact about them, and speculates whether induced personas could become stable or portable.
“our work shows that what models say about themselves is not a fact about them…”
— paper abstract (quoted by skybrian) -
"As a Language Model..." is one of the beginnings of a sentence I hate the most from LLMs and is the reason why I support free (as in "Liberty"), local models. I'm well aware that it is not a doctor and cannot replace a real doctor with multiple years of experience, I don't need to waste braincell activity on reading that it "as a Language Model"…
2 more of the top 3 · 3 posts in this stretch
-
> our work shows that what models say about themselves is not a fact about themIt seems like should be obvious given that they can play multiple characters, but it’s good to have more confirmation.Although, I do wonder to what extent these personas might become stable entities. Could personas become portable and spread like memes? It seems like…
-
Very cool innovation in steering - but a lot of introspection only emerges at the highest weight classes - this research would be fascinating to run on bigger models.
-
-
2
Commenters debate whether chatbot personas are a dark pattern
Hacker News users argue about whether packaging LLMs with friendly, first-person identities is deliberately manipulative versus a natural consequence of system design.
“The AI companies have chosen to package LLMs as friendly chatbots because they know that will be engaging for humans, but it's manipulative dark pattern.”
— ForHackernews -
In my view, these models should never be set up to output first-person "experiential" (from the abstract) language. It's too easy to humans to anthropomorphize software that presents itself as having an identity.The AI companies have chosen to package LLMs as friendly chatbots because they know that will be engaging for humans, but it's…
1 more of the top 2 · 2 posts in this stretch
-
> yet what drives them is not well understoodPresumably the fact that they're heavily trained to reply in this way? I don't know about the rest of the paper, but this part sticks out as a really odd claim unless I'm entirely misunderstanding this part.
-
-
1
Story surfaces on Hacker News front page
The arXiv submission is posted to Hacker News and syndicated via the HN Frontpage RSS and a Mastodon bot account, drawing an initial trickle of engagement before a larger comment thread forms.
-
background
Researchers post paper on LLM self-referential voice to arXiv — The paper 'As a Language Model": Chat Template Switches LLM Self-Referential Voice' appears on arXiv, examining how models' first-person self-descriptions shift depending on chat-template formatting.
What people are saying 6 voices from 1 site · best of 15 · verbatim
- Could induced personas become stable or portable entities that spread across contexts like memes, as one commenter speculated?
- Would the introspection effects described in the paper hold up on larger/different model classes?
- Sep 27
-
Oh, I had similar case. First, I used qwen without any ChatML-like syntax and it continued speaking and speaking, then I used <|im_start|>/<|im_end|> to control it somehow
-
So bizarre to see the article refer to outputs as the models referencing "themselves". Computers are not a "them".
-
Always makes me think of that Bill Bailey "as a mother" joke. Similar cringe to those UX "As a user, I want to blah blah" things too. Just say "Users want to be able to blah blah", or better yet make a freaking table.
-
This statement should be restricted to answers for provocative questions. If an LLM is being asked a question that goes against the guidelines, then "As a Language Model..." is a valid starting point. Rest, obviously we're aware that a software doesn't have the judgement that a human has.
-
Neither of your wants are realistic or sensible, at least in the way I think you're presenting them?The first is equivalent to "I don't want my operating system to be used to program viruses."The second is "I don't want vendors to include marketing in their product."
-
Maybe I'm missing something deeper here, but isn't it clear that this is driven by post-training and system prompt? Anthropic's constitutional reinforcement (soul document,etc), for example, is very clear about "who" (not so much what) Claude is supposed to be.