conv.

All stories
AIQuiet 7d · day 8

arXiv paper warns LLM language outputs may not reflect internal computation

A new paper coins 'linguistic illegibility' to describe when a model's stated reasoning diverges from what it actually computes, raising security concerns.

newest coverage 7d agosummary written 7d ago4 pieces82 likes & upvotes31 comments MastodonHacker NewsNewswiresLobsters

What to know

  • A new arXiv paper argues LLMs' stated reasoning (including chain-of-thought) may not reliably reflect their actual internal computation, a phenomenon it calls 'linguistic illegibility'.
  • The paper frames this gap as a security concern, since safety approaches that trust a model's self-reported reasoning could be undermined if that reasoning doesn't match internal processing.
  • Discussion of the paper has been limited across Hacker News and Lobsters, with no named authors or detailed argument surfaced in the available coverage beyond the abstract.

tomjakubowski Hacker News submitterCorbin Lobsters submitter

How it unfolded 2 developments, newest first · click a bar or a number to jump articlesposts

Peak 3 pieces in two hours at Sep 18, 2 PM; 4 pieces over 8 days (1 article · 3 posts) Sep 18, 2 PM — 3 pieces · 1 article · 2 posts — Hacker News 1, Mastodon 1, Newswires 1Sep 18, 4 PM — quietSep 18, 6 PM — quietSep 18, 8 PM — quietSep 18, 10 PM — quietSep 19, 12 AM — 1 piece · 1 post — Lobsters 1Sep 19, 2 AM — quietSep 19, 4 AM — quietSep 19, 6 AM — quietSep 19, 8 AM — quietSep 19, 10 AM — quietSep 19, 12 PM — quietSep 19, 2 PM — quietSep 19, 4 PM — quietSep 19, 6 PM — quietSep 19, 8 PM — quietSep 19, 10 PM — quietSep 20, 12 AM — quietSep 20, 2 AM — quietSep 20, 4 AM — quietSep 20, 6 AM — quietSep 20, 8 AM — quietSep 20, 10 AM — quietSep 20, 12 PM — quietSep 20, 2 PM — quietSep 20, 4 PM — quietSep 20, 6 PM — quietSep 20, 8 PM — quietSep 20, 10 PM — quietSep 21, 12 AM — quietSep 21, 2 AM — quietSep 21, 4 AM — quietSep 21, 6 AM — quietSep 21, 8 AM — quietSep 21, 10 AM — quietSep 21, 12 PM — quietSep 21, 2 PM — quietSep 21, 4 PM — quietSep 21, 6 PM — quietSep 21, 8 PM — quietSep 21, 10 PM — quietSep 22, 12 AM — quietSep 22, 2 AM — quietSep 22, 4 AM — quietSep 22, 6 AM — quietSep 22, 8 AM — quietSep 22, 10 AM — quietSep 22, 12 PM — quietSep 22, 2 PM — quietSep 22, 4 PM — quietSep 22, 6 PM — quietSep 22, 8 PM — quietSep 22, 10 PM — quietSep 23, 12 AM — quietSep 23, 2 AM — quietSep 23, 4 AM — quietSep 23, 6 AM — quietSep 23, 8 AM — quietSep 23, 10 AM — quietSep 23, 12 PM — quietSep 23, 2 PM — quietSep 23, 4 PM — quietSep 23, 6 PM — quietSep 23, 8 PM — quietSep 23, 10 PM — quietSep 24, 12 AM — quietSep 24, 2 AM — quietSep 24, 4 AM — quietSep 24, 6 AM — quietSep 24, 8 AM — quietSep 24, 10 AM — quietSep 24, 12 PM — quietSep 24, 2 PM — quietSep 24, 4 PM — quietSep 24, 6 PM — quietSep 24, 8 PM — quietSep 24, 10 PM — quietYesterday, 12 AM — quietYesterday, 2 AM — quietYesterday, 4 AM — quietYesterday, 6 AM — quietYesterday, 8 AM — quietYesterday, 10 AM — quietYesterday, 12 PM — quietYesterday, 2 PM — quietYesterday, 4 PM — quietYesterday, 6 PM — quietYesterday, 8 PM — quietYesterday, 10 PM — quietToday, 12 AM — quietToday, 2 AM — quietToday, 4 AM — quietToday, 6 AM — quietToday, 8 AM — quietToday, 10 AM — quietToday, 12 PM — quiet 12
Sep 19Sep 20Sep 21Sep 22Sep 23Sep 24yesterdaynow · 1:19 PM ET
  1. 2

    Paper is cross-posted to Lobsters

    User Corbin submitted the same arXiv paper to Lobsters roughly a day after its Hacker News appearance, generating a single comment and a score of 1.

    “We introduce the term *linguistic illegibility* to broadly refer to scenarios in which an LLM's externalized or mechanistically-probed language artifacts fail to represent how the model actually thinks.”
    — paper abstract, arXiv preprint · source
  2. 1

    Paper is shared to Hacker News, drawing modest discussion

    User tomjakubowski submitted the paper to Hacker News, where it reached a score of 63 with 26 comments; the same link was also picked up by an HN Frontpage RSS aggregator and a Mastodon bot account.

  3. background

    Paper introduces 'linguistic illegibility' as an LLM security concern — The arXiv paper argues that LLMs' externalized language outputs and mechanistically-extracted linguistic features can be unreliable indicators of what the model is actually computing internally, coining the term 'linguistic illegibility' for this phenomenon and linking it to security risks.