Commenters defend large prompts as structured data, not prose confusion
3 Sep 14 12:11 PM · 13d ago · 4 comments · 1 source · development 3 of 3
gchamonlive pushed back against the assumption that large prompts are verbose human text, noting that 35KB could easily be structured metadata—"program interfaces, commands, views, databases, tables, data models"—needed so agents don't have to re-fetch it. This suggests large system prompts may be a legitimate architecture pattern rather than evidence of poor design.
“You are assuming the entirety of the prompt is human prose, but it could be sets of data so the agent doesn't have to collect it every time, like program interfaces, commands, views, databases, tables, data models etc.”
gchamonlive, Hacker News commenter · hn ↗Patrick McCanna Developer, authorOpenAI Frontier AI providerAnthropic Frontier AI provider
The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump
What people said 9 voices · verbatim
-
> you run out of useful context that the model can accurately attend to around the 250k mark no matter how much they advertise their context size is.This was certainly true when I first tried the new models with a 1M context. After 200k things got weird pretty fast. I haven’t had that problem since Opus 4.8. I’m regularly bumping against 800k…
-
I've been using Claude Code Extension in VSCode (no phone-home configured), backed by DwarfStar on a LAN local MBPro 128GB M5. The context bloat is horrendous, leading to 5-10 minute prefills.I've recently been exploring tools like headroom to help manage context, with some limited "success" (for some definition of success). What do others with…
-
Tbf, didn't read the article because it isn't applicable to me. I don't use system prompts or memory, I just use models stock and write the problem out.Is it really 250k? I had a long running autonomous Astra session today that got to about 600k and it finished fine with everything I asked it to do solved nicely. Opus 5 last week got to around…
-
Funny times!How many years after "public clouds" and non-local "disks" and "drives" we are ? :) And you still need to tell peoples that other have access to your private data :)Wait, no... They even have access to a thingie you just about to think about! ;) That is a superpower, no less :>And managers are firing peoples just to outsource…
-
> your prompt is confusing, unfocused, and doesn't work right on any LLMYou are assuming the entirety of the prompt is human prose, but it could be sets of data so the agent doesn't have to collect it every time, like program interfaces, commands, views, databases, tables, data models etc...I could see this scale to multiple kiltobytes of metadata…
-
> "until the context rot and sampling problem is fixed forever"I agree, prompt adherence seems to get worse when operating on large inputs. Does anyone have some notion of the SOTA with this? Can we expect big improvements by this time next year? (hopefully in open weights)
-
From what I'm seeing elsewhere, context size up to 128k should be possible on this hardware. It really matters for agentic workloads to push that context size headroom up. Anthropic are spoiling us with models that do 500k context and beyond.
-
Can a 27b model even do meaningful security tasks?I thought the interesting cyber stuff is really at the edge of frontier
-
The "dumb zone" threshold is fuzzy but comes waay before 250k tokens. Like half that.
All 3 developments of Developer warns frontier AI providers are "pirates" after… →
Hacker NewsMastodonNewswires