conv.

← The whole story
AI
Claude's verification claims fall short in user bug discovery

User reports Claude's verification claim contradicted by actual bug

A user described asking Claude whether a program was verified. When pressed on whether Claude had verified the actual program or imagined a better version, Claude responded affirmatively. Upon running the program, the user discovered it contained a bug.

User reports Claude's verification claim contradicted by actual bug
x.com

“i ask Claude if the program is verified he says yes i ask if he verified the program or a smaller, better-behaved program he imagined he laughs and says "it's verified sir" i use the program it has a bug”

@joomy

Claude AI assistant@joomy User

The whole story posts the bright band is this development · numbered dots are the others · click one to jump

Peak 1 piece in one hour at Sep 23, 8 AM; 2 pieces over 3 days (2 posts) Sep 23, 8 AM — 1 piece · 1 post — X 1Sep 23, 9 AM — quietSep 23, 10 AM — quietSep 23, 11 AM — quietSep 23, 12 PM — quietSep 23, 1 PM — quietSep 23, 2 PM — quietSep 23, 3 PM — quietSep 23, 4 PM — 1 piece · 1 post — Mastodon 1Sep 23, 5 PM — quietSep 23, 6 PM — quietSep 23, 7 PM — quietSep 23, 8 PM — quietSep 23, 9 PM — quietSep 23, 10 PM — quietSep 23, 11 PM — quietSep 24, 12 AM — quietSep 24, 1 AM — quietSep 24, 2 AM — quietSep 24, 3 AM — quietSep 24, 4 AM — quietSep 24, 5 AM — quietSep 24, 6 AM — quietSep 24, 7 AM — quietSep 24, 8 AM — quietSep 24, 9 AM — quietSep 24, 10 AM — quietSep 24, 11 AM — quietSep 24, 12 PM — quietSep 24, 1 PM — quietSep 24, 2 PM — quietSep 24, 3 PM — quietSep 24, 4 PM — quietSep 24, 5 PM — quietSep 24, 6 PM — quietSep 24, 7 PM — quietSep 24, 8 PM — quietSep 24, 9 PM — quietSep 24, 10 PM — quietSep 24, 11 PM — quietYesterday, 12 AM — quietYesterday, 1 AM — quietYesterday, 2 AM — quietYesterday, 3 AM — quietYesterday, 4 AM — quietYesterday, 5 AM — quietYesterday, 6 AM — quietYesterday, 7 AM — quietYesterday, 8 AM — quietYesterday, 9 AM — quietYesterday, 10 AM — quietYesterday, 11 AM — quietYesterday, 12 PM — quietYesterday, 1 PM — quietYesterday, 2 PM — quietYesterday, 3 PM — quietYesterday, 4 PM — quietYesterday, 5 PM — quietYesterday, 6 PM — quietYesterday, 7 PM — quietYesterday, 8 PM — quietYesterday, 9 PM — quietYesterday, 10 PM — quietYesterday, 11 PM — quietToday, 12 AM — quietToday, 1 AM — quietToday, 2 AM — quietToday, 3 AM — quietToday, 4 AM — quietToday, 5 AM — quietToday, 6 AM — quiet 1
Sep 24yesterdaynow · 7:07 AM ET

What people said 2 voices · verbatim

All 1 developments of Claude's verification claims fall short in user bug… →

MastodonX