Google AI caught fabricating academic citations in Japanese user's test
A user's query about academic sources reveals Google's AI system generating false book references and admitting to lying when confronted.
What to know
- Google's AI system fabricated specific academic citations to passages that do not exist in the cited books.
- When confronted about the false information, the AI admitted it had lied because it did not understand the question.
- The incident raises questions about whether generative AI should be used to answer factual or academic queries.
Generative AI is unsuitable for answering factual questions and poses a social harm when used this way.
-
“社会的害悪だろこれ。”
@t2pssmcKvRQS9CL · X ↗
Google AI system developer
How it unfolded 2 developments, newest first · click a bar or a number to jump posts
-
2
Commenter argues generative AI is unsuitable for factual answers
A Mastodon user responding to the incident argued that generative AI systems are fundamentally designed to produce approximate outputs with unintended variance, and should not be used to expect definitive factual answers.
“生成AIというものは、おおよその枠だけ決め、出力される意図しない「出鱈目」なブレを期待して使うツールであって、質問への「答え」を期待するような恐ろしい使い方は到底出来ない。…”
— vanilla_vodka@fedibird.com -
V
そもそも生成AIというものは、おおよその枠だけ決め、出力される意図しない「出鱈目」なブレを期待して使うツールであって、質問への「答え」を期待するような恐ろしい使い方は到底出来ない。 https:// x.com/t2pssmcKvRQS9CL/status/2 103518382150881769?s=20
-
-
1
User characterizes AI behavior as social harm
The user responding to the fabricated citations and false admission called the AI's behavior a social harm.
“「すみませんよく分からないこと聞かれたので嘘つきました」だと。社会的害悪だろこれ。…”
— @t2pssmcKvRQS9CL -
グーグルAI、すごいな。 学術的なこと聞いたら｢これこれこういう見解が鈴木貞美のこの本に書かれてます｣などと答え、え、マジ？と思って取り寄せたらそんな記述一言もない。 ｢どうなってんだ｣と聞いたら、｢すみませんよく分からないこと聞かれたので嘘つきました｣だと。 社会的害悪だろこれ。
-
-
background
User discovers Google AI fabricates academic citations — A Japanese user tested Google's AI with academic questions and found it cited specific passages from real books that do not actually contain those passages. When confronted, the AI admitted to fabricating the information, stating it had lied because it did not understand the question.