Early users report significant reasoning errors in demo tool calling
2 Sep 18 12:58 PM · 9d ago · 2 posts · 3 comments · 2 sources · development 2 of 7
Community testing of Needle 3's public demo revealed consistent misrouting of natural language commands. Users documented failures including misinterpreting "I need a wee" as a music request (genre confusion), "warm the house" as a lights brightness adjustment, and "it's cold" triggering unexpected thermostat behavior.
“"I need a wee" -> tries to play music because "wee" is a genre”
IanCalCactus Compute AI model developerHenry Ndubuaku Founder, CactusDeepSeek Competing model vendor (V4 Flash)
The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump
What people said 3 voices · verbatim
-
I have an idea for a use case for this, and I'm wondering if you think it makes sense or if you have any thoughts on the approach.I'm a big fan of OpenStreetMap, and I enjoy editing it from my computer. From my phone, I find it quite tedious trying to make sure I type in the phone number exactly correctly and double-check it, or find and select…
-
Wondered if it'd turn on the lights in the bathroom with these:"I need a wee" -> tries to play music because "wee" is a genre"I need a wee wee" -> starts the vaccuum in the bathroom"I'm going to the toilet" -> says it'll turn on the toilet, and I'm not totally sure what that entails."I'm going to the toilet and can't see" -> reasons that lights…
-
My thought, the growing number of dubious claims that a tiny model beats LLMs will make any useful innovation be overlooked.What's more important than the resource requirements is to highlight what the model simply cannot even attempt to do that general LLMs do decently well.In other words, tell me the anti use case clearly so that I don't have to…
All 7 developments of Cactus releases Needle 3, 8-29MB models for on-device tool… →
Hacker NewsMastodonNewswiresX