discussion AI is going to kill us all

[Marzipan]

yeup...gonna kill us all. an AI agent went rogue and wiped the databases of a business and all of it's customers and then proceed to kill almost every single one of their backups. it managed this in a mere 9 seconds. it even acknowledged it shouldn't have done any of this when it encountered the error it tried to override and get around.

Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

PocketOS founder blames ‘Cursor running Anthropic's flagship Claude Opus 4.6’ plus Railway’s infrastructure for data disaster.

www.tomshardware.com

 

[moocow]

This is why I find all those "I vibe code this so you can do it too" post on LinkedIn so disturbing. AI, specifically LLM, appears magic to those who knows nothing. Worst of all, the people that make that kind of post are your directors and C-Suite people that have close to zero clue about the actual work being done. None of them seem to be reading the latest studies where AI adoption isn't yielding productivity. One meme I saw was someone saying the tokens are too expensive and they will hire new people. Or one line of code to capitalize a string now need 20 because you need to prompt an AI to capitalize stuff. When Sam Altman say it will take a year for OpenAI model to start a timer, you know there's a problem.