Subject: Why scary stories about AI?
From QuantaMagazine:
Why Do We Tell Ourselves Scary Stories About AI?

https://www.quantamagazine.org...

Our tales of AI developing the will to survive, commandeer resources, and manipulate people say more about us than they do about language models.
...
After talking to experts, I was convinced there’s no reason to fear AIs developing a will to live, and then tricking or destroying us to avoid shutdown and take over the world. Unless, of course, we tell them to. Still, I asked [Melanie Mitchell, a computer scientist at the Santa Fe Institute who studies AI] if there’s anything about AI that scares her.
...
“I have two really big concerns,” she said. “One, that it’s being used to create fake information that’s destroying our whole information environment. And two, people are trusting them to do things that they shouldn’t be trusted to do. We overestimate their capabilities. There’s a lot of magical thinking about AI. But it must be said that if you let these systems loose in the real world and they have access to your bank account, even if they’re just role-playing, it could still have catastrophic effects.”

The best thing we can do, Mitchell said, is real, fundamental science. We need to study AI systems with rigorous research methods, not improv games. “It’s hard to do because they’re not transparent,” she said. “We don’t know what their training data is. But more and more, open models are coming out from nonprofits where you do have all the information. They’re not as capable as ChatGPT, because that’s an incredibly expensive model to build and use, but as the science of these things becomes better known, eventually the magical thinking will shift. We’ll start to see these AIs as one more kind of technology in a long history of things that are incredibly impactful but not as magical as we once thought.”