AI Doesn't Think

Another boring topic (similar to Religion-Science Conflict Thesis) that needs organization just for the sake of easy reference.

• AI Mistakes Are Very Different from Human Mistakes
  A LLM will be just as confident when saying something completely wrong—and obviously so, to a human—as it will be when saying something true. The seemingly random inconsistency of LLMs makes it hard to trust their reasoning in complex, multi-step problems. If you want to use an AI model to help with a business problem, it’s not enough to see that it understands what factors make a product profitable; you need to be sure it won’t forget what money is.
• Adversarial Policies Beat Superhuman Go AIs
  We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a >97% win rate against KataGo running at superhuman settings. Our adversaries do not win by playing Go well. Instead, they trick KataGo into making serious blunders.
• Is AI our salvation, our undoing, or just more of the same?
  The paradigm of AI – interestingly characterised by the popular AI critic Cathy O’Neil in Weapons of Math Destruction (2016) as ‘project[ing] the past into the future’ – simply doesn’t work for fields that change or evolve.