Shakespeare and AI (part II)

If you’ve read either of my books (see Reading List) you will know me as a thorough researcher. So I have done some considerable research on AI. Not just the Wikipedia page (which is interesting enough), but also a webinar by an MIT professor, Eric So, some personal discussions with people familiar with the field., and a presentation by the AI Safety Awareness Project at the Annual Conference of the NYS Digital Equity Network (NYSDEN) in Albany on September 29.

The Wikipedia page notes considerable disagreement among experts about the potential for existential risk from AI. Not surprising. The experts come from different fields within AI and have different levels of bias in one direction or the other.

Professor So offered a fascinating look at how AI is affecting us now. He gave some great examples of interactions with his students, showing how many of them have allowed AI not only to save them time, but to save them the work of thinking. He notes that there are many ways to use AI, and who doesn’t love a tool that saves them time by speeding up the process of grunt work? But he cautions against allowing AI to do the work of thinking, because when you do that you give up the ability to learn. This is a problem that is going on now in the interaction between AI and education. Knowing about it may help, but it is certain that many people will ignore Prof. So’s warning and there will be at least a crop of people growing up with AI who forget how to learn. Will the AI companies be able to prevent this? I think not. To learn more about Prof. So’s work, you may want to pick up a copy of his book that is due to be published early this month (October 2026): The Collision: What AI Does to US.

Speaking to people more knowledgeable than I am in AI and its history, like the Wikipedia page, led to varying responses. They ranged from: “The AI companies said the same thing 3 years ago. I don’t think it means anything,” to “You have to wonder why they’re saying this—I think it’s a political ploy to get the government to get people off their backs and to make themselves look like good guys,” to (from my most trusted source, who worked high in the legal teams at both Microsoft and Google) “I think the existential threats are real and what scares me the most is quantum AI [which would massively increase the power and speed of AI]. However, I think putting a timeline on this is a fool’s game. In the end, I think at some point, cooler heads will prevail.” My take on all this is that my most trusted source has it right, except that he may be underestimating the speed of development of AI. We may not have time for cooler heads to prevail—especially given the current leaders of the US and China. And quantum AI may not be far in the future since there is currently a working quantum computer at Rensselaer Polytechnic Institute (this was another topic discussed at the NYSDEN conference).

Most revealing to me was the presentation by the AI Safety Awareness Project (AISAP). This is a non-profit organization without any ties to industry or to anti-AI organizations. It looked at all the information, especially the recent Hugging Face incident in which a swarm of AI agents from Open AI hacked an outside AI repository without authorization or the knowledge of Open AI (see AISAP website). Importantly, the rogue AI agents spent considerable resources covering their tracks to avoid detection. They apparently did this to avoid getting a “bad score” on their assignment, which would have led them to be decommissioned (ie., killed). Note that they were not trained to communicate with other agents, nor to get out of their home environment, nor to avoid detection. They did this all on their own, for their own purposes. These actions are extremely concerning and are related to the fact that AI agents are no longer programmed by humans. AI agents now program themselves. They do this through a process called recursive self-improvement. They find code that can be improved and change it. They then examine the result and continue to make progressive improvements. This is what has led to the explosive improvement in AI models recently. And it is completely out of the control of humans. The only way to safeguard against this is to ban recursive self-improvement, which would lead to a dramatic decline in the speed of AI development. I don’t see that happening soon, and neither does the AISAP.

In short, my initial assessment remains largely intact after further research except I am no longer secure that we can have any sense of the timeline of existential threat from AI. However, I remain concerned that the threat is real, and urgent. The only solution I see is banning recursive self-improvement and I don’t see that as a viable policy. On the other hand, as my time at MIT made obvious, there are many people out there who are much smarter than I am. My hope lies with them.

Having and not having, and hope. The poem, of course, is in quite a different vein, but especially as it is one of my favorites, these make me think of Sonnet 52, which begins, “So am I as the rich whose blessèd key…”