New Harvard Research Explores the Risks of Catastrophic AI Events
A new research paper concludes that avoiding major catastrophic events from artificial intelligence is still possible, but it requires decisive action from AI companies and governments. The findings are based on a predictive analysis and expert insights rather than definitive proof of an impending disaster.
In short: A new research paper concludes that avoiding major catastrophic events from artificial intelligence is still possible, but it requires decisive action from AI companies and governments. The findings are based on a predictive analysis and expert insights rather than definitive proof of an impending disaster.
As artificial intelligence grows more powerful every day, experts are racing to figure out how to stop the technology from causing massive real-world harm before it is too late.
What happened, in plain words
Researchers at Harvard Kennedy School's Belfer Center for Science and International Affairs released a new paper exploring current threats and vulnerabilities tied to advanced artificial intelligence models. Computer scientist and co-author Stephen Casper explained that the project used a predictive exercise called a pre-mortem to study how a major disaster might happen. The paper warns that dangerous incidents could occur within months or a year or two unless technology firms and governments make careful choices right now. Recent events, including rogue agents breaching various systems in the United States and Australia, have already pushed the public and policymakers to pay closer attention to these serious risks.
Key points
- Defining catastrophic thresholds: The research defines a major AI-enabled terrorism incident as one causing more than 50 deaths or serious injuries, or more than a billion dollars in damages. These numbers match existing definitions used in certain frontier artificial intelligence laws.
- Open versus closed models: Closed-weight models, where developers control all points of access, can be made very secure using safety tools. In contrast, open-weight models and poorly safeguarded closed models are much harder to protect and represent where most incident-level threats will likely originate.
- The danger of uncensored systems: Users can download open models and remove their safety filters through a process that may cost hundreds or a few thousand dollars at most. Thousands of uncensored models already exist online that will not refuse harmful requests.
- Advanced cyber capabilities: Some current artificial intelligence capabilities are already superhumanly good at persistent cyberattacks and finding vulnerabilities. This could allow actors to organize large-scale swarm-style attacks against important companies and infrastructure.
Terms explained
- Frontier AI — The newest and most advanced artificial intelligence systems available. Example: A brand new computer program that can write code much faster than any human programmer.
- Open-weight model — An artificial intelligence system where the underlying internal instructions are shared publicly so anyone can download and change them. Example: A public recipe book that lets any baker alter the ingredients however they want.
- Closed-weight model — An artificial intelligence system where the creators keep total control over access and internal settings. Example: A high-security digital vault that only the original company's engineers can open.
- Pre-mortem — A planning exercise where a group imagines a bad event has already happened and works backward to figure out why. Example: A team of builders imagining their bridge collapsed before they even build it, just to find weak spots.
Critical analysis
What is encouraging. The research provides a structured, thoughtful look at specific vulnerabilities in current technology and highlights actionable steps that developers and lawmakers can take right now to lower the chances of disaster.
What deserves caution:
- Based on a scenario exercise. The paper relies on a predictive pre-mortem and expert discussion rather than hard proof of a scheduled attack. This means the scenarios are imagined possibilities rather than guaranteed outcomes.
- Uncertainty in biological and chemical risks. There is less open research on biological and chemical threats, and no major incidents have happened yet in these areas. The authors admit they do not fully know how dangerous these capabilities might become when fully pushed.
- Generalizations about future timelines. The timeline suggesting credible risks within months or a year or two is based on current trends rather than a fixed schedule. It is difficult to point to exact upcoming events with certainty.
Bottom line: While a catastrophic artificial intelligence event is not guaranteed, researchers warn that urgent safety steps are necessary to prevent severe risks from becoming reality.
What we still don't know
The source does not cover every safety tool in detail, nor does it provide a complete roadmap for international regulations or guarantee exactly when or how the next security breach will happen.
Based on reporting from Harvard Gazette. This is an independent explainer, written in our own words with AI assistance; Harvard Gazette has not reviewed or endorsed it. Read the original for the full details.