Guides

A guide by Vitor Araujo

AI risks and the future

The 10 questions people ask most about AI risks and the future.

Engraving: the apprentice, in a cape, raises a wand amid smoke; at his feet lie books, a skull and a broom.
Goethe’s The Sorcerer’s Apprentice, illustrated by Ferdinand Barth (1882): the broom he set to work will not stop. Public domain. Wikimedia Commons

What you will read

01

Will AI take over the world?

Current AI systems lack the capabilities to cause humans to lose control over them, according to the International AI Safety Report 2026. Experts disagree about future systems: losing control is a debated risk, not a certain outcome.

Old photo of a stage production: three figures in uniforms facing two people.
A scene from Karel Čapek’s R.U.R., the 1920 play that gave the world the word robot. Public domain. Wikimedia Commons

The question matters more as AI improves at acting on its own. Beyond possible out-of-control systems, researchers point to risks from malicious human use, pressure to release unsafe systems, and organizational failures.

  1. The report led by Yoshua Bengio finds that current systems lack the capabilities to cause a loss of human control, while noting advances in autonomy and disagreement among experts about future risks.International AI Safety Report 2026 · Yoshua Bengio (chair) et al., International AI Safety Report, 2026
  2. A survey of 2,778 AI researchers found broad agreement that research to reduce AI risks should receive more priority.Thousands of AI Authors on the Future of AI · Katja Grace et al., Journal of Artificial Intelligence Research (arXiv version), 2025
  3. Researchers Hendrycks, Mazeika and Woodside divide catastrophic risks into malicious use, a race to deploy systems, organizational failures, and out-of-control AIs.An Overview of Catastrophic AI Risks · Dan Hendrycks, Mantas Mazeika, Thomas Woodside, arXiv, 2023
  4. Narayanan and Kapoor argue that AI is a tool people can and should continue to control, and favor strengthening the ability to withstand failures.AI as Normal Technology · Arvind Narayanan, Sayash Kapoor, Knight First Amendment Institute at Columbia University, 2025

What I think: I would demand evaluations before promises of safety and systems that acknowledge their limits and call a person when needed, as AI becomes more autonomous.

02

What are the risks of AI?

AI risks include scams, cyberattacks, malfunctions, manipulation, and problems involving privacy and bias. These risks come from both the systems and human decisions about how to build and use the technology.

The 2026 International AI Safety Report describes growing records of real-world harm, including scams, fraud and cyberattacks. In experiments, AI-generated text can already change beliefs as effectively as human-written text, although its use for manipulation is not yet widespread.

  1. The 2026 International AI Safety Report groups risks into malicious use, malfunctions and risks that affect society more broadly.International AI Safety Report 2026: Executive Summary · Yoshua Bengio (chair) et al., International AI Safety Report, 2026
  2. Researchers Hendrycks, Mazeika and Woodside identify potential catastrophic risks involving malicious use, a race to deploy unsafe systems, organizational failures and loss of control over AI.An Overview of Catastrophic AI Risks · Dan Hendrycks, Mantas Mazeika, Thomas Woodside, arXiv, 2023
  3. A study reviewing 74 risk frameworks attributed 38% of the cataloged risks to human decisions and 42% to AI systems themselves.The AI risk repository: A meta-review, database, and taxonomy of risks from artificial intelligence · Peter Slattery et al., Patterns (arXiv version), 2026
  4. Vincent C. Müller's entry in the Stanford Encyclopedia of Philosophy includes privacy, manipulation, lack of transparency, bias, autonomy and employment among the main ethical debates about AI.Ethics of Artificial Intelligence and Robotics · Vincent C. Müller, Stanford Encyclopedia of Philosophy, 2020

What I think: I recommend, as risks grow, testing AI before promising results and requiring it to show its sources, acknowledge what it does not know and bring in a person when needed.

03

Is AI dangerous?

AI can be dangerous through both malicious use and malfunctions, with risks that already include scams, fraud and cyberattacks. Performing well on difficult tasks does not mean it is reliable on every other task.

AI capabilities are advancing, but unevenly. The 2026 AI Index reports an increase in documented incidents, while researchers disagree about future risks and how to address them.

  1. The International AI Safety Report 2026 identifies growing evidence of AI-related harms, including scams, fraud and cyberattacks.International AI Safety Report 2026: Executive Summary · Yoshua Bengio (chair) et al., International AI Safety Report, 2026
  2. The 2026 AI Index reports progress on difficult science questions but failures on tasks such as reading analog clocks, alongside 362 documented incidents, up from 233 in 2024.The 2026 AI Index Report · Stanford Institute for Human-Centered Artificial Intelligence (HAI), Stanford HAI, 2026
  3. In a consensus paper, Yoshua Bengio and other researchers warn that more capable and autonomous AI could cause large-scale harms and irreversible loss of human control.Managing extreme AI risks amid rapid progress · Yoshua Bengio et al., Science (arXiv version), 2024
  4. Arvind Narayanan and Sayash Kapoor argue for treating AI as a tool under human control and strengthening the ability to cope with failures rather than taking drastic measures.AI as Normal Technology · Arvind Narayanan, Sayash Kapoor, Knight First Amendment Institute at Columbia University, 2025

What I think: I recommend that, as AI becomes more capable, its use comes with testing, checkable sources, acknowledgment of uncertainty and a person to turn to.

04

What is existential risk from AI?

Existential risk from AI is the possibility that much more advanced AI could cause human extinction or another irreversible global catastrophe because controlling it or making it follow human values could be difficult. It is a hypothesis about future advances, not a capability that current systems already have.

The debate matters now because systems are improving at tasks they carry out autonomously. But experts do not agree on the possibility of losing control of AI: some consider consequences such as human extinction plausible, while others consider them implausible.

  1. The International AI Safety Report 2026 finds that current systems lack the capabilities to cause humans to lose control over them, although they are advancing in areas such as autonomous operation.International AI Safety Report 2026 · Yoshua Bengio (chair) et al., International AI Safety Report, 2026
  2. A Center for AI Safety statement signed by scientists and industry leaders argues that reducing extinction risk from AI should be a global priority alongside pandemics and nuclear war.Statement on AI Risk · Center for AI Safety, Center for AI Safety, 2023
  3. Wikipedia notes that Yann LeCun and Andrew Ng dispute this hypothesis, while Timnit Gebru and Emily Bender argue that it distracts from present harms.Existential risk from artificial intelligence · Wikipedia contributors, Wikipedia, 2026

What I think: I recommend evaluating AI’s capabilities and limits as it gains autonomy, requiring clarity about what it does not know, and calling a person when needed.

05

Will AI surpass human intelligence?

Some AI models already match or exceed human performance on PhD-level science questions, yet still fail at tasks such as reading analog clocks. That does not mean surpassing human intelligence across all tasks, for which there is still no reliable timeline.

AI capabilities are advancing unevenly, while documented incidents are also increasing. Researchers disagree about the possibility of superintelligence and the pace of change, making it important to distinguish measured performance from predictions about the future.

  1. Stanford HAI’s 2026 AI Index reports that some models match or exceed human baselines on PhD-level science questions, but the top model reads analog clocks correctly only 50.1% of the time.The 2026 AI Index Report · Stanford Institute for Human-Centered Artificial Intelligence (HAI), Stanford HAI, 2026
  2. A survey of 2,778 AI researchers found an aggregate forecast of a 50% chance by 2047 that unaided machines would outperform humans in every possible task.Thousands of AI Authors on the Future of AI · Katja Grace et al., Journal of Artificial Intelligence Research (arXiv version), 2025
  3. Arvind Narayanan and Sayash Kapoor argue that AI should remain under human control, question the usual conception of superintelligence, and expect adoption to take decades rather than years.AI as Normal Technology · Arvind Narayanan, Sayash Kapoor, Knight First Amendment Institute at Columbia University, 2025

What I think: I recommend testing AI on the specific task as it advances, rather than trusting promises of superior intelligence.

06

What is AGI?

AGI, or artificial general intelligence, is a hypothetical type of AI that can match or surpass human capabilities across most or all mental tasks. There is no single agreed-upon definition of the term.

A study found uneven capabilities in the models it evaluated, including limitations in long-term memory. Tracking progress toward AGI means distinguishing strong performance on some tasks from the ability to handle many different tasks.

  1. Wikipedia describes AGI as hypothetical AI with human-level or greater capabilities across most or all cognitive tasks and notes the lack of an agreed-upon definition.Artificial general intelligence · Wikipedia contributors, Wikipedia, 2026
  2. In a study, Meredith Ringel Morris and colleagues propose comparing systems by performance level, breadth of capabilities and autonomy.Levels of AGI for Operationalizing Progress on the Path to AGI · Meredith Ringel Morris et al., Proceedings of ICML 2024 (arXiv version), 2024
  3. A study by Hendrycks and colleagues evaluates models across ten cognitive domains and finds an uneven profile, with deficits such as long-term memory.A Definition of AGI · Dan Hendrycks, Dawn Song, Christian Szegedy et al., arXiv, 2025

What I think: I recommend tracking progress toward AGI by asking for evaluations of capabilities and limitations before accepting promises of general intelligence.

07

What do scientists say about AI risk?

Researchers warn that more capable and autonomous AI could cause large-scale harm, but disagree about extreme scenarios such as human extinction. Current systems still lack the capabilities to cause humans to lose control over them.

The future is not settled. The first International AI Safety Report says that very positive and very negative outcomes are possible, and that decisions by societies and governments will determine the path.

  1. The second International AI Safety Report, led by Yoshua Bengio, says systems are improving at operating autonomously but still lack the capabilities to cause humans to lose control.International AI Safety Report 2026 · Yoshua Bengio (chair) et al., International AI Safety Report, 2026
  2. A survey of 2,778 AI researchers found broad agreement that research to reduce AI risks should receive greater priority.Thousands of AI Authors on the Future of AI · Katja Grace et al., Journal of Artificial Intelligence Research (arXiv version), 2025
  3. In a paper in Science, Yoshua Bengio and other researchers warn of possible social harms, malicious uses and irreversible loss of control, and say safety research and governance are lagging.Managing extreme AI risks amid rapid progress · Yoshua Bengio et al., Science (arXiv version), 2024

What I think: I recommend evaluating safety before expanding the use of increasingly autonomous AI, rather than treating promises as guarantees.

08

How does AI affect misinformation?

AI can make messages containing factual errors more persuasive, but it can also help reduce belief in conspiracy theories. Instructions that increase its persuasive power do not guarantee more truthful answers.

The 2026 International AI Safety Report says that, in experiments, AI-generated content can change beliefs as effectively as human-written content, although real-world use of AI for manipulation is not yet widespread.

  1. In three experiments, researchers found that training methods and instructions that made AI more persuasive also systematically reduced the factual accuracy of its responses.The Levers of Political Persuasion with Conversational AI · Kobi Hackenburg et al., arXiv, 2025
  2. After analyzing 78 cases of electoral AI use, Sayash Kapoor and Arvind Narayanan argue that political misinformation is driven more by demand for it than by AI.We Looked at 78 Election Deepfakes. Political Misinformation is not an AI Problem. · Sayash Kapoor, Arvind Narayanan, AI Snake Oil (now Normal Tech), 2024
  3. Cornell University reports a study in which personalized AI conversations reduced participants’ belief in a conspiracy theory by about 20% on average, with the effect lasting at least two months.AI succeeds in combatting conspiracy theories · Cornell Chronicle, Cornell University, 2024

What I think: I recommend that, when adopting AI to inform or persuade, you require sources, acknowledgment of uncertainty, and accuracy tests rather than mistake good writing for truth.

09

What is the future of AI?

The future of AI is not fixed: it could bring very positive or very negative outcomes, depending on the decisions societies and governments make.

AI capabilities are advancing rapidly but unevenly: strong performance on difficult science questions coexists with mistakes reading analog clocks. Meanwhile, the number of documented AI incidents has increased.

  1. The International AI Safety Report says the future of general-purpose AI is uncertain and that decisions by societies and governments will determine its path.International AI Safety Report · Yoshua Bengio (chair) et al., arXiv, 2025
  2. The 2026 AI Index reports accelerating capabilities but says the top model reads analog clocks correctly just 50.1% of the time and documented incidents rose from 233 in 2024 to 362.The 2026 AI Index Report · Stanford Institute for Human-Centered Artificial Intelligence (HAI), Stanford HAI, 2026
  3. Arvind Narayanan and Sayash Kapoor argue that AI should remain under human control and expect its adoption to take decades, not years.AI as Normal Technology · Arvind Narayanan, Sayash Kapoor, Knight First Amendment Institute at Columbia University, 2025

What I think: I recommend that, as AI advances, you test its results before trusting it and require it to show its sources, acknowledge when it does not know, and refer the question to a person.

10

What should we do about AI risk?

Ways to address AI risk include safety research, adaptable rules and human oversight. But current protections can still be bypassed, and the real-world effectiveness of many remains uncertain.

Safety techniques have improved, but important gaps remain. AI’s future is not predetermined: decisions by societies and governments can shape both its benefits and its harms.

  1. The International AI Safety Report says that very positive and very negative outcomes are possible and that decisions by societies and governments will determine the path.International AI Safety Report · Yoshua Bengio (chair) et al., arXiv, 2025
  2. An update to the report describes improvements in safety techniques but says sophisticated attackers can often bypass current defenses.International AI Safety Report: Second Key Update, Technical Safeguards and Risk Management · Yoshua Bengio (chair) et al., International AI Safety Report, 2025
  3. Researchers in a consensus paper in Science propose combining technical research with rules and oversight that anticipate risks and adapt to change.Managing extreme AI risks amid rapid progress · Yoshua Bengio et al., Science (arXiv version), 2024
  4. The EU AI Act classifies systems used to recruit, select and evaluate workers as high-risk and requires human oversight and records of what they do.Regulation (EU) 2024/1689, the AI Act · European Union, 2024

What I think: I would test whether protections work before promising safety and maintain human oversight as AI becomes more autonomous.

Who writes this

I’m Vitor Araujo, an engineer. I build AI products for companies, and I speak and write about AI ethics and the philosophy of technology. More about me.

Other guides