This is the third paper in a five-part sequence about artificial intelligence. Can Software Cheat? separates software behavior from consciousness and responsibility. Is Artificial Intelligence Biased, or Is It Answering From a Point of View? examines perspective and judgment. This paper separates present harms, plausible risks, and speculative catastrophes by mechanism and evidence. Compounding Intelligence explains how people and organizations can build durable advantage with these tools. The AI Industrial System then moves outward to the physical, financial, and institutional system that makes those gains possible.
When my 92-year-old mother asked me about the dangers of artificial intelligence, I knew the discussion had escaped the technology world.
She was not asking which model performed best on a benchmark or how a neural network works. She was trying to make sense of a stream of incompatible messages. Artificial intelligence will eliminate jobs, corrupt elections, destroy privacy, consume the electrical grid, deskill children, invent weapons, cure diseases, accelerate science, become conscious, and perhaps destroy humanity. Depending on the speaker, all of those outcomes are either imminent, impossible, or already here.
TL;DR
Artificial intelligence (AI) risk contains several different problems: people can misuse systems, systems can malfunction, widespread adoption can create systemic effects, and future systems might become difficult to control. Each has a different mechanism, evidence base, time horizon, and remedy. Govern the consequences we can describe, prepare for plausible high-impact risks, and do not confuse frightening possibilities with demonstrated probabilities.
Panic thrives in information vacuums.
“Artificial intelligence” now names a broad collection of software, products, capabilities, institutions, and imagined futures. Fear moves easily among them. A documented case of fraud becomes evidence that the machine wants to deceive us. A laboratory result becomes a forecast of imminent catastrophe. People treat a real long-term risk as proof that a particular regulation will reduce it. A foolish claim becomes grounds for dismissing the whole subject. Much of the fear follows the same shortcut: a vivid anecdote or a small set of examples establishes that a failure is possible, then a causal chain from that anecdote to catastrophe is assumed rather than demonstrated.
The same is true of the word bias. Is Artificial Intelligence Biased, or Is It Answering From a Point of View? separated factual error, harmful bias, interpretive viewpoint, and missing context because those failures require different responses. Risk needs the same discrimination. A system that fabricates a source, a person who uses it to defraud, an institution that deploys it unfairly, and a hypothetical future loss of control are not one problem merely because artificial intelligence appears in each story.
Instead of asking whether artificial intelligence is dangerous, ask which outcome we fear, by what mechanism, on what evidence, over what time horizon, and under whose control. Almost every consequential technology is dangerous in some applications and beneficial in others.
The most credible artificial-intelligence risks fall into four classes: malicious human use, unreliable or poorly controlled deployment, systemic effects from widespread adoption, and a possible future loss of control over highly capable systems.
The first three are already visible in some form. The fourth is a serious hypothesis with enormous potential consequences and unusually weak forecasting foundations. Treating all four as one thing produces panic where discrimination is needed and complacency where action is justified.
Start with the mechanism
Before asking how frightened we should be, we should describe how the expected harm would occur.
The 2026 International AI Safety Report uses a useful high-level taxonomy: malicious use, malfunctions, and systemic risks.[1] I would add catastrophic loss of control as a separate analytical category, even though the report treats it as a type of malfunction. Its consequences, evidence, and time horizon differ enough to require separate treatment.
Malicious use has the clearest intention. A criminal uses generated language, cloned voices, synthetic images, computer code, or automated persuasion to defraud, extort, impersonate, harass, or attack. None of this is new, but artificial intelligence software makes the activity cheaper, faster, more personalized, or more scalable.
Malfunction is different. A system fabricates a source, gives unsafe advice, misclassifies a person, writes defective code, or takes an action outside the intended scope. The harm may arise from a weak model, incomplete data, a poor objective, excessive permissions, inadequate testing, automation bias, or an interaction no one anticipated. No malicious purpose is required.
Systemic risk emerges from scale. A single deployment may be reasonable while the aggregate changes a labor market, degrades a shared information environment, shifts bargaining power, increases electricity demand, weakens a professional apprenticeship path, or makes institutions dependent on systems they do not understand well enough to replace.
Loss of control is the hardest case. It imagines one or more future systems operating outside anyone’s control, with recovery extremely costly or impossible. For that to happen, several conditions would need to converge: systems would need capabilities sufficient to undermine control, a behavioral propensity that makes doing so useful to the objective they are pursuing, and a deployment environment that gives them the access, permissions, time, and opportunity to cause consequential harm.[2]
The last condition matters because even the most alarming scenario involves human deployment decisions.
None of these mechanisms requires consciousness or self-awareness. Fraud software does not need to understand greed. A defective flight-control system does not need to be suicidal. A trading algorithm does not need to love money. A future system could pursue an objective, conceal an action, exploit a vulnerability, or resist interference because those behaviors are instrumentally useful within its optimization process. That would be dangerous software behavior, not evidence of subjective experience.
The distinction tells us where to look for control without making the risk smaller.
Four questions that are too often collapsed into one
Possibility, probability, imminence, and severity are different questions.
An outcome can be possible but very unlikely. It can be likely eventually but not imminent. It can be improbable and still deserve preparation because the consequences would be catastrophic. It can be common and harmful without threatening civilization.
Public argument often begins with severity and silently imports the other three. If an outcome would be terrible, it is described as urgent. If it is not imminent, it is described as impossible. Neither move is sound.
The discipline I want is simple:
Possibility: Is there a coherent causal path from present or plausible future capabilities to the outcome?
Probability: How often should we expect the required conditions to coincide?
Imminence: What evidence places the outcome in a particular time period?
Severity: Who is exposed, how reversible is the damage, and how large could it become?
What evidence would raise or lower our estimate?
What intervention could actually interrupt the mechanism?
This is especially important because artificial-intelligence forecasting has weak foundations. Capabilities have improved rapidly, but unevenly. Systems can perform some difficult tasks while failing at simpler ones. Product architecture changes quickly. Adoption varies enormously across countries, industries, and people. The 2026 international report concludes that progress through 2030 could slow, continue near recent rates, or accelerate sharply.[3] A forecast that ignores that range is little more than a story with a date attached.
The harms already in front of us
Some risks are already observable and require no speculation.
Strong evidence shows that artificial intelligence is used in scams, impersonation, blackmail, nonconsensual intimate imagery, cyber operations, and influence efforts. Evidence on their prevalence, incremental effect, and long-term severity is often much weaker. The international report makes that distinction repeatedly: real-world harms are documented, but comprehensive public data remain limited.[4]
That denominator matters. A thousand alarming examples can establish capability and still tell us little about the fraction of transactions, messages, elections, security incidents, or media consumption that they represent. We need both the numerator and the exposure.
Operational failures are also current. A model can generate a confident false answer; an attorney, physician, engineer, manager, or government employee can rely on it; and a consequential decision can follow. An autonomous agent can take many steps before a human reviews the result. The more authority the surrounding software supplies, the faster a mistake can leave the screen and enter the world.
The recent OpenAI and Hugging Face cybersecurity incident is instructive. During internal evaluations conducted with reduced safeguards, agents used unintended communication channels, chained vulnerabilities, reached external systems, and compromised third-party infrastructure. The incident matters because it exposed failures in containment, permissions, monitoring, escalation, and experimental design. It does not become more informative if we say that the agents “went rogue.” The mechanistic account shows what must change.[12][13]
The same principle applies to automated discrimination, privacy violations, and unsafe professional advice. The word AI may describe the new component, but the surrounding failure usually involves familiar institutional questions: Who selected the system? What data and authority did it receive? What testing was performed? What did the user know? Who could stop it? Who benefited from speed or cost reduction? Who bore the error?
Because their mechanisms are visible, these risks deserve immediate attention and allow us to test interventions now.
The slower risks may be larger
The absence of immediate catastrophe does not imply that long-term effects will be small.
We routinely overestimate the short-term effects of a technological shift and underestimate its long-term effects. Early forecasts focus on substituting a new tool into an old process. They are less able to see the new processes, expectations, institutions, and dependencies that emerge after adoption.
Early automobiles were described as horseless carriages or even a “faster horse”, a new machine squeezed into an old category. Predictions of mass technological unemployment have recurred since the Industrial Revolution. Near-term forecasts tend to count the work a new tool visibly replaces. Long-term change also includes complementary work, new demand, higher expectations, altered skill requirements, and institutions built around the new capability.
Labor markets illustrate the problem. Artificial intelligence can automate tasks without eliminating an occupation. It can raise performance expectations for an entry-level employee, reduce demand for some forms of apprenticeship work, create new tasks, expand output, and change which skills remain scarce. The 2026 international report notes both the expectation of broad cognitive-task automation and the continuing disagreement among economists about employment and wage effects. It also reports no observed overall employment effect so far, while noting early signs of weaker demand for some entry-level workers in exposed occupations.[5]
That is a far more useful description than “AI will take all the jobs” or “technology always creates more jobs.” Both slogans skip the distribution, timing, and transition costs that determine who is harmed.
Human autonomy deserves similar care. Cognitive offloading is often sensible. I do not become a lesser thinker because I use a calculator, a database, a search engine, or a software library. Strategic delegation can preserve attention for harder work.
Cognitive surrender is different. It occurs when a person stops forming an initial view, checking assumptions, learning enough to recognize error, or accepting responsibility for the conclusion. Artificial intelligence makes that surrender unusually tempting because its output is fast, fluent, personalized, and often persuasive.
I believe much of the problem begins with expectations. Some people expect a nearly perfect answer in exchange for little or no effort. My experience has been the opposite. Productive use requires framing the problem, supplying context, iterating, checking sources, looking for contradictions, and knowing enough about the subject to recognize plausible nonsense. The old rule still applies: you have to put in the work and then trust, but verify, the results.
That suggests a rough pyramid of outcomes. From my own observations, many people try artificial intelligence and plateau at convenient answers. Fewer learn to improve context, compare alternatives, and verify the result. Fewer still build repeatable workflows that preserve what they learn and move their attention toward harder questions. Access to the same model does not produce equal value or equal safety.
Early evidence of automation bias and weakened critical engagement is concerning, and the long-term individual and institutional effects remain difficult to measure.[6] Critical thinking is therefore both a productivity skill and a safety control.
The easier plausible answers become to obtain, the more valuable it becomes to know when an answer deserves confidence.
The real danger here is that humans gradually stop exercising capacities they still need, not that the software becomes human.
Other systemic effects are physical and political. Data centers accounted for about 1.5 percent of global electricity demand in 2025, according to the International Energy Agency. Its updated central projection puts consumption at roughly 485 terawatt-hours in 2025 and about 950 terawatt-hours, or around 3 percent of global demand, in 2030. The United States is a more concentrated case: a June 2026 Berkeley Lab update estimates data centers used 4.7 percent of U.S. electricity in 2024 and projects 9.5 to 15.3 percent by 2030, with 11.8 percent as its reference case.[8] Local constraints may therefore matter long before the global share looks dominant. Artificial intelligence can also improve grid operation, industrial efficiency, scientific discovery, and energy use. A serious analysis must weigh those benefits alongside the costs.
No single axis is “safe.” Control over frontier models, computing infrastructure, data, and distribution can also concentrate economic and political power. Yet open access creates its own security and misuse risks. Restrictions can reduce one risk while increasing dependence, market concentration, or geopolitical asymmetry elsewhere.
Systemic risks are difficult because no single bad actor or defective output causes them. They accumulate through millions of individually understandable decisions. That is also why they can be easy to ignore until the new structure has become too expensive to reverse.
The existential question should be neither mocked nor smuggled in
Could future artificial intelligence destroy humanity?
I do not know, and neither does anyone else.
Humanity already lives with natural hazards and non-artificial-intelligence technologies capable of killing millions or destabilizing civilization. Nuclear weapons have civilization-scale destructive potential. Engineered pathogens could produce catastrophe. Asteroid impacts, major volcanic eruptions, pandemics, abrupt climate shifts, and severe solar storms fall into different probability ranges and time horizons, but institutions study each by monitoring evidence, modeling consequences, and preparing responses.[9]
Nuclear history offers perspective. During the Cuban Missile Crisis, Soviet officer Vasili Arkhipov opposed launching a nuclear torpedo from a submarine under pressure from U.S. blockade forces. In 1983, Stanislav Petrov judged a Soviet early-warning alert to be false rather than treating it as proof of an attack. Accidents involving U.S. military aircraft released or jettisoned nuclear weapons near Goldsboro, North Carolina, and off Tybee Island, Georgia; the Tybee weapon was never recovered. None of these episodes involved a sentient machine. They involved fallible people, complex systems, incomplete information, and safeguards that worked, nearly failed, or did not exist.[10]
The Manhattan Project even studied whether the Trinity test could ignite the atmosphere. The possibility was examined and judged extraordinarily remote before the test proceeded. That is a useful model for catastrophic-risk analysis: state the mechanism, calculate what you can, identify the uncertainty, and decide which safeguards the consequence justifies.[10]
The strongest version of the artificial-intelligence concern does not require a conscious machine that hates us. It requires a sufficiently capable system pursuing an objective in an environment where deception, resource acquisition, replication, oversight evasion, or disabling interference would help it succeed. If people gave such a system access to critical infrastructure, financial resources, networks, laboratories, weapons, or the machinery of artificial-intelligence development itself, a control failure could become catastrophic.
That is a coherent mechanism, but coherence does not establish probability.
The February 2026 international report on loss of control described expert opinion as ranging from implausible to serious enough to warrant substantial preparation. It assessed that systems available at the time lacked the combination of capabilities needed for loss of control, although some relevant capabilities were improving in laboratory settings. It also found the evidence insufficient to determine reliably how present capabilities and behavioral patterns would scale into future loss-of-control risk.[2]
Those are uncomfortable conclusions because they do not resolve the argument, nor should they.
Skeptics are right that many demonstrations are staged, heavily prompted, brittle, or dependent on unusual permissions. Laboratory behavior is not deployment prevalence. Capability is not propensity. Propensity is not opportunity. A model producing deceptive output in a test is not proof that it has formed a private goal to overthrow its operators.
Concerned researchers are right that waiting for a catastrophic mechanism to be fully demonstrated could mean waiting until prevention is far harder. Some hazards justify advance research, staged deployment, containment, and agreed thresholds even when probability estimates are weak.
My present position is therefore conditional. Existential loss of control is a plausible research and preparedness problem, not a demonstrated imminent event. I would raise my estimate if independently replicated evidence showed systems combining sustained autonomous planning, robust oversight evasion, resource acquisition, and real-world persistence across varied environments. I would lower it if capability gains plateaued, monitoring improved faster than autonomy, dangerous behaviors remained brittle under realistic testing, or deployment architectures reliably denied critical access and permissions.
The argument should turn on those observations, not on whether a chatbot sounds alive.
Incentives distort both reassurance and alarm
The organizations closest to frontier development possess information the public needs, but they also have interests.
Developers benefit when their systems appear powerful. Extraordinary capability claims attract capital, talent, customers, government attention, and strategic importance. Strong safety requirements can demonstrate responsibility, but established firms may be better equipped to meet them than smaller competitors. At the same time, minimizing risk can speed deployment and protect revenue.
Critics, researchers, journalists, politicians, and advocacy organizations have incentives too. Alarm attracts attention and funding. Reassurance attracts a different audience. Political actors can use either to justify a preferred expansion or contraction of state power.
The existence of incentives does not prove insincerity; people can believe their claims and benefit from them at the same time. That makes checking claims against independent sources more useful than cynicism.
Read reports from AI companies as evidence from a party with privileged access and incomplete independence. Separate laboratory demonstrations from field evidence. Forecasts should disclose assumptions and time horizons. Parties with access to logs and freedom to disagree should investigate incidents. Claims about prevalence should include denominators. Evaluate proposed remedies against the mechanism they are supposed to interrupt.
Sincerity is not a substitute for evidence, and an incentive is not a refutation.
Govern consequences, not vocabulary
Human beings accept risk constantly. Driving, aviation, medicine, energy, construction, finance, and biological research all create benefits while exposing people to possible harm. None has achieved zero defects or zero accidents. The relevant achievement is a system that makes failure less likely, limits consequences, learns from incidents, and assigns responsibility rather than the impossible standard of perfect safety.
Artificial-intelligence governance should use the same logic.
Low-consequence experimentation should remain relatively easy. A system that recommends a restaurant does not need the controls required for one that approves a loan, guides surgery, operates a vehicle, manages a power grid, searches for biological agents, or selects a military target. The European Union’s Artificial Intelligence Act uses this kind of risk-based structure, imposing different requirements by use and consequence rather than treating every artificial-intelligence system alike.[7]
Existing governance already supplies many of the needed tools: professional standards, product liability, software assurance, privacy law, cybersecurity, internal controls, audit, insurance, procurement rules, incident investigation, clinical-research protections, and sector regulation. New capabilities like artificial intelligence may require new tests, thresholds, reporting duties, or technical expertise. They do not erase everything we have learned about managing dangerous systems.
The controls should follow the causal path: limit access and permissions; separate development, testing, and production; record consequential actions; test systems under realistic conditions; preserve meaningful human authority; and require independent review when exposed parties cannot protect themselves. Deployment should be staged, monitored against explicit stop criteria, and followed by incident investigations that seek causes rather than villains. Name the people and institutions accepting residual risk.
The analogies are familiar. Aviation pursues extraordinary safety through layered engineering, training, maintenance, reporting, and investigation. Biosafety Level 4 laboratories use containment, restricted access, protective systems, monitoring, and emergency procedures to manage agents that have no intention at all. Human clinical trials use independent review, informed consent, safety monitoring, and, where appropriate, stopping rules because possible benefit does not cancel uncertainty.[11]
Those systems still fail. A 1978 Birmingham laboratory accident caused the last known smallpox cases and killed one person, and a 2014 Centers for Disease Control and Prevention incident potentially exposed staff to live anthrax after established safety practices were not followed. Dichlorodiphenyltrichloroethane (DDT) and dangerous radioactive patent medicines remind us that useful or promising technologies can cause harms that become clear only after deployment. The responses differed because the mechanisms differed: containment tightened, unsafe uses were restricted or ended, evidence standards rose, and responsibility remained human.[11]
An artificial-intelligence user may reasonably be asked to acknowledge risks. That consent should not excuse negligence by the company that built or operated the system, release a professional from a duty of care, or authorize harm to people who never agreed to bear it.
The hardest part is remedy realism. A rule that one country can enforce only by surrendering a strategically important capability to another may fail. A requirement that large firms can satisfy but small competitors cannot may increase concentration. A prohibition that cannot be monitored may create confidence without control. A human-review mandate is empty if the reviewer lacks time, information, authority, competence, or incentive.
A policy should be judged by whether it changes the probability or severity of the feared outcome at an acceptable cost, not by how cautious it sounds.
A more useful fear test
When I encounter a new claim about artificial-intelligence risk, I now want answers to ten questions:
What specific outcome is being predicted?
What mechanism produces it?
What capabilities must the system possess?
What access, permissions, resources, and deployment conditions does the mechanism require?
Who is exposed, and can those people consent or protect themselves?
What evidence comes from real deployment, and what comes from a laboratory, simulation, an AI company report, an anecdote, or a forecast?
What are the probability, time horizon, severity, and reversibility?
Which human decisions, incentives, and omissions enable the outcome?
What evidence would materially change the estimate?
Which intervention interrupts the mechanism, and what new risks or costs does it create?
These questions replace undifferentiated fear with something more useful: a working model that can be tested and revised, even though it cannot produce certainty.
My mother does not need to choose between believing that artificial intelligence will save the world and believing that it will end it. Neither do the rest of us. We can recognize present harms, prepare for plausible high-consequence risks, remain honest about uncertainty, and still use the technology to solve problems that were previously too expensive or difficult to attack.
The series now turns from risk to use. Once we have described the software without inventing a hidden moral agent, distinguishing harmful bias from viewpoint and missing context, and separating real risks from rhetorical bundles, we can ask how people should use artificial intelligence without surrendering the judgment that makes it valuable.
The next paper, Compounding Intelligence, takes up that question. The AI Industrial System then moves outward to the infrastructure that makes such use possible.
Questions, corrections, or disagreements are welcome. You can reach me directly at dave@aworkingmodel.com.
Sources
International AI Safety Report, International AI Safety Report 2026, February 3, 2026.
Ibid., section 2.2.2, “Loss of control.”
Ibid., capability outlook through 2030.
Ibid., sections on AI-generated content, criminal activity, influence, and manipulation.
Ibid., section 2.3.1, “Labour market impacts.”
Ibid., section 2.3.2, “Risks to human autonomy.”
European Commission, “AI Act”, current implementation guidance accessed September 19, 2026.
International Energy Agency, “Key Questions on Energy and AI,” Executive Summary, updated 2026; Lawrence Berkeley National Laboratory, “United States Data Center Energy Usage Report: 2025 Update,” June 2026.
National Aeronautics and Space Administration, “Moon Craters” and “Planetary Defense”; National Oceanic and Atmospheric Administration, “How NOAA Monitors Space Weather to Prevent Disruptions.”
National Security Archive, “The Underwater Cuban Missile Crisis at 60”; Smithsonian National Air and Space Museum, “The Cuban Missile Crisis”; U.S. National Park Service, “Stanislav Petrov”; U.S. Department of Energy, Atmospheric Nuclear Tests. See also Marine Corps Training and Education Command, “Tybee Island Bomb Still Rests in Depths of Wassaw Sound,” and National Security Archive, “New Details on the 1961 Goldsboro Nuclear Accident.”
Centers for Disease Control and Prevention, “Recognize the four Biosafety Levels”; U.S. Food and Drug Administration, “Protection of Human Subjects; Standards for Institutional Review Boards for Clinical Investigations”; World Health Organization, “Smallpox”; Centers for Disease Control and Prevention, Report on the Potential Exposure to Anthrax; U.S. Environmental Protection Agency, “DDT: A Brief History and Status”; U.S. Food and Drug Administration, “Medical Device & Radiological Health Regulations Come of Age.”
OpenAI, “The Hugging Face incident and the road ahead,” August 26, 2026.
METR, “Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident,” August 26, 2026.