Daniel Kokotajlo, Former OpenAI Governance Researcher: Probability AI Wipes out Humanity is 70 percent! By Brian Simpson
According to former OpenAI governance researcher Daniel Kokotajlo, the probability of artificial intelligence either destroying or catastrophically harming humanity is 70 percent. And that is not great odds for survival. He is one of many who have made similar claims. Still, we live in an age where disasters and existential threats are as plentiful as blackberries in the hills, so like every prediction, this must be taken with more than a pinch of salt. Still, he may be right. Certainly, the job replacement capacities of AI are very real. But will the threshold be reached, the singularity of supposedly AI surpassing human intelligence? But as well, will this "intelligence" be "conscious" as we know it, having intentions that are truly its own, outside of programming, being a true emergent phenomenon? I have dipped into the philosophy material on this, and they are still out on this one, as even human consciousness itself is still a mystery, at least as far as mainstream science goes, as argued by, for example by Colin McGinn, The Problem of Consciousness, (1991).
Nevertheless, it is disturbing that AI developments are proceeding at rocket speed with no concern by the rogue profession itself, outside of the few dissents, for the possible negative ramifications.
"A former OpenAI governance researcher has made a chilling prediction: the odds of AI either destroying or catastrophically harming humankind sit at 70 percent.
In a recent interview with the New York Times, Daniel Kokotajlo, a former OpenAI governance researcher and signee of an open letter claiming that employees are being silenced against raising safety issues, accused the company of ignoring the monumental risks posed by artificial general intelligence (AGI) due to its decision-makers being enthralled with its possibilities. "OpenAI is really excited about building AGI," Kokotajlo stated, "and they are recklessly racing to be the first there."
Kokotajlo's most alarming claim was that the chance AI will wreck humanity is around 70 percent—odds that would be unacceptable for any major life event, yet OpenAI and its peers are barreling ahead with anyway. The term "p(doom)," which refers to the probability that AI will usher in doom for humankind, is a topic of constant controversy in the machine learning world.
After joining OpenAI in 2022 and being asked to forecast the technology's progress, the 31-year-old became convinced not only that the industry would achieve AGI by 2027 but also that there was a great probability it would catastrophically harm or even destroy humanity. Kokotajlo and his colleagues, including former and current employees at Google DeepMind and Anthropic, as well as Geoffrey Hinton, the "Godfather of AI" who left Google last year over similar concerns, are asserting their "right to warn" the public about the risks posed by AI.
Kokotajlo became so convinced of the massive risks AI posed to humanity that he personally urged OpenAI CEO Sam Altman to "pivot to safety" and spend more time implementing guardrails to reign in the technology rather than continue making it smarter. Although Altman seemed to agree with him at the time, Kokotajlo felt it was merely lip service.
Fed up, Kokotajlo quit the firm in April, telling his team in an email that he had "lost confidence that OpenAI will behave responsibly" as it continues trying to build near-human-level AI. "The world isn't ready, and we aren't ready," he wrote. "And I'm concerned we are rushing forward regardless and rationalizing our actions.""
Comments