The New York Times: From AI Doomsday to Founding Anthropic, Dario Amodei’s Obsession and the Safety Gamble

Author: The New York Times

Translated by: Jia Huan, ChainCatcher

One evening in 2007, Dario Amodei, a Ph.D. student at Princeton University, hosted a visiting friend. He had just finished reading Ray Kurzweil's book *The Singularity Is Near*, published in 2005, and could not wait to share its ideas with him.

In the book, Kurzweil predicted that artificial intelligence would eventually surpass the human brain, ushering in a moment known as the "singularity." As Dario wound a finger around a strand of hair, he told his friend Colin Heckman that this moment might arrive within their lifetimes.

"He was hooked," said Heckman, who attended high school with Dario in San Francisco. "He said AI has to be created the right way, or we're doomed."

At 24, Dario was still a physics doctoral student, in a phase of voraciously absorbing information. He became deeply engrossed in reading niche blogs like *Accelerating Future* and *Overcoming Bias*, where bloggers discussed the possibility of AI destroying humanity.

He also delved into GiveWell's online forums. GiveWell is an organization dedicated to maximizing the impact of charitable donations, where intellectuals discussed how to generate the most funds for charitable causes.

We interviewed more than two dozen of Dario's classmates, friends, teachers, and colleagues. Their accounts reveal that these books, blogs, and online communities continue to influence him today, offering clues to how the 43-year-old chief executive of Anthropic views artificial intelligence. Anthropic is expected to go public this month, which could be one of the largest public offerings in the world.

According to his friends and colleagues, the core of Dario's worldview consists of two interwoven schools of thought. Both firmly believe that mathematical reasoning can change humanity's future.

One is "rationalism," which emphasizes viewing the world in a calm, logical manner. Self-taught AI researcher Eliezer Yudkowsky and others subscribe to this philosophy, arguing that AI will become powerful enough to potentially destroy society.

The other is "effective altruism," which developed alongside rationalism and attempts to maximize social well-being through precise mathematical calculations.

His friends and colleagues say that for Dario, these two currents merged into a firm conviction: AI could be extremely dangerous, and only he and like-minded technologists deserve to be entrusted with developing it. This drives him to race ahead of those who do not care about human safety to create AI, and fuels his desire to control the technology.

In an essay published in September calling for a slowdown in AI development, Dario wrote, "Only if we build this technology the right way can we reap its benefits. As long as we can make good use of the time we've gained, it's worth being exceptionally careful to get things right."

These ideas form the foundation of Anthropic. In 2021, Dario co-founded the company with his sister Daniela Amodei and five others. All of them had been exposed to the rationalist and effective altruism communities. As the company grew, their perspectives garnered increasing attention, and Dario has consistently and tirelessly championed them in public.

Caption: Anthropic co-founder Daniela Amodei, photographed at the San Francisco office in 2023. She shares a close bond with her brother, Dario.

Dario has also engaged in discussions on geopolitical issues. He stated that the U.S. should restrict chip sales to China to weaken the country's competitive edge in AI. He has called for AI regulation to prevent the technology from spiraling out of control. During disputes with the U.S. Department of Defense, he also asserted that AI should not be used for mass surveillance of Americans or deployed in autonomous weapons.

"He truly believes we face enormous existential risks," said Nathan Avery. Avery raced cross-country with Dario in high school, and although the two have grown apart over the years, they've had numerous in-depth conversations. "He feels he needs to position himself where he can actually take action on this."

Dario is a billionaire whose net worth is expected to grow as Anthropic goes public. He declined to be interviewed through a company representative.

His ideas have won him supporters, particularly among some professionals in the AI safety field. They believe Dario is developing AI responsibly.

But it has also made him enemies in Silicon Valley and Washington, drawing criticism from figures like Meta's Mark Zuckerberg and Nvidia's Jensen Huang. They have attacked him as an "apocalypticalist" attempting to concentrate power in the AI field in his own hands. In September, the U.S. Department of Defense posted on social media: "Americanism, not effective altruism."

Dario appears aware that his ideas have angered some people, leading him to try publicly distancing his beliefs from rationalism and effective altruism. In 2023, he told The New York Times' podcast *Hard Fork* that he does not consider himself an effective altruist.

"I agree with some of the ideas," he said, but he opposes "dividing people into different tribes."

"Dario, relax!"

Dario and his sister Daniela grew up in San Francisco's Mission District. Their father, Riccardo Amodei, was a leather craftsman, and their mother, Elena Engel, oversaw renovation projects for public libraries. Divorce records show the couple separated in 1991.

While in eighth grade at Laidlaw Academy, Dario was already standing out. In 1997, he and a classmate were named the students "most likely to succeed," pictured holding dollar bills in their yearbook photo.

At Lowell High School, one of San Francisco's top public high schools, Dario studied with unquenchable hunger and took classes at the University of California, Berkeley. Classmates say he pondered philosophies like utilitarianism, which holds that people should create the greatest happiness for the greatest number of lives. He also worried that the Y2K bug—when computer programs using two-digit year formats might mistake "00" for 1900 after 2000—could paralyze global computer systems.

Richard Shapiro, then a physics teacher at Lowell High, said Dario often felt he underperformed on exams.

"I'd yell at him, 'Dario, chill out!'" Shapiro said.

In 2001, Dario enrolled at the California Institute of Technology in Pasadena, but he frequently returned to San Francisco to visit friends and family. On one trip home, he went to a sushi restaurant with Avery and Heckman, where the trio discussed utilitarianism. Dario was a vegetarian, but he saw no issue with eating crab because crabs are invertebrates.

Over dinner, Heckman posed a thought experiment: If someone keeps killing chickens, under what circumstances would it be justified to kill that person to stop the slaughter?

Avery recalled that Dario chuckled lightly before saying that, from a utilitarian perspective, once the number of chickens reaches a certain point, their collective lives might outweigh the life of a single person.

Friends say Dario once described himself as an atheist, yet he held firm convictions. In 2003, he drove Avery and others to San Francisco to participate in protests against the U.S. invasion of Iraq.

The following year, Dario transferred to Stanford University, located in the heart of Silicon Valley. The move also brought him closer to his sister, Daniela, who was attending high school nearby.

In 2023, speaking about his sister on a podcast, he said, "We've always wanted to save the world together."

Reflections on Time and Life

After graduating from Stanford in 2006, Dario entered Princeton University for a Ph.D. in physics, ultimately focusing on applying physics to biology and computational neuroscience. That same year, his father, who suffered from hepatitis C, passed away.

Dario threw himself into research on retinal cells and began developing an interest in creating technologies that could accelerate scientific progress.

Jade Wang, who dated Dario and co-founded a science discussion group with him, said they frequently debated whether scientists who spend their entire careers studying a single disease could make a greater impact if they shifted their focus to developing technologies or methods for the broader field.

She said Dario brought up his own father: he died before treatments became available that could cure most hepatitis C patients.

Caption: Ray Kurzweil, author of *The Singularity Is Near*. Dario showed great interest in his writings on artificial intelligence.

By then, Dario had begun aligning himself with the works of Kurzweil and rationalist leader Yudkowsky. He was particularly drawn to one idea from Kurzweil: advanced technology could potentially help humans transcend their natural limitations. This philosophy is known as transhumanism.

When Heckman visited him at Princeton in 2007, Dario told him the singularity would arrive before 2040 and that humanity might one day merge minds with machines.

Heckman recalled that Dario remarked at the time, "Over time, we'll become a new superintelligent AI, or a hybrid of AI and humans."

Dario was so fascinated by this idea that he encouraged friends and his sister to attend annual conferences hosted by Yudkowsky's Singularity Institute. The organization is now known as the Machine Intelligence Research Institute.

In a 2008 email seen by The New York Times, Dario told a friend that Yudkowsky was "one of the deepest thinkers I've ever met" and was "explicitly trying to engineer a pro-human singularity."

In an email to The New York Times, Yudkowsky stated that Dario "does not currently seek or follow my advice, nor do I recall him ever having done so."

Caption: Self-taught AI researcher Eliezer Yudkowsky. Dario began aligning himself with his work. Yudkowsky argues that AI will become powerful enough to destroy society.

Around the same time, Dario also began corresponding with Elon Musk, founder of the aerospace company SpaceX. Two people familiar with their exchanges said Dario once told friends he admired the tech mogul and even considered working for him, but ultimately did not join the space startup.

In 2008, Dario discovered GiveWell while reading economist Tyler Cowen's blog, which noted that the organization offered "interesting advice" on charitable groups.

Dario became active on GiveWell's mailing list and forums, discussing how to conduct charity and science work in an effective altruist manner. An email seen by The New York Times revealed that in 2010, he wrote that he was "considering going into finance so I can donate more money than I am now."

That same year, Dario discussed his decision to donate $10,000 in a GiveWell blog post. He was choosing between two recipients: Partnership to Control Tuberculosis (which focuses on preventing adult tuberculosis) and VillageReach (which helps vaccinate children in African villages).

Dario wrote that Partnership to Control Tuberculosis offered better "cost-effectiveness" because it served adults, and "adults have deeper, more meaningful experiences than infants." He added, "I think the death of one adult might be two to three times worse than the death of an infant."

But after further comparing how the two organizations executed their projects, Dario ultimately changed his mind, concluding that VillageReach's work was more effective.

In 2011, he began a postdoc at Stanford University researching cancer cells. Friends say that in his personal life, he experimented with anti-aging supplements, adhered to a rigorous swimming regimen, and became interested in reducing calorie intake. Some studies suggest that caloric restriction can extend the lifespan of laboratory mice.

"He often talked about the finiteness of time, and that time is the only non-renewable resource," Avery said. He added that Dario was "almost ruthless with his time."

Two people who know him said that in 2014, Dario was diagnosed with testicular cancer, which further deepened this sense of urgency. He underwent chemotherapy and eventually made a full recovery.

By then, he had forged a close relationship with Holden Karnofsky, one of GiveWell's founders. GiveWell began collaborating with Good Ventures, the philanthropy funded by Facebook co-founder Dustin Moskovitz and his wife Cari Tuna, which supports effective altruism-related causes.

Dario began providing advice on life sciences funding for GiveWell Labs, the joint venture between the two organizations.

Caption: Holden Karnofsky (right), one of GiveWell's founders, photographed in 2012. Dario was active in GiveWell's online forums.

At the time, Karnofsky was dating Daniela. The pair moved into a shared apartment in San Francisco, which later housed Dario, theoretical physicist Jared Kaplan, and college dropout Christopher Olah. Dario had previously tutored Olah in math.

Three people who have visited the residence said the roommates discussed Yudkowsky's essays and explored rationalism and effective altruism. They also attended Burning Man, the annual counterculture gathering in the Nevada desert, sometimes wearing animal costumes.

Kaplan and Olah later co-founded Anthropic with the Amodei siblings. In 2025, Karnofsky also began working for the company.

From Baidu and Google to OpenAI

In the early 2010s, machine learning made advances, prompting tech companies to ramp up investment in AI. Machine learning enables computers to learn how to complete tasks by analyzing data.

One of the companies diving in was Chinese internet giant Baidu. Baidu opened a lab in Sunnyvale, California, and hired AI pioneer Andrew Ng from Google as its chief scientist.

Dario joined Baidu in 2014 and quickly became known for pulling all-nighters at the office and speaking his mind.

Caption: AI pioneer Andrew Ng, who worked at both Google and Baidu. Dario joined Baidu to conduct AI research.

Gregory Diamos, a researcher who worked with Dario at Baidu, said, "Dario argued very intensely almost every time he got the chance."

Even in the early stages of this wave of AI development, Dario was already concerned about the technology's potential dangers. Diamos recalled that in 2015, Dario told colleagues that AI might soon match or exceed human brain capabilities, and therefore companies developing AI should not shy away from the possibility of losing control of the technology.

That same year, Dario joined Google's AI lab Google Brain, focusing on AI safety. Along with five other researchers, including his roommate Olah, he co-authored the 2016 paper "Specific Problems in AI Safety," which outlined abnormal behaviors AI might exhibit if developed improperly.

In 2016, Dario joined a new AI project: the nonprofit OpenAI. He led OpenAI's safety team, as well as another team researching "scaling laws." Scaling laws describe how AI model performance may improve as model size, training data, and compute resources increase.

At OpenAI: Ideals, Power, and Fractures

Seven people who formerly worked with Dario said that during his time at OpenAI, his conviction about the right way to develop AI grew stronger. He spoke more frequently about AI's power and potential harms, and took action to weave his ideas into the technology's development.

For example, in November 2016, just three days after Donald Trump was elected U.S. president, Dario wrote in OpenAI's internal communications app that he feared the United States was sliding toward authoritarianism.

Messages seen by The New York Times showed him stating he "couldn't sleep peacefully knowing I was researching technology with serious military applications in an authoritarian regime."

Internal messages revealed that Dario also wrote that investor Peter Thiel was "morally culpable" for donating $1.25 million to Trump's campaign, and OpenAI should no longer accept his money. (Thiel's venture capital firm, Founders Fund, later invested in Anthropic.)

Dario also helped secure a partnership between OpenAI and funders who shared his views. Three people familiar with the arrangement said that in 2017, he facilitated a connection between OpenAI and Karnofsky's philanthropy, where he served as a technical advisor.

Karnofsky's organization, now named Coefficient Giving, donated $30 million to OpenAI. Karnofsky also joined OpenAI's board of directors.

Caption: Dario working at OpenAI in 2017, and Daniela working at OpenAI in 2019.

Later that year, Karnofsky married Daniela. According to their wedding website, guests received plush toys, and anyone who arranged "the most romantic pairing between characters" would win a prize.

Dario attended the wedding wearing a panda mascot head, with the cocktail attire dress code being "comic formal." In a photograph seen by The New York Times, he is pictured in this outfit alongside Olah, who was wearing a giraffe head, and his then-girlfriend Cami Clark. He married Clark in 2022.

In 2018, Daniela joined OpenAI, eventually becoming the vice president of safety and policy. Some employees soon began questioning the siblings' growing influence.

Three people familiar with the discussions said that in December of that year, OpenAI researcher Joshua Achiam told CEO Sam Altman that appointing Dario as the company's chief ethics officer was unwise. Achiam cited the Amodei siblings' escalating power and argued that oversight of AI should be shared among people with differing viewpoints.

When Achiam insisted that an external expert should be appointed as chief ethics officer, Dario confronted him face-to-face. Insiders said Dario raised his voice, asserting that no one was better suited for the role than himself.

In 2018, the Amodei siblings took over a company project named GPT. GPT stands for Generative Pre-trained Transformer.

Six former employees said they explored a safety technology designed to teach AI to complete tasks through human instructions.

This approach later became a hallmark of Dario's career: pushing AI closer to potential danger, then stress-testing it with safety protocols.

OpenAI President Greg Brockman wanted to get involved in the GPT project but faced resistance from the Amodei siblings. Two people familiar with company affairs said Dario repeatedly told Altman that he would leave if Brockman wasn't kept off the work.

A person familiar with Altman's mindset said Altman viewed these ultimatums as excessive and rejected the siblings' demands.

In December 2020, the Amodei siblings and other colleagues left OpenAI. Four weeks later, they founded Anthropic.

Caption: This May, at the Anthropic developer conference in San Francisco, the Amodei siblings appeared on stage with company Chief Product Officer Ami Vora (left).