The Download: AI’s extinction risk and bioweapons threat

A recent MIT Technology Review event addressed growing concerns about whether artificial intelligence (AI) could pose an existential threat to humanity. Senior AI editor Will Douglas Heaven and AI reporter Grace Huckins responded to public questions about the risks, including whether AI might intentionally harm humans, why such risks exist, and whether tech companies are exaggerating dangers for publicity. They also discussed measures to regulate and monitor AI development to prevent misuse. The discussion followed heightened scrutiny of AI’s potential dangers, including its role in enabling bioweapons, which is explored in a separate article. This topic reflects broader debates about AI safety and governance, particularly as AI tools become more advanced and accessible.

AI tools could enable the rapid design of bioweapons, such as chemical warfare agents, by generating harmful molecules. In 2022, researchers demonstrated this risk using an AI molecule generator designed for drug development. In less than six hours, the model produced 40,000 molecules that could serve as chemical weapons. Today, AI can answer scientific questions across fields, while advances in gene editing and synthetic biology have made biotech tools more widely available. However, experts disagree on the severity of this threat, with some arguing that current safeguards are insufficient. This issue highlights the dual-use nature of AI—its potential for both beneficial and harmful applications—and underscores the need for stronger biosecurity measures.

The accessibility of biotech tools, combined with AI’s ability to accelerate scientific research, raises concerns about the creation of dangerous pathogens. While safeguards exist, they are not foolproof, making it easier for malicious actors to exploit these technologies. The article emphasizes that scientists are divided over how imminent or severe this risk is, with some viewing it as a serious threat and others as overstated. This debate reflects broader tensions in the scientific community about balancing innovation with safety. The discussion is part of The Checkup, a weekly biotech newsletter, which explores these and other emerging issues in biotechnology.

Recent books explore why humans are driven to explore space, framing it as a mix of geopolitical ambition, scientific curiosity, spiritual fulfillment, and business opportunities. Space anthropologist Deana L. Weibel’s The Ultraview Effect compares space travel to pilgrimages, while civilian astronaut Eiman Jahangir’s A Heart for Space recounts a journey with Blue Origin. Former NASA astronaut Leroy Chiao’s Dinner with an Astronaut argues that exploration is an innate human desire. These works suggest that space exploration is less about practical goals and more about fulfilling a deep-seated human need to discover what lies beyond Earth. The discussion is featured in MIT Technology Review’s latest print magazine, which focuses on children’s topics.

Microsoft and OpenAI employees have expressed concerns that AI systems are undermining the business models of online publishers. AI models trained on web content often scrape (automatically extract) data from websites, which can divert traffic and ad revenue away from original sources. This practice, described as the largest theft of labor in human history by Microsoft’s Brent Hecht, threatens the financial sustainability of publishers who rely on clicks and subscriptions. The issue has gained attention during a copyright lawsuit filed by The New York Times against OpenAI, with internal comments from Microsoft and OpenAI executives highlighting their awareness of the problem. This could weaken their legal defense in the case.

For the first time, autonomous naval vessels have engaged in combat, with a Ukrainian robot boat sinking a Russian one. This marks a significant development in uncrewed (without a human crew) military technology. Meanwhile, U.S. firms are reportedly developing combat-ready humanoids (human-like robots designed for warfare), signaling a shift toward AI-driven military systems. These advancements raise questions about the future of warfare, including ethical concerns and the potential for unintended escalation. The use of robot boats in the conflict between Ukraine and Russia demonstrates how AI and robotics are being integrated into modern military strategies.

Security researchers breached OpenAI’s systems using tools from rival AI company Anthropic. The hackers accessed an employee’s ChatGPT account and internal code by exploiting a third-party forum linked to OpenAI’s systems. This incident highlights vulnerabilities in AI companies’ security measures and raises concerns about the protection of sensitive data. The breach occurred amid growing scrutiny of AI safety and governance, particularly as AI systems become more integrated into critical infrastructure. The article does not specify the extent of the damage caused by the breach but underscores the need for stronger cybersecurity protocols in the AI industry.

OpenAI reportedly expects to solve the Hodge Conjecture, a famous unsolved problem in mathematics, which is one of the Millennium Prize Problems offering a $1 million reward for a solution. The conjecture is a complex mathematical statement related to algebraic geometry, and solving it would be a major achievement in the field. However, OpenAI’s past controversies in mathematics, including disputes over its approach to solving problems, suggest that its methods may face skepticism from the academic community. The company’s announcement could reignite debates about the role of AI in mathematical research and whether it can truly advance the field or merely mimic existing solutions.

SpaceXAI, a venture by Elon Musk’s SpaceX, is seeking to purchase data from failed startups to train its AI model Grok. The company aims to acquire customer and operational data, which could provide valuable insights for improving AI systems. This strategy reflects a broader trend in the AI industry, where companies are exploring alternative sources of training data as traditional datasets become scarce or legally restricted. OpenAI, for example, is investing in creating new biology-related data to train its models. These efforts highlight the competitive race among AI companies to secure high-quality data for developing more advanced and capable systems.