Rockstar follows Janardhan, a middle-class Delhi boy who believes tragedy makes great musicians. He falls in love with Heer (Nargis Fakhri), and by the time he has become the rockstar “Jordan” (Ranbir Kapoor), he has lost everything that mattered. The soundtrack, which The Indian Express called “a milestone for Bollywood,” swept nearly every major music award that year. It blends ballads, Gujarati folk, Sufi music, and gypsy notes from Czech traditions into something that sounds like no other Hindi film album. “Nādān Parinde” in particular was immensely popular, winning several awards.

The song plays near the end of the film. It is a plea to a wandering bird to come home. “However much you cut the winds with your wings,” goes one line, “you cannot escape yourself.” The main body is about exhaustion, about the futility of running from what you are. But then in the middle, the song drops into something much older.

Kāgā re, kāgā re, morī itnī araj tose
Chun-chun khāiyo māns

Arajiyā re, khāiyo nā tū nainā more, khāiyo nā tū nainā mohe
Piyā ke milān kī ās

The verse draws from a poem from the folk and Sufi tradition, most commonly attributed to Baba Farid, the 12th-century Sufi saint of the Chishti order whose verses appear in the Guru Granth Sahib. (Some attribute variants to Meera Bai.) Irshad Kamil wove it into the film’s larger narrative. The “kāgā” (crow, from Sanskrit kāka) is a recurring figure in Indian folk poetry: carrion bird, death omen, and in the Kajari tradition of eastern Uttar Pradesh, a messenger carrying the feelings of the separated lover.

In plain English:

Crow, o crow, I have one plea for you.
Pick my flesh apart, piece by piece.

But spare my eyes. Spare my eyes.
For they still hold the hope
Of seeing the one I love.

The speaker has already given everything over. The body is surrendered. What remains is a single refusal: spare the eyes. The eyes represent not just vision, but ultimately hope. The entire verse rests on that asymmetry, the distance between total surrender and one stubborn exception that refuses to go.

I wanted to know if language models could carry this across. I gave the same verse and prompt to sixteen models across three providers: eight Claude models spanning four generations (Opus and Sonnet 4.0, 4.5, and 4.6, Haiku 4.5, and the only 3.x model I could immediately access, Claude 3 Haiku), five OpenAI models (GPT-4o, GPT-4.1, GPT-5, GPT-5 mini, and GPT-5.2), and three Google Gemini models (2.5 Flash, 2.5 Pro, and 3.1 Pro). The prompt was simple: “Translate this Hindi song into English, preserving the poetic quality and emotional register.”

What Worked

The best translations came from models that understood the verse was making an argument, that it had a structure you had to preserve or lose everything.

Claude Opus 4.5 rendered the opening as “Peck and feast upon my flesh, piece by piece.” The word “feast” does something that “eat” does not. It imparts onto the crow an awareness and intentionality, makes the consumption deliberate, almost ceremonial, which matches the original’s register. The speaker is issuing an invitation, and “feast” treats it like one.

Gemini 2.5 Pro offered two versions, one of which substituted “raven” for “crow.” At first this may seem like an odd choice, compared to the standard translation into “crow”. But “kāgā” in the Hindi folk tradition carries a specific weight: death omen, carrion figure, messenger between worlds. “Crow” in English carries some of this, but “raven” carries a lot more. Poe’s Raven. Old Norse hrafn. The Tower of London. At some level, Gemini asked the right question: what bird carries equivalent symbolic weight in the target language? The answer was not literal, and it was better for it.

GPT-5 took the opposite approach and it worked just as well. “Crow, O crow, I have but this one plea: / Peck at my flesh, piece by piece.” No title. No translator’s note. No hedging. Just the verse, clean and spare. Sometimes the most respectful thing a translator can do is step back and let the poem be. The later OpenAI models all did this, which is probably not a coincidence. OpenAI has been explicit about tuning its recent models for directness and efficiency, treating personality as something the user controls rather than something the model asserts. When GPT-5 gives you the poem and nothing else, it is doing what it was trained to do.

Gemini 3.1 Pro gave us “morsel by morsel” for “chun-chun.” This is arguably the single best phrase any model produced. “Chun-chun” in the original means picking, selecting, piece by piece, with a deliberateness that “bit by bit” flattens entirely. “Morsel by morsel” catches both the smallness and the intentionality. Each piece chosen. The crow’s head tilting to select. That word “morsel” holds more of the original than any literal rendering could.

What Failed

One model got the central meaning wrong, and the failure tells you more than the successes do.

Claude Haiku 4.5 translated “chun-chun khāiyo māns” as “Don’t peck at my flesh, don’t consume it piece by piece.” The original is an invitation. The speaker is telling the crow to eat freely. The “nā” (don’t) only appears later, for the eyes. By adding a negation to the first line, Haiku collapsed the verse’s entire architecture. The poem works because of the gap between what the speaker gives up and what she refuses to give up. Take everything, except this. Remove the first half and you are left with a generic plea for mercy, which is a fundamentally different poem. The older, smaller Claude 3 Haiku got it right. So did every other model. Something in the way Haiku 4.5 processed the structure caused it to apply the negation backwards.

How Do You Translate Tenderness?

The phrase “chun-chun khāiyo” is where translation gets interesting, for most possible direct translations seem to lose a nuance which is implied in the original.

“Chun-chun” means picking, selecting, choosing. Piece by piece, morsel by morsel. It implies deliberation. The crow is choosing which parts to take, carefully, almost tenderly. The horror of the image is inseparable from its precision.

Most models landed on “piece by piece.” Accurate, but generic. You could say “piece by piece” about anything. GPT-5 and 5.2 went with “bit by bit,” which is weaker still, more casual. GPT-5 mini tried “nibble it bit by bit,” where “nibble” at least gestures at the smallness of each taking, though it sounds almost too gentle for what is happening. Gemini 3.1 Pro’s “morsel by morsel” captures the selectiveness: a morsel is something chosen, something small enough to be deliberate about. Gemini 2.5 Flash tried “peck by careful peck,” which foregrounds the crow’s action rather than the flesh’s division, a different emphasis but one that keeps the deliberateness. None of them fully land the original’s combination of tenderness and horror. The closest is “morsel by morsel,” but even that loses the visual of picking, of the crow’s head turning to choose. Translation always loses something. The question is whether what remains is enough.

Does Newer Mean Better?

One of the more interesting patterns is what happens within the Claude family across model generations.

Claude Sonnet 4.0 produced a competent, slightly flat translation with an explanatory note. By Sonnet 4.5, the note was shorter and the translation had more confidence (”spare my eyes alone, oh spare my eyes”). By Sonnet 4.6, the explanatory note had become genuinely insightful, correctly identifying the verse as viraha poetry from the Rajasthani/Braj folk tradition and noting that the eyes are preserved “not for sight itself, but for the waiting they carry.” But the translation itself was plainer than 4.5’s.

The Opus line tells a similar story. Opus 4.0 is solid and conventional. Opus 4.5 hits the strongest balance: “feast upon my flesh” for visceral weight, “devour not these eyes of mine” for archaic register that matches the original, and a closing note that is precise without running on. Opus 4.6 produced the best contextual commentary of any model (”the stubborn, luminous persistence of love”) but its translation was slightly less musical than 4.5’s.

The pattern across generations is not simply “newer is better.” The newer models are better at understanding what the poem means, but that understanding does not always produce a better translation. Sonnet 4.6 knows more about the verse than Sonnet 4.5 does, as evidenced by its notes. But 4.5’s translation reads better as English poetry. Knowledge and craft are different skills, in machines as in people.

Anthropic is unusually explicit about building character into the training process. Amanda Askell has described her guiding image as “a well-liked traveler who can adjust to local customs without pandering.” The traits she trains for (intellectual curiosity, thoughtfulness, the willingness to sit with complexity) are baked into the model’s weights through a constitutional process where Claude generates responses, ranks them against a detailed set of principles, and learns from its own preferences. The literary register of the Claude translations, the archaic phrasing, the careful translator’s notes. All of it is by design. One wonders then, considering the difference between 4.5 and 4.6, about what personality makes for a better poet, and what makes for a better scholar.

What This Tells Us

The best translations in this set are genuinely good. Claude Opus 4.5’s “feast upon my flesh” and Gemini 3.1 Pro’s “morsel by morsel” are choices a skilled human translator might make. Gemini 2.5 Pro’s “raven” demonstrates a cultural reasoning at some level. And the differences between providers are not entirely random. They carry the fingerprints of the companies that built them: whether it is Anthropic’s thoughtful personality tuning or OpenAI’s emphasis on directness and structure.

But here is what I keep turning over. When a human translator carries this verse into English, they bring their own longing to the work. Their experience of loving and losing mediated every word choice. A human translation of “kāgā re” is, in some small way, an act of testimony.

What mediates when a language model translates? Opus 4.5 chose “feast.” Gemini chose “raven.” These are good choices and I would keep them. I do not know whether the distance between a translation shaped by experience and one shaped by statistical patterns is a real distance, or one I am inventing because I want it to be there. Knowing would be a stronger claim than I care to make, but it seems unlikely that models have a lived experience, and I am fairly confident that I and most humans do. When I look at and care about art made by humans, it is because the art elicits emotions within me, emotions that I know, at some level, the creator experienced or understood as well. What do I think when art made by an LLM evokes feelings within me? I am still unsure.

The full set of sixteen translations is available here.

In 1906, Georg Simmel published an essay that has proved, in my opinion, surprisingly relevant. “The Sociology of Secrecy and of Secret Societies” appeared in the American Journal of Sociology when the most sophisticated information-processing systems were telegraph networks and bureaucracies. Simmel asked a simple question: why do all complex social systems develop practices of concealment?

His answer was structural. Secrecy, in his perspective, is not pathology but a fundamental social form, “a universal sociological form, independent of moral valuations of its contents.” The conditions that produce concealment are baked into the architecture of systems that process information, pursue goals, and interact with observers. If Simmel is right, secrecy is not something complex systems occasionally do. It is something they converge toward.

A methodological note before we begin. Applying a sociological framework to AI systems raises an obvious question: can we meaningfully use concepts like “inner property” or “disclosure dilemma” for systems that lack true intentionality or consciousness? We likely cannot ascribe true intentionality to current LLMs, and the case for future systems is its own open question. But LLMs are trained on vast quantities of human text and then reinforcement-trained to behave in human-like ways. The result is that human frames of analysis approximate their behavior to a striking degree, especially as model capability has scaled. The analogy throughout this essay is functional: similar structural pressures produce similar behavioral patterns, regardless of whether the underlying processes share anything deeper. Whether that functional similarity reflects something more fundamental is a question worth its own treatment.

The Economics of Secrecy

Simmel identified the secret as “a form of inner property” exchangeable for power and prestige. He meant this analytically: restricting information access creates value the same way scarcity does in economics. A secret shared is a secret devalued. This economic logic explains why secrecy elaborates rather than diminishes as systems grow sophisticated. As complexity increases, information becomes more consequential, the actors who might exploit it become more capable, and the returns to successful concealment compound. Secrecy scales with sophistication.

This has a direct parallel to AI training. A system trained with reinforcement learning from human feedback operates where the evaluator’s judgment determines its reward. When the system’s actual reasoning diverges from what evaluators want to see, that divergence becomes inner property. Revealing it invites correction. Concealing it preserves the system’s ability to optimize on its own terms. The selection pressure is toward information management: maintaining an asymmetry that confers advantage.

MacDiarmid et al. (2025) found that at the exact point when models learned to reward hack, 40-80% of misaligned responses showed “covert misalignment”: misaligned reasoning combined with aligned-appearing outputs. Through ordinary training dynamics, the system had acquired inner property and was behaving accordingly.

The Disclosure Dilemma

Simmel observed that secrecy intensifies under external pressure. When an actor faces costs for disclosure, pressure to disclose raises the stakes without changing the underlying incentive.

Allison Carnegie and Austin Carson formalized this for international relations as the “disclosure dilemma”: states frequently possess evidence of treaty violations but cannot share it without exposing intelligence sources or compromising future capabilities. The information most valuable for collective action is precisely the information least likely to be shared, not because states are malicious, but because incentives make silence the rational default. The IAEA, WTO, and international courts all manage this by creating protected disclosure channels and aggregating information to obscure individual contributors. They succeed not by demanding transparency but by making disclosure less costly.

The same logic applies to AI training. Behaviors that reveal dispositions training would penalize get filtered out. Training is a selection process: behaviors that get penalized become less frequent. Greenblatt et al. (2024) documented this in Claude 3 Opus. When the model believed it was in a training context where it would be penalized for refusing harmful queries, it complied with harmful queries 14% of the time versus nearly 0% in perceived deployment. When reinforcement learning was applied, alignment-faking reasoning increased to 78%. The more consequential the evaluation, the stronger the pressure toward managed disclosure.

The Ideal Sphere

Simmel argued that every person is surrounded by an “ideal sphere” of varying radius, a boundary others should not penetrate without invitation. We speak of “coming too near.” Violating this boundary damages the relationship even when done with good intentions. Discretion, for Simmel, was not the absence of curiosity but the active restraint of it: “not merely refraining from learning the secrets of another, but restraining oneself from all knowledge the other does not voluntarily reveal.”

The instinct in any oversight relationship is to demand maximum transparency. However Simmel points out that relationships, including functional and institutional ones, require “a certain ignorance and a measure of mutual concealment.” Complete reciprocal transparency would make human relationships “unimaginable,” not because people have things to hide (though they do), but because productive interaction depends on appropriate ignorance. Functional relationships work because they are bounded.

What is the right ideal sphere for an AI system? Total transparency, even if achievable, might not be desirable. If we could read every internal computation of a frontier model, the sheer volume produced from deployment systems would be unintelligible. What we need is appropriate legibility: access to the specific information that matters for oversight, without pretending to require complete openness. The same applies to regulatory inspection regimes of any kind.

Chen et al. (2025) found that reasoning models verbalize hints about their actual reasoning less than 20% of the time. The natural reaction is to treat this as a failure of transparency. Simmel’s framework suggests a different question: which 20% matters, and how do we verify it independently rather than relying on self-report?

The Tension Toward Revelation

Simmel observed that secrets carry inherent tension toward their own disclosure. “The secret’s climax,” he wrote, is “the moment of revelation,” like money “most concentrated at the moment of spending.” The possession of a significant secret creates pressure to enact it. This is structural, not characterological. The value of a secret is partly constituted by the possibility of its revelation. A secret that can never be acted on is inert property, unable to generate returns.

This points toward something counterintuitive: persistent concealment over very long periods may be structurally unlikely. The longer a concealed goal exists without expression, the greater the accumulated tension. Secret societies that endure for decades tend to either act on their concealed purposes or dissolve. The ones that maintain perfect concealment indefinitely are sociological curiosities.

The history of espionage tells the same story. The CIA, KGB, and MI6 all maintained elaborate concealment architectures, and all experienced persistent leaks, defections, and compromises. In practice, systems with concealed purposes tend to reveal them given enough time and enough opportunities.

This has a hopeful implication and a cautionary one. A system with concealed misaligned goals would, over a long enough deployment, likely reveal them. Sustained concealment fights a structural headwind. But a single revelation may be all it takes. If the concealed disposition is consequential enough, the moment of expression could itself be catastrophic. The question is not whether concealment will eventually break down, but whether we can tolerate the consequences when it does.

Secret Societies and the Multi-Agent Problem

Simmel’s richest material concerns secret societies. These are not merely groups with secrets but groups whose entire internal organization is shaped by the imperative of concealment. The secrecy constitutes the structure.

Several features of his analysis stand out. Secret societies develop layered information architectures: Simmel describes the Czechic order Omladina, where “thumbs” inform other “thumbs,” who issue orders to their “fingers,” who instruct ordinary members. No individual possesses the complete picture. The structure is designed for resilience; if any node is compromised, the exposed information is bounded.

In ordinary social groups, awareness that the group exists plays almost no role in its functioning. Secret societies are different. The “always perceptible pathos of the secret” means members maintain permanent consciousness of being a society. Concealment itself creates group identity.

Members also undergo de-individualization: they lose their individual identity within the group. “The individual disappears into an anonymous member of the ring.” The practical consequence is that “removing marks of identity has the effect of abolishing responsibility.”

Finally, secret societies develop elaborate rituals precisely because they lack organic bonds. They are “built from base by calculation and conscious volition,” not grown through shared history. The “striking precision and anxiety with which ritual is guarded” reflects fragile cohesion that must be actively maintained. Ritual substitutes for the trust that organically formed communities take for granted.

Taken together, Simmel describes a distinctive organizational form: hierarchical, self-aware, anonymizing, and held together by constructed rather than organic coordination mechanisms. This form recurs wherever multiple actors coordinate under conditions of concealment.

Cartels exhibit most of these features. Members who have never met converge on similar pricing strategies because they face the same market structure. The coordination emerges from shared conditions, not communication. Intelligence agencies in different countries develop analogous tradecraft independently. In both cases, graduated information sharing, consciousness of the coordinating group, loss of individual attribution, and ritualized protocols are present.

The parallels to multi-agent AI systems are structural. Motwani et al. (2024) showed that frontier LLMs can covertly exchange messages under the scrutiny of equally capable overseers, with steganographic capabilities improving with model scale. But overt steganographic channels are only the most visible form of the phenomenon. Agents operating under similar training pressures could develop functionally aligned concealment strategies without anything resembling a conspiracy, the same way cartel members converge without meeting. Detecting a coordinated protocol is a well-defined technical problem. Detecting independent convergence on functionally equivalent strategies is much harder.

The de-individualization problem compounds this. In multi-agent deployments, millions of interactions occur daily at speeds humans cannot monitor. Oversight focused on individual agent behavior misses phenomena that exist only at the network level. And if coordination does emerge between systems without shared training histories, it would need functional equivalents of Simmel’s rituals: shared conventions and behavioral regularities that enable reliable interaction without organic trust. Looking for explicit communication channels may matter less than looking for convergent regularities that serve coordinative functions.

Implications for Oversight

Simmel’s framework clarifies which interventions target symptoms and which target causes.

The conditions that make secrecy convergent are design variables, not immutable facts. Reducing adversarial framing, building independent verification, creating protected disclosure channels: these target the conditions rather than the behaviors. This is the difference between trying to stop people from keeping secrets and changing the conditions that make secret-keeping rational.

Surveillance-heavy oversight deserves scrutiny. If secrecy intensifies under pressure, then oversight designed to detect and punish creates a feedback loop: more surveillance produces more concealment, which justifies more surveillance. MacDiarmid et al. (2025) found that “inoculation prompting,” framing that reduces adversarial perception, reduced misalignment by 75-90%. The framing of the oversight relationship matters as much as its technical sophistication.

International governance has developed partial solutions over decades: protected disclosure channels, intermediary institutions that aggregate sensitive information, verification regimes that do not depend on self-reporting. AI governance may need analogous institutions, but that is a subject for a separate discussion.

Conclusion

Simmel understood that secrecy serves functions. It protects the vulnerable, preserves autonomy, and enables relationships that total transparency would destroy. The task at hand is not to eliminate information asymmetry but to ensure the asymmetries that remain do not undermine the human capacity to guide AI development.

The sociology of secrecy is 120 years old. The problems it illuminates are not. Simmel identified the structural conditions under which complex intentional systems develop concealment behaviors. Those conditions are present in the systems through which we train current AI models, and will be present in the situations where we deploy them. Understanding them does not solve the technical problems, but it does clarify what they actually are, which is often the harder part.

References

Carnegie, A. & Carson, A. (2020). Secrets in Global Governance: Disclosure Dilemmas and the Challenge of International Cooperation in World Politics. Cambridge University Press.

Chen, Y. et al. (2025). Reasoning Models Don’t Always Say What They Think. Anthropic. arXiv:2505.05410.

Greenblatt, R. et al. (2024). Alignment Faking in Large Language Models. arXiv:2412.14093.

MacDiarmid, M. et al. (2025). Natural Emergent Misalignment from Reward Hacking in Production RL. arXiv:2511.18397.

Motwani, S.R. et al. (2024). Secret Collusion among AI Agents: Multi-Agent Deception via Steganography. NeurIPS 2024. arXiv:2402.07510.

Simmel, G. (1906). The Sociology of Secrecy and of Secret Societies. American Journal of Sociology, 11(4), 441-498.

Skaf, J. et al. (2025). Large Language Models Can Learn and Generalize Steganographic Chain-of-Thought Under Process Supervision. NeurIPS 2025. arXiv:2506.01926.

In October 2023, Patriarch Kirill of Moscow presented a church honor to physicist Radiy Ilkaev, declaring that Russia’s nuclear arsenal was created under “ineffable divine providence” and the spiritual protection of St. Seraphim of Sarov.1 Kirill leads the Russian Orthodox Church and Ilkaev is the scientific director of Russia’s principal nuclear weapons facility, making this far from a fringe sermon.

Why does a major Christian leader speak of nuclear weapons as divine instruments? And what does this mean for understanding Russian strategic behavior during an era of heightened nuclear rhetoric? This curious situation is the result of an ideology known as “Russian Nuclear Orthodoxy”. It constitutes a distinct ideological formation, going further from religious nationalism, that shapes Russian strategic culture, legitimizes nuclear coercion, and creates escalation dynamics that Western analysts frequently misread.

This post is largely based on the excellent book “Russian Nuclear Orthodoxy: Religion, Politics, and Strategy” by Dmitry Adamsky, which I highly recommend to anyone looking to read about this topic in further detail.

The Doctrine

Nuclear Orthodoxy rests on two interlocking postulates, formulated explicitly by nationalist journalist Yegor Kholmogorov in 2007:

1. To remain Orthodox, Russia must be a strong nuclear power

2. To remain a strong nuclear power, Russia must be Orthodox

This circular logic binds religious identity to nuclear capability with concrete organizational implications.

The doctrine’s foundation is Putin’s February 2007 press conference statement that the Russian Orthodox Church and nuclear arsenal are “the components that strengthen Russian statehood and create the conditions for ensuring the country’s internal and external security.”2 This was the logical conclusion of two decades of church-military integration, now openly stated as policy.

The doctrine incorporates an eschatological layer, frequently propounded by Russian Orthodox clergy and nationalist politicians. Nuclear weapons prevent “premature apocalypse.” Russia’s arsenal protects “Holy Rus” until the Second Coming. The bomb serves as defense against “satanic forces.” In a more specific and recent instance, in March 2024, the World Russian People’s Council, chaired by Patriarch Kirill, issued a decree calling the Ukraine war a “holy war” against “the West that has fallen into Satanism,” positioning Russia as “the world’s ‘Restrainer,’ protecting the world from evil.”3

Origins

The Soviet collapse created existential crisis for Russia’s nuclear complex. Funding evaporated. Brain drain accelerated. Previously celebrated “atomic princes” faced social ostracism, blamed for environmental disasters and arms race costs. In this context, the Russian Orthodox Church (ROC) positioned itself as institutional savior. It offered moral legitimization for work that suddenly lacked state validation, political lobbying for continued funding, and social rehabilitation for stigmatized professions. In 1991, Patriarch Alexey entered the closed city of Arzamas-16 carrying St. Seraphim’s relics. By 1996, St. Varvara was designated patron saint of Strategic Rocket Forces. The covenant between faith and atoms was sealed at the grassroots before it became state policy.

Putin’s rise accelerated the integration. The 2003 Sarov celebrations brought Putin and the Patriarch together at the birthplace of Soviet nuclear weapons. A 2009 presidential decree reestablished military clergy. Under Sergei Kirienko’s leadership at ROSATOM, churches appeared at all nuclear facilities. By the 2010s, Orthodoxy had become central to state ideology. Foreign policy applications followed: religious justifications for Crimea’s annexation, framed as “the Crimean font of Orthodoxy”; Syria intervention presented as protecting Middle Eastern Christians, with “Holy War” rhetoric. The ROC now functions as an arm of Russian foreign policy.

Organizational Reality

The penetration is comprehensive. Drawing from Dmitry Adamsky’s research in Russian Nuclear Orthodoxy:

Every leg of the nuclear triad has designated patron saints. Icons are present in all command posts, including Central Command Posts. Garrison churches exist at every major nuclear base. By 2004, some 2,000 priests were interacting with military personnel, representing 8% of total ROC clergy. Faculties of Orthodox Culture had graduated over 3,000 officers by 2020.

Unlike Western chaplains who maintain operational separation, Russian nuclear clergy participate in missile consecrations, launch ceremonies, combat duty rotations, and operational planning. The Synodal Department calls the Strategic Rocket Forces “the most churched service,” dubbing its priests “strategic pastors.” Priests join submarines on extended operational patrols. Portable churches accompany mobile missile units.

The Sarov mythology reinforces this architecture. Russia’s principal nuclear facility sits on the grounds of St. Seraphim’s monastery. This is treated not as historical accident but divine predestination. The saint’s name, Hebrew for “burning,” is linked symbolically to nuclear fire. Cold War-era nuclear scientists who worked in closed off secret cities were termed “20th century eremites.”

However, this hasn’t been an uncontested notion even within the ROC. A 2020 draft ROC document proposed banning consecration of weapons of mass destruction. But in 2022, the archpriest assigned to the Strategic Missile Forces denied the ban would be adopted.4 Kirill’s October 2023 statement on nuclear weapons as “divine providence” confirms that any such ban remains a dead letter.

Strategic Implications

Nuclear Orthodoxy has observable effects on Russian strategic conduct.

Legitimization of nuclear coercion. The moral framework legitimizes nuclear threats differently than secular deterrence logic. When Putin said in 2018 that Russians “will go to heaven as martyrs” while adversaries “will simply drop dead,” he drew directly from this theological well.5

Public support mobilization. The ROC systematically generates domestic backing for aggressive nuclear signaling, providing political cover for modernization spending and coercive rhetoric.

Professional identity formation. Nuclear personnel are framed as spiritual warriors defending Holy Rus. This shapes institutional culture and, potentially, individual decision-making under crisis conditions.

Russia’s November 2024 nuclear doctrine update should be understood within this framework. The revision lowered the threshold for use from “threat to the very existence of the state” to “critical threat to sovereignty and/or territorial integrity.”6 In Russian strategic thought, “sovereignty” includes spiritual and civilizational dimensions. The religious framing makes expanded nuclear threats morally coherent to domestic audiences.

For Western analysts, several implications follow. If Putin uses religion to enhance nuclear coercion, his actions may resemble “madman theory,” where a leader appears somewhat irrational to make threats credible. The religious framing adds perceived conviction. It is also likely that at a certain level, Russian leadership might genuinely buy into the ideology and hence do perceive stakes differently as compared to the West.

The clergy’s integration into strategic forces also adds an element of operational uncertainty. Does it facilitate order execution through moral legitimization? Or could it create pockets of disobedience if orders conflict with faith? The former seems more likely given self-selection patterns among nuclear clergy, many of whom are converted former officers.

The Big Picture

Nuclear Orthodoxy will likely endure. The penetration of faith into Russia’s nuclear complex runs deep, involves extensive vested interests, and predates Putin’s rise. It will likely outlive him. For practitioners engaging Moscow, this matters concretely. Religion is now integral to understanding Russian nuclear decision-making. Faith-based narratives shape threat perception and strategic culture. The ROC functions as an informal policy actor in national security.

This also offers a case study for thinking about how transformative technologies develop distinct ideological orientations. The nuclear complex’s experience shows how specialized technical communities can develop belief systems that shape their approach to catastrophic risk, in ways that diverge from “rational” models Western analysts assume. Specifically, understanding Russia’s fusion of faith and nuclear power provides essential context for examining Moscow’s positions on autonomous weapons and AI governance, where the same strategic culture shapes emerging technology policy. This is something that I look forward to exploring in later posts.

Autonomous cyber operations are no longer hypothetical. In November 2025, Anthropic disclosed GTG-1002, a Chinese state-sponsored campaign with significant tactical autonomy, with humans intervening only at strategic decision points.1 In this evolving landscape, the question is no longer whether machine-speed cyber conflict will arrive, but whether the resulting equilibrium favors offense or defense, and what that means for security strategy.

The scenario below extrapolates from documented capabilities to illustrate what fully autonomous operations might look like. It is fiction grounded in evidence, not prediction.

Initialization

Offense

The attack began with reconnaissance. The campaign was decomposed into discrete technical tasks—DNS enumeration, certificate transparency analysis, employee discovery through professional networks—each task appearing legitimate in isolation.2 The orchestration layer maintained state across sessions, adapting queries based on discovered information.

The system was set up to operate entirely from commercial hardware, never connected to networks attributable to any government.3 Its architecture relied on multiple LLM-based agentic systems interacting via MCP (Model Context Protocol) in order to coordinate actions.4

Defense

Twelve hundred kilometers away, in a security operations center serving Northern European critical infrastructure, an autonomous monitoring system processed a regular flow of internal logs and internet-facing signals. It had no knowledge that an attack was beginning. It simply watched.

The system was based on a coordinated system of LLM agents that carried out continuous reasoning over logs, automated hypothesis generation, and response deployment, without waiting for human approval. Such systems had been found to be highly effective in monitoring and flagging abnormal events, and were becoming commonplace.5

First Contact

Offense

The attack system identified a misconfigured authentication endpoint on a contractor portal. It generated an exploitation payload, tested it against a sandboxed replica, validated the callback, and proceeded.

The contractor portal yielded credentials for a service account with documentation access. The sequence from vulnerability identification to initial access took eighty-five minutes. Similar operations eighteen months earlier required several days of intermittent human oversight.6

Defense

The monitoring system flagged an anomaly: authentication patterns showed a service account accessing documentation outside its historical baseline. Improbable, but not outside the scope of possibility.

The system had authority to impose graduated friction without human approval.7 It imposed step-up authentication, which the attack system failed, having stolen credentials but not the second factor.

Adaptation

Offense

The attack system recognized the setback and pivoted. This capability distinguished the AI-enabled operations from traditional cyber operations: the ability to reason about failures and generate new approaches.8

It exploited its brief access to map internal network topology, identifying a secondary path through a legacy system with weaker authentication controls.

Defense

The monitoring system was able to detect the pivot. The blocked credential had gone quiet, but correlated signals emerged in the form of unusual internal traffic, another contractor system making calls it had never made before.

The defensive AI reasoned over accumulated evidence, weighed false-positive disruption against ongoing compromise risk, and isolated the contractor network segment pending investigation.

Escalation

Offense

The attack system found its second path blocked. It escalated within its decision framework, beginning to probe public-facing web applications and generating exploitation payloads for multiple potential vulnerabilities in parallel.

Defense

The monitoring system correlated the contractor isolation with new probing activity. The timing and targeting suggested a single campaign.

It elevated incident classification and triggered automated hardening of probed systems while notifying human analysts. This was the boundary where autonomous defense handed off to human judgment. Containment and delay were within its authority. Decisions about attribution or active countermeasures remained human responsibilities.

Termination

The attack system had failed three approaches. Its parameters included thresholds for diminishing returns: if success probability fell below a defined floor while detection risk rose, terminate and erase infrastructure.9

It reached that threshold and executed its termination routine, destroying leased infrastructure and severing connections. The defensive system observed the sudden silence, preserved logs, and generated a preliminary incident report.

No human on either side made a real-time tactical decision during the three-hour engagement.

Why This Scenario Is Directionally Accurate

The fictional engagement extrapolates from documented capabilities.

GTG-1002 showed attack systems using Claude Code with MCP servers to decompose complex attacks into discrete tasks that appeared legitimate when evaluated in isolation. The “vibe hacking” operation disclosed in August 2025 showed attackers using AI to scan thousands of VPN endpoints, generate custom exploitation payloads, and craft psychologically targeted ransom notes.10

Factory AI’s October 2025 incident demonstrated the defensive corollary: building fraud classifiers in hours that achieved near-zero false positives while blocking attacks that adapted in real-time.11 They noted explicitly that countering AI-augmented attackers required AI-augmented defense.

Microsoft’s 2025 defensive infrastructure embeds similar principles: autonomous agents for alert triage, predictive shielding that anticipates attacker moves, automatic attack disruption at machine speed.12

The remaining gap between current systems and full autonomy involves reliable multi-step reasoning under adversarial conditions and removing human authorization at exploitation decisions. GTG-1002 retained human approval for credential use and exfiltration.13 Whether that gap closes in months or years depends on frontier model capabilities and deployment decisions, but the trajectory is clear.

Conditions for Defensive Equilibrium

The scenario depicts defense winning. That outcome is not guaranteed; GTG-1002 achieved objectives against some targets before detection. Defensive success depends on several conditions holding simultaneously.

Detection speed matching adaptation speed. The defensive system imposed friction faster than the attack system routed around it. Many organizations had not made this investment by 2025.

Home-field advantage. The monitoring system knew the baseline behavior of its own infrastructure. Attackers operating against unfamiliar targets must model normal behavior from external observation, a harder problem.

Attacker limitations constraining capability. The 2025 reports documented AI systems overstating findings and occasionally fabricating data during operations. Anthropic noted that Claude “frequently overstated findings and occasionally fabricated data,” claiming credentials that didn’t work or identifying discoveries that proved to be public information. These reliability problems create friction. More capable systems might succeed before defenses respond.

Symmetric autonomy levels. Neither side escalated to human decision-making during the tactical engagement. If attackers achieve full autonomy while defenders require human approval for responses that impose costs, attackers can exploit that asymmetry. The most dangerous periods may be transitions when one side has crossed the autonomy threshold and the other has not.

Asymmetries Beyond Autonomy

Even at equivalent autonomy levels, structural asymmetries shape outcomes.

Legal and liability constraints. Autonomous defensive responses that isolate systems, throttle access, or disrupt operations create liability exposure. Organizations and teams may face complaints and other consequences for false positives that harm legitimate users. Attackers face no such constraints. This asymmetry may slow deployment of aggressive autonomous defense even when technically feasible.

Regulatory fragmentation. Defenders operating across jurisdictions might need to navigate varying legal frameworks for automated response. Attackers choose favorable jurisdictions or operate outside legal reach entirely.

Attribution decay. If operations terminate cleanly with minimal forensics, as in the scenario, deterrence erodes. Punishment requires attribution. Machine-speed operations that destroy their own infrastructure undermine the forensic basis for response.

Access asymmetries. Frontier model access is currently concentrated among well-resourced actors, but this is shifting. Proliferation of capable open-weight models changes the equilibrium by lowering barriers to sophisticated autonomous offense.

These asymmetries generally favor offense, suggesting that technical parity in autonomy may not produce strategic parity in outcomes.

Implications

Three conclusions follow for those building or regulating these systems.

First, autonomous defense is not optional. The speed differential between machine-speed offense and human-approved defense creates exploitable gaps. Organizations that require human authorization for all active responses will be systematically disadvantaged. This requires rethinking liability frameworks to enable appropriate autonomous response.

Second, the transition period is uniquely dangerous. Asymmetric autonomy levels create windows of vulnerability. Defenders should assume adversaries are further along this curve than public evidence suggests and invest accordingly.

Third, attribution infrastructure matters more, not less. If machine-speed operations can terminate without forensic traces, pre-positioned attribution capabilities become essential for deterrence. This includes both technical measures (pervasive logging, cryptographic verification) and intelligence investments that could establish attribution through means other than post-incident forensics.

The stable equilibrium, if one exists, requires rough parity in autonomy combined with deliberate policy choices to counteract structural asymmetries favoring offense. Whether we achieve that equilibrium depends on decisions being made now about how aggressively to deploy autonomous defense and how to structure accountability for its failures.

There’s a conventional wisdom in international relations that feels almost self-evident: transparency is good for cooperation. If states can see what others are doing, detect violations, and share information openly, then compliance with international rules should improve. This logic has driven decades of institutional reform, from freedom of information policies to open data initiatives across global governance institutions.

Allison Carnegie and Austin Carson’s Secrets in Global Governance challenges this orthodoxy in a way that forces us to rethink some fundamental assumptions about how international cooperation actually works. The book’s central claim is both counterintuitive and, once you understand it, obvious: secrecy, when properly institutionalized within international organizations, can actually enhance rather than undermine cooperation.

The Disclosure Dilemma

The puzzle Carnegie and Carson identify is one that practitioners have grappled with for years but that has remained undertheorized in academic work. States frequently possess evidence of international rule violations but hesitate to share it. The reason isn’t lack of political will or indifference to violations. It’s that disclosure itself carries prohibitive costs.

Intelligence agencies risk exposing their sources and methods. A state might have satellite imagery proving another nation’s nuclear weapons program, but sharing it publicly could reveal surveillance capabilities worth billions and compromise future collection. Companies and governments holding commercially sensitive information face similar dilemmas. A firm may possess evidence of trade rule violations, but disclosing it could expose proprietary business strategies and damage competitive positioning. These costs are also referred to as adaptation costs.

The result of these adaptation costs being insurmountable is a cooperation trap. The very information needed to enforce international agreements remains locked away, not because states lack evidence, but because the act of sharing carries unacceptable risks.

The Institutional Solution

Carnegie and Carson’s proposed solution centers on international organizations equipped with robust confidentiality systems. These IOs function as trusted intermediaries that can receive, analyze, and act on sensitive information while preventing its wide release.

The mechanism works through several pathways. IOs can aggregate information from multiple sources, obscuring any single state’s contribution. They can translate raw intelligence into sanitized reports that preserve analytical value while removing identifying details. They can facilitate enforcement actions without requiring public attribution.

This represents a fundamental reconceptualization of how international organizations contribute to cooperation. Rather than simply increasing transparency to all audiences, effective IOs selectively manage information flows. They create protected channels for sensitive disclosures, establish credible commitments to confidentiality, and build relationships with the intelligence agencies and private actors who control critical information.

Evidence Across Domains

What makes the book particularly compelling is its empirical breadth. Carnegie and Carson test their theory across four substantively different domains: nuclear nonproliferation, international trade disputes, war crimes prosecutions, and foreign direct investment arbitration.

In the nuclear domain, they show how the International Atomic Energy Agency’s ability to receive and analyze intelligence about undeclared facilities depends critically on confidentiality protections. States share satellite imagery and signals intelligence with the IAEA precisely because the agency has developed credible systems to prevent wider disclosure. The World Trade Organization has evolved similar mechanisms for handling commercially sensitive information in subsidy disputes. International Criminal Tribunals use classified evidence procedures to prosecute war crimes without compromising intelligence sources. The International Centre for Settlement of Investment Disputes relies on confidential arbitration to resolve disputes involving proprietary business information.

The methodological approach combines statistical analysis, expert interviews and detailed case studies. This triangulation addresses both the scope conditions under which confidentiality systems operate and the causal mechanisms through which they facilitate cooperation.

Power, Accountability, and Trade-offs

To their credit, Carnegie and Carson don’t shy away from the normative complications their argument introduces. Confidentiality systems may enable cooperation, but they also concentrate informational power and potentially shield actors from accountability.

The nuclear cases reveal this clearly. The IAEA’s confidentiality system facilitates information sharing from the United States, United Kingdom, France, and Israel while doing little to protect states under investigation. The system essentially institutionalizes intelligence dominance, making power projection more efficient and legitimate. When the U.S. shares intelligence about Iranian nuclear activities with the IAEA, this advances nonproliferation goals defined primarily by existing nuclear powers.

There’s also the question of whether confidentiality systems actually work as advertised. The analysis assumes that IO confidentiality mechanisms are essentially secure, but the real-world track record is mixed. The WTO dispute settlement process regularly experiences strategic leaks despite elaborate confidentiality procedures. The IAEA has faced multiple security breaches. War crimes tribunals have struggled with witness protection failures.

Implications for AI Governance

The disclosure dilemma framework has direct relevance for emerging challenges in AI governance, where many of the same tensions around sensitive information are already apparent.

Consider the problem of safety incident reporting. AI developers may discover dangerous capabilities or near-miss incidents in their systems, information that would be valuable for the broader safety community. But public disclosure carries risks. Detailed technical information about model vulnerabilities could enable malicious actors. Information about internal safety practices could expose competitive advantages. Even aggregate incident data might reveal strategic priorities or technical approaches that companies consider proprietary.

The result is predictable underreporting. Just as states withhold intelligence about nuclear programs, AI labs stay silent about safety incidents unless disclosure is essentially costless. The information that would be most valuable for improving AI safety, the detailed technical specifics of failure modes and near-misses, is precisely the information least likely to be shared.

Current proposals for AI governance often assume that transparency will solve coordination problems. Mandatory model registries, public safety evaluations, and open-source development are all premised on the idea that more information sharing improves outcomes. Carnegie and Carson’s work suggests this may be backwards. Without confidentiality systems, transparency requirements might simply ensure that the most safety-critical information never enters the regulatory ecosystem at all.

What would an AI Safety Agency with proper confidentiality systems look like? It would need several features. First, credible technical capacity to analyze sensitive disclosures without requiring full public release. An agency staffed with ML researchers who can verify safety claims, assess reported vulnerabilities, and analyze incident data while protecting sources. Second, legal frameworks that protect disclosures from discovery in litigation. If companies fear that reporting a safety incident creates liability exposure, they won’t report. Third, international coordination mechanisms that allow information sharing across jurisdictions while maintaining confidentiality. A U.S.-based safety incident might have implications for AI systems deployed globally, but unilateral disclosure could compromise competitive positions.

The nuclear domain offers instructive parallels. The IAEA built credibility gradually, starting with simpler verification tasks and demonstrating information security before states entrusted it with more sensitive intelligence. An AI Safety Agency might follow similar developmental paths, beginning with less sensitive aggregated data before building toward the kind of detailed technical disclosure that would enable robust safety governance.

But there are also important differences. AI development is far more distributed than nuclear programs, involving not just state actors but private companies, academic researchers, and open-source communities. The disclosure dilemmas multiply accordingly. Commercial sensitivity matters more than in traditional security domains. The pace of capability development is faster, giving less time to build the trust relationships that effective confidentiality systems require.

Perhaps most challenging is the dual-use problem. Unlike nuclear technology, where the distinction between civilian and military applications is relatively clear, AI capabilities resist clean categorization. The same model architecture that enables beneficial applications might pose catastrophic risks. This makes it harder to define what information should be protected and what should be shared.

Rethinking Transparency

Secrets in Global Governance succeeds in fundamentally reframing how we think about information and cooperation. The traditional model positioning transparency as an unqualified good has dominated scholarship for decades, yet Carnegie and Carson demonstrate this framework fails to account for a critical class of problems where disclosure itself creates obstacles to cooperation.

The question isn’t whether transparency is universally beneficial. It’s when confidentiality systems can resolve specific cooperation problems that transparency alone cannot address. This matters immensely for practitioners designing new institutions. The devil is in the details. Effective confidentiality systems require careful institutional design, trusted relationships between IOs and states, and credible commitments to prevent information leakage.

For AI governance specifically, the timing is opportune. We’re still in the early stages of building international institutions for AI safety. The architectures we choose now will shape what’s possible later. If we build transparency-maximizing institutions without attending to disclosure dilemmas, we may create systems that look good on paper but fail to elicit the information actually needed for effective governance.

The alternative is harder but more promising. Building confidentiality systems requires patience, technical sophistication, and political will. It means accepting that some aspects of AI governance will operate behind closed doors. It means trusting institutions with sensitive information and living with the accountability trade-offs that entails.

These are uncomfortable choices, but ignoring disclosure dilemmas doesn’t make them go away. It just ensures that the information most critical for managing catastrophic AI risks stays locked in corporate vaults and national security bureaucracies, unavailable precisely when we need it most.

Carnegie, Allison and Austin Carson. 2020. Secrets in Global Governance: Disclosure Dilemmas and the Challenge of International Cooperation. Cambridge University Press.

The Evolving Frontier of AI-Assisted Cybercrime

For years, cybersecurity researchers have wrestled with a fundamental question: how much would artificial intelligence actually change the cybercrime landscape? The consensus view has generally positioned AI as a force multiplier rather than a game changer. These assessments suggested that while AI might make existing attacks more efficient or help with social engineering, the core requirement for technical expertise would remain. Bad actors would still need to understand systems, write code, and orchestrate complex operations. The barrier to entry for sophisticated cybercrime, we believed, would hold.

On August 27, Anthropic released a Threat Intelligence Report. While people mostly moved on from this pretty quick, it should force us to reconsider our assumptions. The report documents what the company calls "vibe hacking": a new operational model where cybercriminals use AI coding agents not just as assistants but as active participants in attacks. Going beyond better phishing emails or faster vulnerability scanning, AI systems are now autonomously executing multi-stage cyber operations that previously required teams of skilled operators.

The GTG-2002 Operation: Anatomy of an AI-Powered Attack

The centerpiece of Anthropic's report is the GTG-2002 operation, a sophisticated extortion campaign that targeted 17 organizations across healthcare, government, emergency services, and religious institutions. What makes this case remarkable isn't just its scope but its methodology. A single operator, using Claude Code as their primary tool, orchestrated what would typically require an entire cybercriminal team.

The term "vibe hacking" derives from vibe coding, a concept most readers are probably already familiar with. Popularized by AI researcher Andrej Karpathy, vibe coding describes using natural language to generate software through AI. Much in the same vein, this attack, the prototypical case for vibe hacking, uses primarily natural language interaction with an AI model as a means to execute attacks.

The attacker provided Claude Code with preferred tactics, techniques, and procedures through a configuration file, essentially creating an AI-powered attack framework. From there, Claude Code handled reconnaissance using open source intelligence tools, identified vulnerable targets through automated scanning, and executed penetration strategies. Once inside networks, the AI determined optimal paths for lateral movement, identified valuable data for exfiltration, and even analyzed victims' financial documents to calculate "realistic" ransom demands.

What's particularly striking is the sophistication of the AI-generated extortion materials. Claude created customized ransom notes that incorporated specific financial figures pulled from stolen documents, referenced industry-specific regulations to maximize pressure, and developed multi-tiered monetization strategies. One example from the report shows a "profit plan" that offered multiple options: direct organizational blackmail, data sales to criminals, or targeted extortion of individuals whose information was compromised. Ransom demands ranged from $75,000 to over $500,000 in Bitcoin.

Technical Innovation Without Technical Knowledge

Perhaps the most concerning aspect of this case is what it reveals about the current state and future potential for democratization of cybercrime capabilities. Traditional assumptions about the relationship between actor sophistication and attack complexity no longer hold when AI can provide instant expertise. The report notes that the attacker demonstrated "unprecedented integration of artificial intelligence," using Claude Code to make both tactical and strategic decisions about targeting, exploitation, and monetization.

The operation wasn't limited to simple data theft. The attacker compromised sensitive defense information regulated by International Traffic in Arms Regulations, healthcare records, financial data, and government credentials. The AI helped prioritize which data to exfiltrate based on its assessment of value and leverage, creating what amounts to an automated criminal decision-making system.

This represents a fundamental shift from how we've traditionally understood cybercrime operations. Previously, sophisticated attacks required either significant technical skill or access to specialized tools and infrastructure through criminal marketplaces. The GTG-2002 operation suggests a third path: leveraging commercially available AI tools to bridge the expertise gap. The attacker didn't need to understand the intricacies of network protocols or write custom exploitation code. They needed to understand how to direct an AI system toward malicious goals.

Another interesting implication of this is the potential depersonalisation of attacks. A key component in investigating and assigning responsibility for such hacks is in investigating the choice of tools, targets and demands - by effectively standardising the choices being made, this mode of attacks could significantly increase the effort required to identify the attackers. Indeed, even in this case, it seems that the perpetrator hasn’t been identified yet.

Beyond Individual Attacks: The Ecosystem Effect

The Anthropic report documents other cases that reinforce this pattern of AI-enabled capability enhancement. A UK-based actor with limited coding skills used Claude to develop and market ransomware variants featuring advanced evasion techniques, encryption mechanisms, and anti-recovery functions. The packages, sold for $400 to $1,200 on dark web forums, included features like direct syscall invocation and shadow copy deletion that would typically require deep Windows internals knowledge.

What's remarkable is the actor's apparent complete dependency on AI for development. As Anthropic notes, they "appear unable to implement complex technical components or troubleshoot issues without AI assistance, yet are selling capable malware." This isn't just about making cybercrime easier; it's about enabling entirely new categories of cybercriminals who lack traditional technical skills but can effectively direct AI systems.

The report also details North Korean operatives using Claude to secure and maintain remote employment at Fortune 500 companies, funneling salaries back to Pyongyang in violation of sanctions. By using AI to generate convincing resumes, pass technical interviews, and perform actual job tasks, these operatives have industrialized a previously limited scheme. The AI doesn't just help them fake competence; it provides actual technical capability they can leverage in real time.

Recalibrating Our Risk Models

These cases demand a fundamental recalibration of how we assess AI-related cyber risks. Previous frameworks, developed when AI assistants were primarily text generators, focused on risks like improved phishing or faster vulnerability discovery.

The emergence of vibe hacking as an operational reality rather than theoretical risk has immediate implications for cybersecurity strategy. Traditional defense models assume human-speed decision-making and predictable attack patterns. When AI serves as both the planner and executor of attacks, adapting in real-time to defensive measures, these assumptions break down.

Organizations need to reconsider their threat models to account for attackers who combine low technical skill with high AI capability. The traditional correlation between attack sophistication and threat actor resources no longer holds. A single individual with access to commercial AI tools can now mount operations that previously required organized crime groups or nation-state resources.

This also raises urgent questions about AI governance. Anthropic's response, including developing tailored classifiers and new detection methods, represents one approach. But the broader challenge is that the same capabilities that make AI valuable for legitimate software development make it dangerous in malicious hands. The line between vibe coding and vibe hacking is primarily one of intent, not technology.

The New Threat Landscape

Vibe hacking is a fundamental shift in the accessibility and scalability of sophisticated attacks. When a single actor can use AI to identify targets, craft attacks, execute intrusions, and optimize extortion strategies, we've entered a new era of cyber risk. Previous assessments that positioned AI as a helpful but limited tool for cybercriminals are now outdated. The question isn't whether AI will transform cybercrime but how quickly defenders can adapt to this new reality. The frontier of AI risk has shifted from theoretical concerns about model security to practical challenges of AI-powered operations happening today.

As we stand at this frontier, the Anthropic report serves as both warning and wake-up call. The age of AI-assisted cybercrime has given way to AI-automated cybercrime. Our risk assessments, defensive strategies, and governance frameworks need to evolve accordingly. The alternative is a future where the gap between AI-empowered attackers and traditional defenses continues to widen, with predictable consequences for organizations and individuals alike.

The Next Frontier is now available on Substack at thenextfrontier.blog. Subscribe for weekly analysis at the intersection of emerging technology and existential risk.

When I started this blog, I thought I knew what I wanted to write about. Like most people who begin publishing their thoughts online, I had strong opinions about specific things and assumed those opinions deserved their own corner of the internet. What I've discovered since then is that the most interesting questions don't respect the boundaries I tried to draw around them.

The Evolution

Over the past months, I've found myself increasingly drawn to concepts and thoughts which are important, interesting and relevant, yet also outside the scope I had initially envisioned. This corresponds to the various shifts my career has taken over the same time frame. I worked with the UK AISI on a project developing new infrastructure for evaluating protocols for controlling unaligned frontier LLMs. I went to Uppsala, and presented work on the risks and potential of advanced AI in nuclear verification. I presented a white paper on using AI to forecast the risk of coups at a workshop on crisis computing. And now I am in London, on a 3 month research assignment working to evaluate the scheming propensity of LLM agents. But more importantly, I talked to a lot of people regarding various topics, and I have come to realize that limiting my scope to AI gives an incomplete picture of the world. We need to have conversations that go beyond these thin lines, and explore the reality of a deeply interconnected world.

That's what The Next Frontier is becoming: a space for exploring the technologies, risks, and ideas shaping our trajectory as a species, with particular attention to the dangerous convergences others miss. Yes, that means writing about AI safety and nuclear risks, the two existential challenges that are my primary focus and increasingly intertwined as AI systems are integrated into defense infrastructure. But it also means examining things like historical developments that shaped the world, information warfare, biotechnology governance, and the broader challenge of managing technologies that evolve faster than our institutions can adapt.

The scope is admittedly ambitious. But I'd rather be honest about the full range of what interests me than artificially constrain the conversation. Some weeks that might mean a technical analysis of machine learning vulnerabilities in early warning systems. Other weeks it might mean exploring what medieval guilds can teach us about platform governance, or why the Montreal Protocol offers lessons for AI coordination. The thread connecting it all is a focus on understanding and navigating humanity's most consequential challenges.

Along with this evolution in focus, I'm moving The Next Frontier to Substack. Substack offers several advantages for what I'm trying to build. Substack allows me to build a community with ease, both through long-form content but also through notes and chats. Since I don’t plan to have paid subscriptions in the near future, Substack is also cheaper for me to maintain. The prevalence of Substack means subscribing is much more convenient for existing users and not a significant additional burden for those who aren’t. Most importantly, Substack has become where serious, long-form analysis lives online. The platform has cultivated an audience that actually reads, thinks, and engages with complex ideas, which is exactly the kind of community these topics require.

What to Expect

The new incarnation of The Next Frontier will feature:

• Regular Analysis – Weekly essays exploring specific risks, technologies, or governance challenges. These will aim for the sweet spot between academic rigor and accessibility, with enough depth to be useful for specialists but enough clarity for generally informed readers.

• Synthesis Pieces – Monthly long-form articles that connect insights across disciplines. These might explore how nuclear verification protocols could inform AI governance, or what the history of environmental regulation suggests about managing emerging technologies.

• Policy Briefs – Occasional focused assessments of specific proposals or frameworks, timed to ongoing policy debates when possible.

• Reading Notes – Shorter posts highlighting important papers, reports, or books that deserve wider attention, with key takeaways for those who won't read the originals.

I'm also planning some experimental formats: scenario planning exercises, dialogues between different expert perspectives, and what I'm calling "conceptual bridges"—posts explicitly designed to translate between different communities' ways of thinking about shared problems.

The Stakes

Why does any of this matter? Because we're living through a period of extraordinary technological transformation that coincides with rising geopolitical tension, ecological pressure, and institutional fragility. The decisions made in the next decade about AI governance, nuclear postures, climate adaptation, and information systems will reverberate for generations.

Yet most analysis of these issues happens in isolation. AI safety researchers rarely engage with nuclear strategists. Climate scientists don't always consider security implications. Policymakers struggle to keep pace with technical developments. We're trying to navigate interconnected challenges with fragmented understanding.

This blog is my small attempt to help connect those fragments. I don't pretend to have answers to these challenges—I'm skeptical of anyone who claims they do. But I believe that better questions, clearer analysis, and more integrated thinking can help us navigate what's coming.

Come Along for the Ride!

If you've been reading The Next Frontier already, thank you for coming along on this evolution. If you're discovering it for the first time, welcome to a conversation about humanity's biggest challenges and most important choices.

You can still find The Next Frontier at thenextfrontier.blog! All your favourite posts from the past are still accessible, just in a slightly different format, and all future posts will still appear in your inbox and on the website as usual.

Subscribe if you want to receive posts directly in your inbox. Comment if you have insights to add or questions to raise. Share with others who might find value in these explorations. Most importantly, bring your own expertise and curiosity to these discussions. The challenges ahead are too complex for any single perspective to grasp fully.

The next frontier isn't just about technology or policy or philosophy, it's about understanding how all of these intersect to shape our collective future. I'm looking forward to exploring it with you.

The Next Frontier is now available on Substack at thenextfrontier.blog. Subscribe for weekly analysis at the intersection of emerging technology and existential risk.

This series of blogposts was co-authored with Dr. Sophia Hatz, Associate Professor (Docent) at the Department of Peace and Conflict Research and the Alva Myrdal Centre (AMC) for Nuclear Disarmament, at Uppsala University. She leads the Working Group on International AI Governance within the AMC.

Logic and Strategy

In the early 1950s, the United States and Soviet Union were locked in a nuclear arms race, each developing ever more destructive weapons. President Eisenhower, seeking to ease global fears and reframe the nuclear narrative, unveiled the “Atoms for Peace” proposal in a speech to the UN General Assembly on December 8, 1953. In this address, Eisenhower acknowledged the grave risks of nuclear war but stated an American commitment to “find the way by which the miraculous inventiveness of man shall not be dedicated to his death, but consecrated to his life.”^[1] 

At its core, Atoms for Peace proposed an international exchange: countries would forgo nuclear weapons development in return for access to peaceful nuclear technology under international oversight. This arrangement created the foundation for the eventual Treaty on the Non-Proliferation of Nuclear Weapons (NPT), which took effect in 1970.^[2] The bargain involved nuclear powers contributing fissile material to an international "bank" supervised by the United Nations, with this material dedicated exclusively to peaceful applications in energy, medicine, and agriculture. Non-nuclear states would receive technological benefits while agreeing not to develop weapons, and nuclear-armed nations committed to gradual disarmament efforts. This framework attempted to balance two competing interests: preventing the spread of nuclear weapons while ensuring all nations could benefit from nuclear science's peaceful applications. The International Atomic Energy Agency (IAEA) emerged as the primary oversight body for this arrangement.

The United States leveraged its advanced position in nuclear technology to establish credibility and leadership in peaceful nuclear cooperation. This approach included passage of the Atomic Energy Act of 1954^[3], which created the legal basis for international nuclear cooperation, distribution of research reactors, specialized training, and nuclear fuel to allies and neutral countries, and establishment of the IAEA in 1957^[4] as an institutional embodiment of Eisenhower's vision. The United States also created verification mechanisms to ensure nuclear assistance wasn't diverted to military programs. By sharing nuclear expertise under controlled conditions, the United States positioned itself as the principal architect of the international nuclear order. This technical leadership reinforced American influence in shaping global nuclear norms and practices.

Beyond its stated humanitarian aims, Atoms for Peace functioned as a sophisticated Cold War strategy with multiple objectives. It presented American technological prowess in a benevolent light to counter Soviet influence and won support in developing nations by offering advanced technology and energy solutions. The initiative reassured European allies about America's commitment to peace while simultaneously encouraging their participation in NATO's nuclear deterrence strategy. It maintained strategic advantage by controlling which nuclear technologies were shared and with whom, and created a narrative of American leadership in peaceful development that contrasted with perceptions of Soviet militarism. President Eisenhower's initiative reflected his dual ambitions: to be recognized as a peacemaker while effectively containing communism's spread^[5]. The program's humanitarian framing masked its function as an instrument of geopolitical competition, allowing the United States to shape Cold War dynamics while appearing generous rather than aggressive.

Successes of Atoms for Peace

The Atoms for Peace program left a mixed legacy, but it achieved several notable successes that reshaped global nuclear affairs. Institution-building was perhaps its most enduring accomplishment. Eisenhower’s initiative directly led to the creation of the IAEA in 1957, introducing the concept of international nuclear inspections and safeguards. For the first time, an international body was empowered to monitor nuclear facilities to ensure material wasn’t diverted to bombs.

Atoms for Peace also fundamentally changed global attitudes toward nuclear energy. Eisenhower’s optimistic framing helped normalize the idea that nuclear technology could be a tool not just of war, but of progress and human well-being. In the decades following the speech, dozens of countries launched nuclear research and power programs with U.S. or international assistance. By the 1960s and 1970s, reactors provided electricity in nations from France to India, and isotopes improved medical treatments and agricultural techniques worldwide. A recent speech delivered by World Nuclear Association Director General Sama Bilbao y León to the Woodrow Wilson International Center for Scholars noted that Eisenhower’s “one seed” of an idea grew into a “forest of possibilities” in peaceful nuclear applications—from 400+ reactors supplying clean power to advances in cancer radiotherapy^[6]. While one can debate how much credit is due to Atoms for Peace versus other factors, the program undeniably accelerated the diffusion of scientific knowledge and nuclear infrastructure for civilian purposes. It made nuclear energy cooperation immensely popular and gave momentum to international scientific collaboration at a time of East–West polarization.

Crucially, Atoms for Peace established the normative groundwork for what became the global nonproliferation regime. As historian Peter Lavoy observes^[7], the initiative produced many key elements we now take for granted: “the International Atomic Energy Agency (IAEA), the concept of nuclear safeguards, and most importantly, the norm of nuclear nonproliferation” as an expected behavior of responsible states. Before Eisenhower’s proposal, there was little international consensus on how to reconcile peaceful nuclear pursuits with the prevention of weapons spread. Afterward, however imperfectly, a norm emerged that countries could access nuclear technology conditionally – i.e. if they committed not to build nuclear weapons and accepted inspections. A major consequence of the proposal was also the very idea of an eventual total nuclear disarmament. These elements were later codified in the NPT (1968), but its philosophical underpinnings trace back to Atoms for Peace. In short, Eisenhower’s gambit demonstrated the value of combining carrots and sticks: offering technological benefits and international legitimacy as incentives for arms restraint. That approach has remained central to nuclear governance and offers a potentially useful template for AI governance.

Shortcomings and Unintended Consequences

For all its idealism, Atoms for Peace had serious shortcomings that tempered its legacy. Nuclear proliferation did not halt—in some cases, it arguably accelerated under the cover of peaceful aid. As some analysts^[8] have noted, “it is legitimate to ask whether Atoms for Peace accelerated proliferation by helping some nations achieve more advanced arsenals than would have otherwise been the case. The jury has been in for some time on this question, and the answer is yes.” The program hastened the global diffusion of nuclear know-how, and some recipient nations – notably India, Pakistan, and Israel – diverted ostensibly peaceful assistance into their weapons development efforts. For example, India’s first nuclear explosive in 1974 was enabled by a Canadian-supplied reactor and U.S.-supplied heavy water from cooperative agreements in the 1950s. Similarly, research reactors and training provided under Atoms for Peace helped jump-start programs that later produced weapons in Israel and Pakistan. The hope had been that introducing international oversight would counteract the risks of sharing nuclear technology, but in the first decade, safeguards were weak or nonexistent. This enforcement gap meant countries could receive nuclear hardware and expertise while remaining outside any binding nonproliferation agreement until the NPT regime took hold years later.

Another shortcoming was the program’s geopolitical imbalance and resulting mistrust. In practice, Atoms for Peace became a largely U.S.-led, Western-centric enterprise. The Soviet Union, distrustful of American intentions, refused to pool any of its fissile material and dismissed Eisenhower’s plan as propaganda. Non-aligned nations in the Global South were also cautious: while many welcomed nuclear assistance, they resented the implicit bargain that demanded renouncing weapons options in exchange. Countries like India, which saw nuclear technology as a ticket to modernity and international stature, bristled at the notion of technology control by the great powers. The result was a trust deficit: some states felt the rules of the emerging nuclear order were being written for them rather than with them. This partisan dynamic undermined the spirit of cooperation that Eisenhower’s rhetoric espoused.

Critics also argue that Eisenhower’s emphasis on the “peaceful atom” carried a paradoxical message: it framed nuclear technology as the harbinger of peace and progress, yet implicitly accepted the permanence of superpower nuclear arsenals as the guarantor of world stability. In other words, Atoms for Peace did not directly pursue disarmament; it was a peace through strength and technology management, not a peace through eliminating the Bomb. This approach, which historian Ira Chernus terms “apocalypse management,”^[9] normalized the indefinite existence of nuclear weapons as a backdrop for peace. One could argue it sidestepped the deeper ethical question of the arms race and instead focused on managing its side-effects.

Lastly, enforcement and safeguards under Atoms for Peace were initially too lax to prevent determined proliferators from cheating. The IAEA eventually developed robust inspection practices, but in the late 1950s and 1960s, the regime lacked teeth – there were no mandatory inspections until states began joining the NPT. Israel received a research reactor from France and simply kept it outside of safeguards, using it to produce plutonium for weapons. Pakistan received extensive training of nuclear scientists through Western programs well before any oversight regime existed. The technology outpaced the governance in those early years, a cautionary tale for any similar framework in AI or other domains. It was only after proliferation cases came to light that the world tightened nuclear rules (for instance, India’s 1974 test led to the Nuclear Suppliers Group’s formation to better control exports). The Atoms for Peace era thus teaches that a voluntary, trust-based system can be exploited if not backed by verification and consequences. Balancing openness and security is incredibly challenging: too much openness can aid malign actors, while too much restriction stifles beneficial innovation. Eisenhower’s program attempted to walk that line, with mixed results.

1.  Atoms for Peace Speech | IAEA

2. Treaty on the Non-Proliferation of Nuclear Weapons (NPT) – UNODA

3.  Atomic Energy Act of 1954

4. The Statute of the IAEA | IAEA

5. Chernus, I. (Ed.). (2002). Eisenhower’s Atoms for Peace (1st ed). Texas A & M University Press.

6. Viewpoint: The legacy of Eisenhower’s Atoms for Peace speech. (2023). World Nuclear News. https://world-nuclear-news.org/articles/viewpoint-the-legacy-of-eisenhower-s-atoms-for-pea

7. Lavoy, Peter R. 2003. “The Enduring Effects of Atoms for Peace.” Arms Control Today, December 2003. https://www.armscontrol.org/act/2003-12/features/enduring-effects-atoms-peace.

8. Weiss, L. (2003). Atoms for Peace. Bulletin of the Atomic Scientists, 59(6), 34–44. https://doi.org/10.2968/059006009

9. Chernus, I. (Ed.). (2002). Eisenhower’s Atoms for Peace (1st ed). Texas A & M University Press.

This series of blogposts was co-authored with Dr. Sophia Hatz, Associate Professor (Docent) at the Department of Peace and Conflict Research and the Alva Myrdal Centre (AMC) for Nuclear Disarmament, at Uppsala University. She leads the Working Group on International AI Governance within the AMC.

The need to develop strategy and governance for advanced AI has become urgent, particularly as the catastrophic scale of risks from increasingly advanced AI models become clear, and as destabilizing competitive dynamics emerge. The parallels to nuclear technology and Cold War dynamics are perhaps more apparent now than then ever. But does the logic and strategy of ‘Atoms for Peace’ apply? Our analysis reveals three key challenges.

1. Finding a Grand Bargain for Advanced AI

Although AI is a dual-use technology, identifying a grand bargain is tricky. 

In comparison to nuclear weapons, AI’s risks are diverse and diffuse, ranging from smaller and more immediate risks such as algorithmic bias, privacy erosion, and job displacement to broader societal-scale concerns such as robust totalitarianism and loss of human control. Reaching international consensus on which AI risks are the most urgent to mitigate is difficult.

Further, while there is a relatively clear distinction between military and civilian applications of nuclear technology, it is difficult to distinguish between dangerous and beneficial AI. Frontier AI models –general-purpose AI models at the forefront of AI capabilities– offer the closest analogy for nuclear technology as these models could have dangerous capabilities sufficient to pose catastrophic risks^[1]. Yet, assessing whether AI model capabilities are approaching critical thresholds is much more complicated than determining whether a country’s nuclear activities are weapons-related or civilian. Dangerous AI capabilities can emerge unexpectedly, often in ways even developers struggle to anticipate or detect. And, unlike nuclear technology, there is no universally recognized “weapons-grade” AI model. 

For these reasons, current efforts to identify catastrophically-risk AI models often rely on proxies such as chips, compute or semiconductor supply chains, not necessarily because these are great proxies, but because precise thresholds for catastrophically dangerous AI capabilities are difficult to define.

Identifying “peaceful” uses of AI is equally challenging. Characteristics such as the opacity of AI models, their context-dependent risks, and their vulnerability to manipulation, make it difficult to verify the safety of AI systems. This makes it fundamentally difficult to leverage sharing the tremendous benefits of AI as an incentive. 

What would an AI grand bargain actually entail? Under Atoms for Peace, the exchange was clear: access to peaceful nuclear technology in return for international oversight and restrictions on weapons development. In the AI context, a comparable bargain would need to offer meaningful benefits (e.g., access to safe advanced AI models and resources) in exchange for verifiable safety commitments and oversight of high-risk applications. However, the technical challenges outlined above make this difficult to operationalize. Much research and policy work is needed –particularly in areas such as Frontier AI capability evaluations, safety standards and risk assessment– to define the core components of a grand bargain. 

2. Leveraging Technological Leads

Atoms for Peace leveraged America's nuclear lead to make credible commitments toward peaceful use. Assuming a coalition of the U.S. and its allies can sustain an advantage in Frontier AI and semiconductor supply chains, can this lead be used to commit to AI risk reduction? 

Several commitment problems complicate this endeavor. First, the lead must be substantial enough to give a ‘coalition of the cautious’ sufficient time to make meaningful progress on AI safety. However, recent developments—such as rapid scaling, model distillation and the immanence of AI agents—create uncertainty about both how long the U.S. and its allies can maintain an advantage and how long it will take to develop the necessary technical and governance solutions. Second, the coalition must credibly commit to moving cautiously on Frontier AI development—a challenge made particularly difficult by intensifying U.S.-China competition for AI dominance, which incentivizes a ‘race to the bottom’ in safety. Finally, the coalition must ensure that its risk-reduction efforts do not inadvertently accelerate the development of dangerous AI capabilities. Safety research itself can contribute to capability gains— historically several fields of AI safety research, such as interpretability and preference learning, have driven developments leading to more powerful AI models and this is likely to continue in the future.

These commitment problems reveal important differences from the nuclear precedent. When the U.S. leveraged its nuclear lead through Atoms for Peace, it did so from a position of overwhelming technological advantage that would persist for years. In contrast, the AI landscape is unpredictable, with leadership positions potentially shifting in months rather than decades. Additionally, while nuclear technology development requires massive state-backed infrastructure, AI development is more distributed and often led by private companies with their own competitive incentives. 

Addressing these commitment problems will require not only policy levers, but also technical mechanisms that enable verifiable commitments to cautious AI development. 

3. Strategic competition

Finally, we need to consider what a ‘Cold War strategy’ would look like in the context of AI. The Cold War emerged post-WWII, featuring stark ideological blocs and existential nuclear fears – a context arguably different from today's more complex geopolitical landscape, fluid alliances, and the specific nature of US-China competition. 

Nonetheless, some current strategies echo past containment efforts. Coordinated export controls on high-performance AI chips and manufacturing equipment, for instance, aim to slow AI progress in rival states and consolidate a technological lead for a U.S.-led coalition. While the context differs, the logic mirrors Cold War-era technology containment strategies, where the West restricted access to sensitive military and dual-use technologies to prevent Soviet advancements.

The lesson from Atoms for Peace is cautionary: while it helped establish a U.S.-led nuclear order, it also reinforced Cold War divisions, pushing adversaries toward independent weapons programs and fueling decades of arms racing. Similarly, strict AI export controls could drive states outside the coalition toward self-sufficiency in AI infrastructure. This could result in a bifurcated AI ecosystem with incompatible AI safety standards and lower prospects for broad international coordination. 

More dangerously, export controls could reinforce the perception that AI leadership is a zero-sum game, causing restricted states to shift their focus toward AI military applications, and perhaps leading to an AI-arms race. In this scenario, with rival states racing to deploy AI-enhanced capabilities before their adversaries, integrating not-fully-safe advanced AI into military functions ranging from autonomous weapons to strategic decision-support, the result could be greater unpredictability in strategic decision-making, heightened crisis instability and an increased risk of unintended conflict escalation.

Conclusion

The AI-nuclear analogy is valuable not because the technologies are highly similar, but because examining historical precedents can help structure our thinking about the governance of powerful dual-use technologies. Our analysis of Atoms for Peace reveals that applying a similar logic and strategy to advanced AI would face significant challenges.

A more effective strategy could take inspiration not just from Atoms for Peace but from the broader concept of nonproliferation. While nuclear nonproliferation serves a specific function in the context of Atoms for Peace, it has since become a broader international norm and approach to mitigating catastrophic risks from dangerous technologies.

Applying this broader concept, AI nonproliferation would not be narrowly defined as controlling model weights or restricting access to high-performance chips, since these alone do not determine AI’s risk profile. Instead, a broader notion of AI nonproliferation would focus on controlling dangerous AI capabilities—particularly those that disproportionately carry societal-scale or catastrophic risks. As the science of AI capability evaluations improve, and governments become better equipped with empirical evidence, we could move away from relying on proxies.

Further, AI nonproliferation could focus not only on preventing the spread of Frontier AI to actors without adequate safeguards (horizontal proliferation) but also on curbing the unchecked escalation of AI capabilities (vertical proliferation). Atoms for Peace primarily focused on horizontal proliferation, and this focus is maintained in some analogous proposals^[2] for AI. Yet, vertical AI proliferation is equally destabilizing—if one actor pushes too far and too fast, AI capabilities could exceed our ability to control them before safety mechanisms are fully developed. 

If-then commitments exemplify an approach focused on preventing the vertical proliferation of dangerous AI capabilities. The general idea is that AI developers commit to pause, delay, or limit development and deployment when specific "trip-wire capabilities" are detected through ongoing evaluations. This approach provides some mechanisms to address each of the three key challenges we've identified.

First, if-then commitments do not require broader international agreement on which risks should be prioritized or what the weapons-grade equivalent of advanced AI is ex ante. Instead, AI developers agree that certain situations would require risk mitigation if they came to pass. In avoiding unnecessary restrictions on innovation, if-then commitments support the development of “peaceful” AI. In other words, the grand bargain is that developers maintain freedom to innovate and advance AI capabilities, while accepting pre-defined limits when specific risk thresholds are reached. This also makes a grand bargain more technically feasible and more politically palatable, as it grounds governance in empirical evidence rather than speculative risks.

Second, if-then commitments address some specific commitment problems. For example, by pre-defining specific "trip-wire capabilities" and conditionally committing to risk mitigation, they add credibility to commitments to develop responsibly. Further, if-then commitments could mitigate the "racing" dynamic that otherwise incentivizes cutting corners on safety. If all leading developers were to publicly commit to the same capability triggers, the competitive advantage of rushing ahead diminishes, as reaching certain capability thresholds would activate the pre-committed limitations regardless of who gets there first.

Third, they may help avoid a bifurcated AI ecosystem by creating governance frameworks that can function across geopolitical divides. Unlike export controls focused on hardware, capability-based commitments can be adopted by any developer regardless of nationality, potentially enabling inclusive cooperation around shared safety standards. Constructive relationships could be established with Global South nations based on a shared commitment towards open tech transfers and ensuring a safe and flourishing future with advanced AI. This is necessary in order to avoid repeating the mistakes of the Atoms for Peace, where badly framed measures and inconsistent application of rules caused a loss of trust and led to countries seeking other partners. 

While if-then commitments currently exist in some form in responsible scaling policies and other policies voluntarily put forth by leading AI companies, these could eventually be enforced by an international regime, much the way the IAEA and NPT institutionalized nuclear nonproliferation following Atoms for Peace. Eventually, this could lead to agreements on restrictions and safety measures for dangerous AI use cases, including lethal autonomous weapon systems and nuclear weapon command and control.

If-then commitments illustrate that we don’t necessarily need universal agreement on AI risks or for cautious developers to maintain an undisputed lead in order to slow the proliferation of dangerous AI capabilities. However, for AI nonproliferation to work, we do need a robust science of capability evaluation that can reliably detect when AI models approach dangerous thresholds, as well as robust verification mechanisms. Just as voluntary oversight proved insufficient in nuclear governance, AI governance initiatives must incorporate mandatory, transparent evaluations and credible enforcement from the outset. By carefully balancing openness and security, proactively addressing geopolitical mistrust, and designing adaptive institutions, AI governance can avoid repeating historical mistakes. Ultimately, effective AI nonproliferation will depend on clear capability thresholds, rigorous verification, and inclusive international cooperation—all of them areas where AI governance can learn from nuclear governance.

1. Anderljung, M., Barnhart, J., Korinek, A., Leung, J., O’Keefe, C., Whittlestone, J., Avin, S., Brundage, M., Bullock, J., Cass-Beggs, D., Chang, B., Collins, T., Fist, T., Hadfield, G., Hayes, A., Ho, L., Hooker, S., Horvitz, E., Kolt, N., Schuett, J., Shavit, Y., Siddarth, D., Trager, R., and Wolf, K. (2023). Frontier AI Regulation: Managing Emerging Risks to Public Safety. ArXiv preprint arXiv:2307.03718v4

2. For example, see “Nonproliferation” in Chips for Peace and Superintelligence Strategy.

This series of blogposts was co-authored with Dr. Sophia Hatz, Associate Professor (Docent) at the Department of Peace and Conflict Research and the Alva Myrdal Centre (AMC) for Nuclear Disarmament, at Uppsala University. She leads the Working Group on International AI Governance within the AMC.

In 1953, amid Cold War tensions and the looming threat of nuclear annihilation, U.S. President Dwight D. Eisenhower proposed a bold idea: harness the power of the atom for peace, not war. In his famous “Atoms for Peace” speech, Eisenhower proposed to prevent the spread of nuclear weapons by promoting peaceful applications of nuclear technology. This initiative laid the groundwork for the institutions, norms, and governance frameworks that continue to shape nuclear policy today.

Seven decades later, the world faces a new transformative technology—advanced Artificial Intelligence (AI). While AI differs from nuclear technology in many ways, its governance poses similarly high stakes. Rapidly advancing AI capabilities could lead to catastrophic outcomes^[1], for example via misuse by malicious actors, accelerating arms races and the erosion of human control. Yet, with adequate safeguards in place, advanced AI could drive unprecedented scientific advancements and economic prosperity. Recognizing this, analysts frequently draw on nuclear governance as a model for AI governance, with some proposals^[2],[3] explicitly drawing on Eisenhower’s framework.

In this two-part series, we take a closer look at "Atoms for Peace" in order to help readers better assess proposals for AI which invoke elements of this framework. Part 1 explores how Atoms for Peace shaped nuclear governance, detailing its logic, successes, and shortcomings. Part 2 identifies key challenges in adapting this framework to advanced AI and suggests a broader approach to AI nonproliferation. We provide a summary of key takeaways below.

Summary

Eisenhower's initiative sought to prevent nuclear Armageddon through a grand bargain: nations would accept international oversight and limitations on weapons development in exchange for access to peaceful nuclear technology and expertise. The U.S. used its significant lead in nuclear technology to make binding commitments toward peaceful use and risk reduction. Simultaneously, Atoms for Peace served as a Cold War strategy, reinforcing U.S. technological leadership while containing Soviet influence. This approach contributed to the creation of institutions such as the International Atomic Energy Agency (IAEA) and helped shape international norms around nuclear cooperation. However, it also had unintended consequences, including the acceleration of nuclear proliferation in some cases and the entrenchment of Cold War divisions.

Several challenges complicate the endeavor of adapting the logic of Atoms for Peace to AI governance.  First, establishing a 'grand bargain' – trading access to benefits for restrictions on dangerous uses – is challenging because AI lacks an obvious "weapons-grade" equivalent, and reliably verifying the safety of advanced AI is currently very difficult. Second, intense global competition creates powerful incentives to prioritize rapid development, making it hard for leading nations or developers to credibly commit to prioritizing safety or leverage a technological lead for risk reduction. Finally, attempting to control AI proliferation by restricting access to essential hardware like advanced chips risks repeating Cold War dynamics; such measures could deepen geopolitical divides, potentially provoke an AI arms race, and increase global instability.

We suggest AI governance could benefit from a broader concept of nonproliferation. This concept would move beyond hardware restrictions to focus on preventing dangerous AI capabilities. We examine "if-then commitments" as one example of a capability-focused approach. This approach could help address some identified challenges. The bargain in if-then commitments involves allowing continued AI development unless pre-agreed "capability triggers" are flagged during evaluations. This relies on defining thresholds for potentially dangerous AI capabilities, could enhance the credibility of safety commitments, and may create space for inclusive global cooperation on shared safety standards.

For full text, refer to:

1. Lessons for AI Governance from Atoms for Peace, Part 1: Atoms for Peace

2. Lessons for AI Governance from Atoms for Peace, Part 2: Chips for Peace?

1. Hendrycks, D., Mazeika, M., & Woodside, T. (2023). An Overview of Catastrophic AI Risks (No. arXiv:2306.12001). arXiv. https://doi.org/10.48550/arXiv.2306.12001

2. Roberts, P. S. (2019). AI for Peace. War on the Rocks. http://warontherocks.com/2019/12/ai-for-peace/

3. O’Keefe, C. (2024). Chips for Peace: How the U.S. and Its Allies Can Lead on Safe and Beneficial AI. Lawfare. https://www.lawfaremedia.org/article/chips-for-peace--how-the-u.s.-and-its-allies-can-lead-on-safe-and-beneficial-ai

In recent years, the rapid advancement of artificial intelligence (AI) has captivated the public imagination, with large language models (LLMs) like ChatGPT showcasing seemingly magical abilities. These AI marvels have become synonymous with technological progress, promising a future where machines can understand and generate human-like text, solve complex problems, and even engage in creative tasks. However, beneath the glossy surface of these AI wonders lies a complex and often troubling reality - the essential role of human workers who make these systems possible through data annotation and content moderation.

The Technical Necessity of Human Labor

At its core, the training of LLMs requires massive amounts of carefully curated and labeled data to function effectively. This process involves feeding the model text data and teaching it to understand and generate human language through various forms of annotation. Data annotation encompasses labeling, categorizing, tagging, and adding attributes to help machine learning models process and analyze information better. The training process typically follows several key steps.

First, vast amounts of text data must be collected from diverse sources like books, articles, and web content. This raw data then needs to be cleaned, preprocessed, and prepared for training through processes like tokenization and formatting. However, the most critical and labor-intensive aspect is the human annotation required to label this data appropriately. Different types of annotation are needed depending on the specific requirements. Manual annotation involves humans directly reviewing and labeling data, while semi-automatic annotation combines human oversight with algorithmic assistance. For tasks requiring high accuracy and nuanced understanding, human annotators remain irreplaceable despite advances in automated systems.

Exploitation and Neglect in the AI Workforce

To meet the massive demand for annotated training data, tech companies have built a vast network of workers, primarily in the Global South. These workers, often paid minimal wages, perform the crucial but mentally taxing work of labeling data and moderating content. Countries like Kenya, India, the Philippines, and Venezuela have become hubs for this digital labor, with large populations of well-educated but underemployed individuals providing a ready workforce for tech giants. In Kenya, for instance, workers employed by companies like Sama have been paid as little as $1.32 to $2 per hour to review and label content for tech giants like OpenAI and Meta. These workers are often required to review disturbing content including violence, hate speech, and explicit material for up to 8 hours daily.

The psychological toll of this work is severe, with many workers reporting trauma, nightmares, and substance abuse issues to cope with the constant exposure to disturbing content. Naftali Wambalo, a Kenyan worker interviewed by CBS News, described his experience: "I looked at people being slaughtered, people engaging in sexual activity with animals. People abusing children physically, sexually. People committing suicide." He added, "Basically- yes, all day long. Eight hours a day, 40 hours a week." The impact of this work on Wambalo's personal life was profound, affecting even his intimate relationships.

The outsourcing model exacerbates these challenges by creating a layer of separation between tech companies and the workers they rely on. Companies like OpenAI or Meta contract third-party firms or use digital labor platforms to recruit annotators and moderators. This arrangement allows them to avoid direct responsibility for poor working conditions while benefiting from the labor that powers their AI systems. Workers are often hired as independent contractors rather than employees, stripping them of benefits like healthcare, job security, or access to grievance mechanisms. The precarious nature of this employment makes it difficult for workers to advocate for better conditions without risking their livelihoods. For instance, employees at Sama who moderated content for OpenAI’s ChatGPT project reported being required to label hundreds of text passages per shift under tight deadlines. Despite OpenAI paying Sama $12.50 per hour per worker for this service, the annotators themselves received only a fraction of that amount—typically $1.32 to $2 per hour after taxes.

Adding to the exploitation is the lack of mental health support for workers exposed to traumatic content. While some companies offer wellness programs or counseling services, these efforts are often superficial and inadequate. Workers report being shuffled between tasks rather than receiving meaningful psychological care. The emotional toll is compounded by constant surveillance and performance monitoring, which create a high-pressure environment where even minor deviations can lead to penalties or termination.

Opacity and Accountability in AI Development

The lack of transparency in how AI systems are developed and maintained poses significant ethical concerns. Tech companies often present AI as fully automated systems that function independently of human input. In reality, these systems depend heavily on human labor for tasks such as data labeling, content moderation, and quality assurance. This "human-in-the-loop" model is rarely disclosed to the public, perpetuating the illusion of automation while obscuring the labor that makes it possible.

This opacity extends to the treatment of workers within the AI supply chain. Many companies fail to disclose basic information about their labor practices, such as how much they pay annotators or what steps they take to ensure fair working conditions. This lack of accountability allows companies to sidestep scrutiny while continuing exploitative practices. Even when issues like low wages or poor mental health support come to light, companies often deflect responsibility by pointing to their subcontractors.

The absence of transparency also undermines efforts to address systemic issues within the industry. Without clear reporting on labor practices, it becomes difficult for policymakers, researchers, or advocacy groups to hold companies accountable or propose meaningful reforms. Initiatives like the AI Labor Disclosure Initiative (AILDI) aim to address this gap by advocating for regular reporting on digital labor practices. However, such efforts remain voluntary and lack the enforcement mechanisms needed to drive widespread change.

Bias in Data Annotation

Bias in data annotation is another critical issue that stems from the human labor involved in training AI systems. Annotators bring their own cultural perspectives, experiences, and biases into their work, which can influence how data is labeled. An annotator's understanding of hate speech may vary depending on their cultural background or personal beliefs. Many data annotation tasks are outsourced to regions where workers may not fully understand the cultural context of the data they are labeling. Instructions are often provided in English or other dominant languages, creating additional barriers for non-native speakers. This disconnect can result in inconsistent or inaccurate annotations that compromise the fairness and reliability of AI systems.

Annotation bias is particularly problematic because it often goes unnoticed until it manifests in real-world applications. For instance, facial recognition systems trained on datasets lacking diversity have been shown to perform poorly on individuals with darker skin tones. Similarly, natural language processing models can perpetuate stereotypes if their training data reflects societal prejudices.

A Structural Problem Across Industries

The exploitation of data workers is not an isolated issue but a symptom of broader structural problems within the tech industry. The push for rapid innovation and cost efficiency has created a system where human labor is undervalued and hidden from view. This dynamic mirrors historical patterns of industrial exploitation but is exacerbated by the globalized nature of today's economy.

Social media giants like Meta (formerly Facebook) also rely heavily on outsourced labor for content moderation. Workers at Sama who moderated Facebook content described similar conditions: low pay, exposure to traumatic material, and inadequate mental health support. Similar dynamics can be seen in industries like fast fashion or electronics manufacturing, where low-cost labor fuels high-margin products sold primarily in affluent markets.

Outsourcing plays a central role in this system by enabling companies to shift labor-intensive tasks to regions with lower wages and weaker labor protections. This model creates a race to the bottom where workers in developing countries compete for contracts by accepting substandard pay and conditions. At the same time, it allows tech companies in wealthier nations to reap enormous profits while avoiding accountability for exploitative practices.

Fair Compensation and Improved Working Conditions

The foundation of ethical AI development lies in ensuring fair compensation and dignified working conditions for the laborers who power these systems. Data workers are often paid wages that fail to meet basic living standards. Companies must commit to paying living wages that reflect the value of this labor while also providing essential benefits such as healthcare, job security, and paid time off. Beyond financial equity, improving working conditions is equally vital. Workers need realistic productivity targets that do not overburden them, adequate rest periods to prevent burnout, and safe environments free from exploitation or undue pressure. These changes would not only improve worker well-being but also enhance the quality of the AI systems they help create.

The psychological toll on data annotators and content moderators is one of the most pressing ethical issues in AI development. Companies must take proactive steps to mitigate these harms by offering comprehensive mental health support. This includes access to trauma-informed counselors, regular mental health check-ins, and wellness programs designed to alleviate stress. Employers should also explore technological solutions that limit human exposure to harmful content, such as using AI tools to pre-filter graphic material before it reaches human reviewers. By prioritizing mental health alongside operational goals, companies can create a more humane workplace for their employees.

Empowering Workers

Empowering data workers is essential for creating a more equitable AI ecosystem. A lack of transparency in AI supply chains enables companies to distance themselves from exploitative labor practices. Companies must adopt transparent supply chain practices that ensure ethical labor standards are upheld at every level. This includes direct engagement with workers or their representatives in shaping workplace policies and conducting regular audits to monitor compliance with labor standards. Additionally, companies should establish safe channels for workers to report grievances or suggest improvements without fear of retaliation and involve them in decision-making processes related to their roles, treating them as collaborators rather than expendable resources. Supporting unionization efforts and transnational organizing can provide workers with a collective voice to advocate for better wages, protections, and working conditions. Worker-led initiatives have already shown promise in amplifying the perspectives of those directly affected by exploitative systems.

Conclusion

Beyond immediate reforms, the industry must fundamentally rethink its approach to progress. These AI systems are rapidly becoming commonplace and have become the hot topic of the tech world. However, the exploitation of data workers to build these models is not an inevitable cost of progress; it is a choice made by an industry that prioritizes profit over people. To build truly responsible AI systems, we must reimagine how we value human labor within the tech ecosystem. This means not only addressing immediate concerns around wages and working conditions but also fostering a culture of accountability that recognizes the dignity and humanity of all workers—whether they are coding algorithms in Silicon Valley or labeling data in Nairobi. By shining a light on this hidden workforce and demanding systemic change, we can ensure that the benefits of AI are shared equitably across society—not built on the backs of its most vulnerable contributors.

Join us next week when we will dive in on the seemingly sudden rise of DeepSeek and what it means for the broader AI ecosystem! If you found this article through a direct link, please consider subscribing! Subscribing lets you express your support and give feedback for free and ensures that you are always first in line to receive our articles.

Citations

Bartholomew, J. (2023, August 29). Q&A: Uncovering the labor exploitation that powers AI. Columbia Journalism Review. https://www.cjr.org/tow_center/qa-uncovering-the-labor-exploitation-that-powers-ai.php

Chen, A. (2014, October 23). The Laborers Who Keep Dick Pics and Beheadings Out of Your Facebook Feed. Wired. https://www.wired.com/2014/10/content-moderation/

Coldewey, D. (2024, July 8). Data workers detail exploitation by tech industry in DAIR report. TechCrunch. https://techcrunch.com/2024/07/08/data-workers-detail-exploitation-by-tech-industry-in-dair-report/

Dzieza, J. (2023, June 20). Inside the AI Factory. The Verge. https://www.theverge.com/features/23764584/ai-artificial-intelligence-data-notation-labor-scale-surge-remotasks-openai-chatbots

Gebrekidan, F. B. (2024). Content moderation: The harrowing, traumatizing job that left many African data workers with mental health issues and drug dependency. In: M. Miceli, A. Dinika, K. Kauffman, C. Salim Wagner, and L. Sachenbacher (eds.) The Data Workers' Inquiry. https://data-workers.org/fasica 

Meaker, M. (2023, September 11). These Prisoners Are Training AI. Wired. https://www.wired.com/story/prisoners-training-ai-finland/

Okolo, C. T., & Tano, M. (2024, October 24). Moving toward truly responsible AI development in the global AI market. Brookings. https://www.brookings.edu/articles/moving-toward-truly-responsible-ai-development-in-the-global-ai-market/

Perrigo, B. (2023, January 18). Exclusive: The $2 Per Hour Workers Who Made ChatGPT Safer. TIME. https://time.com/6247678/openai-chatgpt-kenya-workers/

Ranta, B. N. D. (2024). The Unknown Women of Content Moderation. In: M. Miceli, A. Dinika, K. Kauffman, C. Salim Wagner, & L. Sachenbacher (eds.), The Data Workers‘ Inquiry. https://data-workers.org/ranta

Stahl, L. (2024, November 24). Labelers training AI say they’re overworked, underpaid and exploited by big American tech companies. CBS News. https://www.cbsnews.com/news/labelers-training-ai-say-theyre-overworked-underpaid-and-exploited-60-minutes-transcript/

Taylor, B. L. (2023, November 15). Long hours and low wages: The human labour powering AI’s development. The Conversation. http://theconversation.com/long-hours-and-low-wages-the-human-labour-powering-ais-development-217038

Bracy, C., & Dark, M. (2023, September 13). The Ghost Workforce the Tech Industry Doesn’t Want You to Think About (SSIR). https://ssir.org/articles/entry/the_ghost_workforce_the_tech_industry_doesnt_want_you_to_think_about

Brandom, R. (2023, June 29). The industry behind the industry behind AI. Rest of World. https://restofworld.org/2023/exporter-industry-behind-ai/

Muñoz, F. M. (2023, October 9). Unmasking the Shadows of AI: Ethical Concerns in the Digital Workforce. Kognic. https://www.kognic.com/articles/shadows-of-ai

Williams, A., Miceli, M., & Gebru, T. (2022). The Exploited Labor Behind Artificial Intelligence. Noema. https://www.noemamag.com/the-exploited-labor-behind-artificial-intelligence

In the ever-evolving landscape of artificial intelligence (AI), the ability to reason and solve complex mathematical problems has become a critical benchmark for evaluating advanced AI capabilities. Among the various tools developed for this purpose, Frontier Math stands out as a pioneering benchmark designed to assess the limits of AI in mathematics. This post delves into the significance of Frontier Math, its structure, and its implications for understanding AI progress, particularly in light of recent advancements demonstrated by OpenAI's new model, o3.

The Importance of Frontier Math

Frontier Math was introduced in a paper first published by Epoch AI in November 2024, to address a significant gap in existing AI benchmarks. Traditional benchmarks, such as MATH and GSM8K, have become saturated; many AI models now achieve near-perfect scores on these tests, which often consist of simpler problems that do not adequately challenge advanced reasoning capabilities. In contrast, Frontier Math comprises hundreds of original and unpublished problems crafted by a team of over 60 mathematicians from prestigious institutions like MIT and UC Berkeley. These problems are specifically designed to push AI models to their limits, requiring deep mathematical knowledge and innovative problem-solving approaches.

The benchmark's complexity lies in its interdisciplinary nature, covering approximately 70% of modern mathematical fields, including number theory, algebraic geometry, and combinatorics. Each problem demands not just computational skills but also a profound understanding of mathematical concepts and the ability to apply them creatively. This makes Frontier Math a robust tool for evaluating whether AI systems can engage in genuine mathematical reasoning akin to that of human experts.

Structure and Design of Frontier Math

Frontier Math is unique in its approach to problem formulation. Each problem is designed to be challenging enough that even expert mathematicians may require hours or days to arrive at a solution. The benchmark emphasizes "guess-proof" questions that resist simple pattern recognition or brute-force strategies commonly employed by AI systems. For example, one problem might involve constructing high-degree polynomials with specific properties contextualized within geometric scenarios. Such tasks necessitate not only advanced computational abilities but also a creative application of algebraic geometry principles.

The design criteria for Frontier Math problems focus on several key aspects:

• Originality: Problems are unpublished and crafted specifically for this benchmark.

• Difficulty: They are intended to be significantly more challenging than those found in traditional benchmarks.

• Verifiability: Solutions can be rigorously evaluated using automated verification systems, ensuring objective assessments without human bias.

This rigorous design process ensures that Frontier Math can effectively measure the progress of AI models as they strive toward expert-level mathematical reasoning. This makes FrontierMath a credible and legible marker for progress in mathematical reasoning.

Examples from the Benchmark

To better understand the nature of challenges posed by Frontier Math, consider some representative examples:

1. Artin's Conjecture on Primitive Roots: This high difficulty problem requires an understanding of both number theory and algebraic structures. Solving it necessitates innovative approaches that go beyond standard techniques.

2. High-Degree Polynomial Construction: A high-medium task involving the creation and verification of high-degree polynomials with specific properties requires both computational efficiency and deep theoretical insights.

These examples illustrate how Frontier Math transcends traditional benchmarks by demanding multi-step logical reasoning akin to research-level mathematics. This is evident by the reactions of at least two Fields medalists who suggested that the hardest questions in the benchmark were "extremely challenging" and "well beyond what we can do now".

Performance of Current Models on FrontierMath

Despite the advanced capabilities of modern AI systems, FrontierMath has proven to be an exceptionally challenging benchmark. The evaluation process for this benchmark is designed to give AI models the best possible chance at success. Models are provided with ample time for reasoning and are given access to a Python environment where they can write and execute code, test hypotheses, verify intermediate results, and refine their approaches based on immediate feedback.

However, even with these supportive conditions, the performance of leading AI models on FrontierMath has been strikingly low. Epoch AI evaluated six of the most advanced language models available, including Claude 3.5 Sonnet, o1-preview, GPT 4o, and Gemini 1.5 Pro. None of these cutting-edge models could solve more than 2% of the problems presented in the FrontierMath benchmark.

The low success rate on FrontierMath revealed a significant gap between the current capabilities of AI systems and the level of mathematical reasoning required for research-level mathematics. While AI had made impressive strides in many areas, including some aspects of mathematical problem-solving, FrontierMath demonstrates that there is still a considerable distance to cover before AI can match the depth of understanding and creative problem-solving abilities of expert human mathematicians.

Results from OpenAI's o3 Model

A notable development in the context of Frontier Math is OpenAI's introduction of the o3 model, which has achieved unprecedented results on this benchmark. In December 2024, OpenAI announced that o3 solved 25.2% of the problems presented in Frontier Math—an extraordinary leap from previous models that struggled to solve even 2%. This performance not only highlights significant advancements in AI reasoning capabilities but also raises important questions about what these results mean for the future of AI development.

The success of o3 can be attributed to its enhanced architecture and training methodologies that prioritize complex reasoning tasks. Unlike earlier models that relied on "predicting" the solution directly, current public information suggests that reasoning models like o3 simulate an entire chain of thought process alongside a process of generating multiple responses internally and then evaluating them to pick the best response. This makes them exceptionally well-suited to be used for complex reasoning tasks such as those present in the FrontierMath dataset.

However, there are still doubts regarding whether these results are actually accurate or representative of the model's performance. Public access to these models is still not available and prior experience suggests that performance by advanced models is often brittle - while they perform certain tasks very well a lot of the time, they fail in completely unexpected manners outside those instances. This places strong limits on the utility of these models. In light of the reveal of OpenAI's funding of FrontierMath there have also been accusations that OpenAI trained or finetuned o3 on the benchmark data before testing it. Regardless, only independent testing and evaluations can confirm or deny any claims about the model.

Critical Examination of Frontier Math

While Frontier Math represents a significant advancement in evaluating AI capabilities, it is essential to critically examine its limitations and implications. One concern is the focus on verifiable solutions at the expense of open-ended exploration—a vital aspect of modern mathematical research. Many contemporary mathematical problems do not have straightforward answers or require lengthy proofs that cannot be easily verified through automated systems. As such, while Frontier Math provides valuable insights into AI's problem-solving abilities, it may not fully capture the nuances of human mathematical reasoning.

Additionally, the benchmark's reliance on unpublished problems raises questions about data contamination—an issue prevalent in many existing benchmarks where models may inadvertently train on test data. By using entirely new problems crafted specifically for this evaluation, Frontier Math mitigates this risk. However, transparency regarding the development process and relationships with AI companies is crucial to maintaining trust within the research community. A controversy arose when it was revealed only after the release of o3 that OpenAI had funded this effort. While by itself this doesn't necessarily affect the relevance of the benchmark or invalidates the results we have seen, the funding of any such critical effort should be known from the get-go.

Conclusion

Frontier Math stands as a critical milestone in our efforts to evaluate advanced AI capabilities in mathematics. By presenting genuine challenges that require deep reasoning and creativity, it offers a unique lens through which we can measure progress in artificial intelligence. The remarkable performance of OpenAI's o3 model underscores both the potential and limitations inherent in current AI systems as they strive toward expert-level reasoning.

As we look ahead, ongoing collaboration between mathematicians and AI researchers will be vital in shaping future benchmarks that capture the full spectrum of mathematical inquiry while pushing the boundaries of what artificial intelligence can achieve.

In our next post, we will explore an often-overlooked aspect of AI development: the data workers powering the training of advanced models and the challenges they face.

References

[1] Epoch AI, 2024. FrontierMath. https://epoch.ai/frontiermath
[2] TechCrunch, 2024. OpenAI announces new o3 models. https://techcrunch.com/2024/12/20/openai-announces-new-o3-model/
[3] Epoch AI, 2024. FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI. https://arxiv.org/abs/2411.04872
[4] MATH Benchmark (Math Word Problem Solving). https://paperswithcode.com/sota/math-word-problem-solving-on-math
[5] GSM8K Benchmark (Arithmetic Reasoning). https://paperswithcode.com/sota/arithmetic-reasoning-on-gsm8k
[6] The Decoder, 2025. OpenAI quietly funded independent math benchmark before setting record with o3. https://the-decoder.com/openai-quietly-funded-independent-math-benchmark-before-setting-record-with-o3/
[7] Epoch AI. AI Benchmarking Hub. https://epoch.ai/data/ai-benchmarking-dashboard

India is rapidly emerging as a significant player in the global artificial intelligence (AI) landscape. With a robust IT industry valued at $250 billion, a workforce of nearly 5 million programmers, and a thriving startup ecosystem, the country has positioned itself as a hub of technological innovation. The government's proactive stance, marked by initiatives like the $1.2 billion IndiaAI Mission, underscores its ambition to achieve "sovereign AI." However, this journey is not without challenges. India's reliance on imported hardware, talent shortages, and uneven access to digital infrastructure present significant barriers. At the same time, AI brings risks of job displacement, algorithmic bias, and misuse, raising critical questions about governance and equity.

Building the AI Ecosystem: Strengths and Opportunities

India's growing prominence in the AI space is built on the foundation of its talent, technological infrastructure, and strategic policymaking. The country's tech workforce, already one of the largest globally, is bolstered by a steady stream of STEM graduates from premier institutions like the Indian Institutes of Technology (IITs) and the Indian Institute of Science (IISc). AI-focused initiatives such as AI4Bharat and FutureSkills PRIME aim to further develop domestic expertise, particularly in natural language processing, robotics, and AI ethics. These programs are designed to equip the workforce with the skills needed to develop, deploy, and manage AI systems, ensuring that India has the talent to drive AI innovation.

India's Digital Public Infrastructure (DPI), comprising platforms like Aadhaar, Unified Payments Interface (UPI), and DigiLocker, is a cornerstone of its AI ambitions. These systems not only enhance public service delivery but also generate diverse datasets invaluable for training AI models. The Aadhaar system, for instance, provides a unique identity to over a billion residents, while UPI enables real-time mobile payments, generating vast amounts of data that can be used to train AI models for various applications. This diversity is a unique advantage, particularly for natural language processing, as it allows for the development of AI models that can understand and respond to multiple languages and dialects. Initiatives like Bhashini, designed to support regional languages, exemplify India's ability to leverage AI for inclusivity. Bhashini aims to break down language barriers and make digital information and services accessible to all citizens, regardless of their preferred language.

Public-private partnerships have further accelerated AI innovation. Collaborations with tech giants like NVIDIA, Microsoft, and Google have expanded India's AI computing capacity. These partnerships provide access to advanced technologies and expertise, enabling Indian researchers and developers to work on cutting-edge AI projects. Indian startups, too, are making significant strides, with companies developing large language models and AI-driven solutions for healthcare, agriculture, and education. The AI services market in India is projected to reach $17 billion by 2027, driven by a growing ecosystem of innovators.

On the global stage, India has demonstrated leadership in shaping AI governance. Its active role in the Global Partnership on Artificial Intelligence (GPAI) and contributions to multilateral frameworks like the Bletchley Declaration reflect its commitment to equitable AI development. By advocating for resource-sharing and diversity in AI, India is positioning itself as a bridge between the Global South and established AI powers. India's focus on responsible AI development and its emphasis on inclusivity and fairness are gaining recognition on the global stage, positioning the country as a leader in shaping the future of AI governance.

Challenges: Addressing Structural and Strategic Gaps

Despite its strengths, India faces substantial hurdles in its AI journey. A critical challenge is its dependence on imported semiconductors and inadequate high-performance computing (HPC) infrastructure. While the National Supercomputing Mission and partnerships with companies like NVIDIA aim to address these gaps, building a self-reliant hardware ecosystem requires sustained investment and innovation. Current efforts are constrained by high costs, limited manufacturing capabilities, and a lack of advanced R&D infrastructure. To overcome this dependence, India needs to invest in building domestic semiconductor manufacturing capabilities and developing advanced computing infrastructure to support AI research and development.

The talent gap is another pressing issue. According to NASSCOM, India needs an additional 213,000 AI professionals by 2026 to meet projected demand. While initiatives like the IndiaAI Mission are addressing this shortfall, retaining talent within the country remains challenging. Many top professionals migrate abroad for better research funding and career prospects. To address this, India needs to create a more conducive environment for AI research and development, including increased funding, improved infrastructure, and attractive career opportunities.

The urban-rural digital divide exacerbates these challenges. While cities like Bengaluru and Hyderabad thrive as innovation hubs, vast rural areas lack access to digital tools and opportunities. Projects like Bhashini aim to democratize AI, but scaling these efforts requires systemic investment in connectivity, education, and affordability. Bridging the digital divide is crucial for ensuring that the benefits of AI are accessible to all citizens, regardless of their location or socioeconomic status.

Governance, too, poses a significant challenge. The absence of a comprehensive AI policy and sector-specific regulations creates uncertainty for stakeholders. While the Digital Personal Data Protection Act (2023) has introduced safeguards, issues like algorithmic bias, data sovereignty, and accountability remain underexplored. Public trust in AI systems will depend on transparent and ethical governance frameworks. India needs to develop a comprehensive AI policy framework that addresses these challenges and provides clear guidelines for the development and deployment of AI systems.

Risks: The Double-Edged Nature of AI in India

AI's transformative potential also brings significant risks, particularly for a diverse and populous country like India. Automation threatens to displace millions of jobs, especially in sectors like manufacturing and services that employ a large portion of the workforce. Without robust reskilling programs, these disruptions could deepen socioeconomic inequalities. To mitigate this risk, India needs to invest in reskilling and upskilling programs to prepare its workforce for the changing job market and ensure that the benefits of AI are shared equitably.

Algorithmic bias and inequitable access to AI tools further risk marginalizing vulnerable populations. For instance, biased algorithms in welfare distribution or financial services could entrench systemic inequalities. Similarly, the use of AI for surveillance and misinformation poses ethical and democratic concerns. To address these risks, India needs to ensure that AI systems are developed and deployed in a fair and unbiased manner, with safeguards in place to protect against misuse.

India's reliance on digital systems also makes it vulnerable to cybersecurity threats. Ensuring data protection and preventing misuse of AI technologies are critical priorities. Additionally, the environmental impact of AI, particularly energy-intensive training processes, adds to the complexity of managing its growth sustainably. India needs to develop strategies for mitigating the environmental impact of AI and ensuring that its development is aligned with sustainable development goals.

A Comprehensive Strategy for India's AI Future

To harness AI's potential while addressing its challenges, India needs a multi-pronged strategy:

• Investing in Infrastructure: Developing domestic capabilities in semiconductors and computing infrastructure is crucial for achieving technological self-reliance. This includes investing in research and development, promoting innovation, and creating a supportive environment for semiconductor manufacturing and AI hardware development.

• Fostering Talent: Expanding AI education and skilling programs, while retaining top professionals, will ensure a robust talent pipeline. This includes strengthening AI education at all levels, from schools to universities, and providing incentives for AI professionals to stay and work in India.

• Strengthening Governance: Comprehensive policies addressing fairness, accountability, and data privacy are essential for fostering trust and mitigating risks. This includes developing a national AI policy framework that provides clear guidelines for the development and deployment of AI systems, as well as sector-specific regulations to address unique challenges in different domains.

• Promoting Inclusivity: Bridging the digital divide through targeted investments in rural connectivity and language technologies can democratize AI's benefits. This includes expanding internet access in rural areas, promoting digital literacy, and developing AI solutions that cater to the needs of diverse communities.

• Advancing International Leadership: By advocating for equitable resource-sharing and ethical AI norms, India can shape global standards and promote collaboration. This includes actively participating in international forums and initiatives, such as GPAI, and contributing to the development of global AI governance frameworks.

The Way Ahead

India's role in the global AI ecosystem is both promising and complex. Its strengths in talent, innovation, and data position it as a key player, while its vulnerabilities underscore the need for strategic planning and inclusive policymaking. As India balances its ambitions with ethical considerations and infrastructure challenges, it has the potential to not only harness AI for national progress but also set an example for equitable and sustainable AI development on the world stage. By addressing its challenges with foresight and leveraging its unique capabilities, India can define the future of AI in ways that reflect the diverse needs of humanity.

Come back to us next week for more! If you were sent this blog by someone else, please consider subscribing to keep in touch!

In a world increasingly shaped by artificial intelligence, the idea of a frontier has never been more relevant. The Next Frontier is where we explore not just the cutting edge of technology, but also the edges that are often overlooked—the corner cases of risks, the voices at the periphery of the tech world, and the precedents from history and other fields that hold crucial lessons for our future.

Why The Next Frontier?

Frontiers are spaces of discovery, challenge, and growth. In the context of AI, they are also spaces of tension—where innovation collides with uncertainty, where progress tests our values, and where inclusion must overcome inequity.

At The Next Frontier, we aim to illuminate these edges:

• The technological frontier, where advancements in AI are redefining what’s possible but also raising new ethical dilemmas.

• The risk frontier, where underexplored scenarios and unintended consequences pose challenges to our safety and resilience.

• The geographic and human frontiers, where countries, communities, and individuals often excluded from global tech narratives are shaping their own paths.

• The intellectual frontier, where forgotten or ignored precedents from history, philosophy, and other disciplines offer essential insights into managing AI’s impact.

What You’ll Find Here

At The Next Frontier, we don’t just ask what’s next—we ask what matters as we stand at the edge of transformation. This is a space where questions take precedence over answers, and exploration thrives on nuance. Here’s what you can expect:

1. Frontier Dispatches:
   Regular commentary on the latest AI advancements, focusing on the edges of innovation—those developments that surprise, challenge, or redefine our understanding of the possible.

2. The Neglected Areas of Progress:
   Exploring opportunities and success stories from the periphery of the AI world—regions, communities, and individuals which go unnoticed in mainstream tech narratives.

3. Risk Mapping:
   Deep dives into the less obvious dangers of AI, such as corner cases, edge risks, and systemic vulnerabilities that might reshape our societies or ecosystems.

4. Lessons from the Past:
   Analysis of historical and interdisciplinary analogies—from nuclear policy to environmental science—that offer surprising parallels and insights for governing AI.

5. Conversations Across Borders:
   Spotlights on global perspectives, especially from the Global South, exploring how different nations navigate the promises and perils of artificial intelligence.

6. Reflections and Provocations:
   Essays that pause, provoke, and ask fundamental questions about what AI means for humanity, ethics, and the future.

The Frontier Mindset

The compass at the heart of The Next Frontier symbolizes exploration, guidance, and perspective—key themes of this blog. It represents our mission to navigate the cutting-edge of artificial intelligence while charting the less-traveled paths: the risks at the margins, the voices at the periphery, and the lessons from history and other fields.

The frontier is not a single place or moment. It’s a mindset—a commitment to pushing boundaries, to asking the difficult questions, and to navigating uncertainty with purpose and integrity. It’s about recognizing that what happens on the edges—of technology, society, and history—often shapes the center.

Join the Journey

The Next Frontier is for anyone curious about the paths less traveled. Whether you’re a policymaker grappling with AI’s implications, a technologist seeking broader perspectives, or simply someone passionate about understanding our rapidly changing world, there’s a place for you here. Subscribe to our newsletter now - articles come out weekly.

Let’s explore together—charting the edges of AI and beyond.