Models · · 2 min read

OpenAI Is Hiring a Threat Modeler to "Own" Its Catastrophic Risk Framework

A new job listing from OpenAI's Preparedness team signals the company is formalizing its modeling approach to frontier AI risk.

OpenAI Is Hiring a Threat Modeler to "Own" Its Catastrophic Risk Framework
Photo by Andrew Neel / Unsplash

OpenAI is recruiting a Threat Modeler for its Preparedness team, a role that would give one person primary ownership of how the company identifies, models, and forecasts catastrophic risks from frontier AI systems.

The job posting — for a San Francisco-based position compensated at $325,000 plus equity — comes as the company works to formalize its internal AI safety operations ahead of increasingly capable model generations.

According to the job listing, the threat modeler will "own OpenAI's holistic approach to identifying, modeling, and forecasting frontier risks" and serve as "a central node connecting technical, governance, and policy perspectives on prioritization, focus and rationale on our approach to frontier risks from AI."

The role is described as the internal explainer of "the 'why' behind our most stringent risk-prevention efforts."

The threat modeler will be expected to develop models across misuse areas including biological, cyber, and attack planning threats, as well as risks from "loss of control, self-improvement, and other possible alignment risks from frontier AI systems." They will also work closely with the company's bio and cyber leads to "size the remaining risk of the designed safeguards and translate threat models into actionable mitigation designs."

Risk Categories That Concern OpenAI Most

OpenAI has previously outlined the risk categories its Preparedness team monitors: individualized persuasion, cybersecurity, chemical/biological/radiological/nuclear (CBRN) threats, and autonomous replication and adaptation (ARA).

The threat modeler will be responsible for ensuring the company's evaluation frameworks are "robust, high-coverage, and forward-looking" across all of them.

The Preparedness team was established in late 2023 as OpenAI's team dedicated to tracking and mitigating what it describes as catastrophic misuse risks. Led by Aleksander Madry, OpenAI said the team was tasked with "tightly connecting capability assessment, evaluations, and internal red teaming for frontier models, from the models we develop in the near future to those with AGI-level capabilities." The new threat modeler role sits at the center of that mandate.

The AI company has been circling around catastrophic risk issues for come time. At launch, OpenAI ran a public Preparedness Challenge, offering $25,000 in API credits to the ten best submissions identifying novel catastrophic misuse scenarios.

The results revealed roughly 70% of entrants emphasized the potential for OpenAI's models to enhance malicious actors' persuasive capabilities, with submissions describing threat models including online radicalization, political influence operations, and large-scale polarization. OpenAI said at the time it was "currently conducting studies on AI's impact on persuasiveness."

Ideal candidates for Threat Modeler are expected to "bring deep experience in threat modeling, risk analysis, or adversarial thinking" from backgrounds in security or national security, while also understanding AI alignment literature and how capability evaluations connect to safeguard sufficiency.

The listing explicitly calls for someone who can "think in systems and naturally anticipate second-order and cascading risks."