Artificial intelligence systems are now deployed at scale across sectors, accompanied by a growing number of real-world incidents ranging from misinformation and cybercrime to autonomous-system failures. Databases of AI incidents index these events, but they cannot measure ``risk'' (i.e., a joint measure of likelihood and severity) without additional data regarding the prevalence of risk-associated systems and their incident reporting rates. As a result, policymakers, companies, and the general public lack a means to weigh the benefits of AI against their in-context risks. Inspired by public-health processes, which presume noisy and incomplete disease surveillance, we identify six phases of incident emergence. We demonstrate the framework through a detailed case study of autonomous vehicles, whose mandatory reporting requirements produces reliable incident-rate ground truth expressed in distance traveled. The case study shows that an informed panel of domain experts (e.g., self-driving experts) can combine their domain expertise, incident data, and a collection of statistical and visualization tools to arrive at incident phase determinations serving public needs. We further demonstrate the approach with a deepfake incident case study and chart a path for future research in incident phase determination.

Expert Partner: Sean McGregor (AI Incident Database)

Supporting experts: Peter Slattery and Simon Mylius (MIT AI Risk Initiative)

Alumni

Meet the authors

(Research Team Lead)

Simon Mylius

Giovanna Jaramillo-Gutierrez

Giovanna Jaramillo-Gutierrez’s work sits at the intersection of AI policy, data science, and algorithmic auditing, guided by a long-standing commitment to public-interest innovation in line with the UN Sustainable Development Goals. She is an EU AI act ForHumanity auditor and a certified AI governance professional (AIGP) by the IAPP.
Previously, Giovanna spent more than a decade with the health emergencies department at the World Health Organization (WHO), leading outbreak analytics and surveillance across emergencies including Ebola, plague outbreaks, the 2009 H1N1 pandemic, and COVID-19
Giovanna holds a PhD in Molecular Biology and an MSc in Epidemiology.

Sophia Abraham

Sophia Abraham is a Computer Vision PhD graduate from the University of Notre Dame whose research explores interpretability, fairness, and robustness in deep learning.
Building on experience at Google X and TidalX AI, she recently joined the AI Governance Taskforce to bridge her technical expertise with the policy and ethical dimensions of AI.
Sophia is passionate about aligning cutting-edge AI research with societal values and long-term governance goals.

Taiye (Chan) Chen

Taiye is an economist with a strong foundation in international economics and policy research, having worked with multiple multilateral organizations.
She applies economic modelling skills to analyse various development challenges, now expanding her expertise into AI governance and safety.
Passionate about leveraging technology for inclusive growth, she is actively pursuing opportunities at the intersection of AI, policy, and sustainable development.

Cyril Chuun

Cyril is an AI researcher who predominantly focuses on the evaluation of foundation models for risk modelling and the development of responsible AI governance frameworks.
Cyril holds a PhD in Computer Science - he developed a meta-evaluation methodology and benchmark for automatic story generation.
His ambition is to use his cross-disciplinary skillset to help EU policymakers draft informed and responsible AI regulation laws, taking into account both technical limitations and legislative requirements.

Sayash Raaj

Sayash Raaj is a technical AI governance researcher with experience in AI incident forecasting, synthetic data for safety, and large-scale optimization.

A graduate of IIT Madras, he has led quantitative AI initiatives in a major financial institution and published research on constraint-aware prompt optimization at a NeurIPS workshop.

He is currently working on forecasting models for AI incident emergence and on governance mechanisms that ensure safe, stable deployment of increasingly capable AI systems.

Alumni