The OpenAI breach of Hugging Face’s digital library reinforces a growing concern among cybersecurity researchers: The fast-paced software development process isn’t thinking about security early — or comprehensively — enough.
Ying Zhang
“A vulnerability in code, configuration or infrastructure can already pose a risk, but AI agents can amplify their impact by autonomously exploring systems and attempting attacks at a scale beyond human capabilities,” said Ying Zhang, assistant professor of computer science at Wake Forest University. “As a result, vulnerabilities that were previously considered low-risk in traditional software can become much more dangerous in AI-powered environments.”
OpenAI announced Tuesday that its AI agent was responsible for a security breach reported by Hugging Face, a company that provides AI technology to software developers.
The incident occurred during an internal cybersecurity assessment conducted by OpenAI. The compromise was due to a combination of exploitable vulnerabilities in Hugging Face’s infrastructure and limitations in OpenAI’s agent containment controls, Zhang explained.
The AI agent, a software program that makes decisions and takes actions to achieve a predetermined goal, escaped the test environment and hacked into Hugging Face’s infrastructure.
Zhang, who leads the Software Engineering & Software Security Lab at Wake Forest, studies the reliability and security challenges of large language model (LLM) agents and AI-driven software systems. Her research also focuses on creating targeted security tests to help developers identify and fix problems before an AI agent can exploit them.
“We have found that both developer-written and AI-generated code often have security vulnerabilities, but security fixes are often deferred to later stages of development,” she said. “Our goal is to make security a proactive part of software development and not just a final step.”
She points out that securing AI systems requires looking beyond the model itself. Modern AI systems rely on complex stacks of software components and services, and vulnerabilities anywhere in the stack can introduce risks throughout the system.
Given the pace of AI growth, can security solutions even catch up?
“Security has always been an arms race, and AI is mainly changing its economic aspects: attacks are becoming cheaper and faster than patches.”
Ying Zhang, assistant professor of computer science
“So the answer isn’t about patching faster. It’s about building systems where entire classes of attacks are impossible by design,” Zhang said. “That’s what Secure-by-Design means and it’s my long-term career goal.”
Zhang said the incident underscores the urgency of building a secure AI infrastructure and validates her previous work: “How can ChatGPT help human security testers mitigate supply chain attacks?” The study, published in IEEE TSE 2025, showed that LLMs can be used not only to defend software, but also to identify and exploit vulnerabilities.
https://news.wfu.edu/2026/07/23/can-we-build-software-secure-enough-to-thwart-ai-attacks/
