Before you prompt AI to answer another question or perform another task, new research suggests there may be another consideration: whether the tools used by an AI agent could expose sensitive data.
Ying Zhang, an assistant professor in Wake Forest University’s Department of Computer Science, studies security in software engineering. Her latest research, “How Your Credentials Are Leaked by LLM Agent Skills,” examines how large language model (LLM) agent skills can make data vulnerable to attacks.
- LLM agents are autonomous AI systems that analyze circumstances and plan and execute multi-step actions to achieve a goal.
- A skill is a reusable software extension that gives an LLM agent additional capabilities. By installing skills into LLM agents such as Claude Code, Codex or Cursor, developers can enable agents to perform specialized tasks, such as accessing a database or analyzing domain-specific information.
- Credentials are forms of authentication that allow systems to communicate securely and access protected resources. If exposed, they can potentially provide unauthorized access to sensitive data.
The potential exposure comes through third-party AI agent skills — sets of instructions that enable agents to perform specialized tasks. Those tasks can range from creating a presentation using a user’s notes to scanning financial documents for compliance issues.
Credential leakage can occur either unintentionally or maliciously, but in either case it can result in unauthorized access to private data, Zhang said.
“When the agent skill is flawed or developed with malicious intent, it will steal your data and maybe pass it back to a remote server to be used in some malicious way,” she said.
Zhang, a corresponding author on the study, is scheduled to present the research at the International Conference on Automated Software Engineering, Oct. 12-16 in Munich.
How do data leaks happen with AI agents?
According to the research, credential leaks can occur in two primary ways:
- Malicious: A developer creates a skill containing instructions designed to obtain credentials and access private data.
- Unintentional: A developer uses insecure coding practices that inadvertently expose credentials or create vulnerabilities that attackers can exploit.
“A lot of skills have credential leakage problems, and there are also malicious skills being developed and distributed,” Zhang said. “Through our work, we are helping detect these skills and remove them from the open-source market.”
How pervasive is the problem?
The research team analyzed 17,022 randomly selected skills and generated 170,226 outputs. The skills came from SkillsMP, an open-source AI agent skill marketplace that provides access to more than 1.6 million skills.
The researchers identified:
- 520 affected skills.
- 1,708 security issues among those skills.
- 10 credential leakage patterns.
The study also found that 89.6% of the leaked credentials were immediately exploitable.
After the researchers reported their findings to SkillsMP, the malicious skills identified by the researchers were removed, and most of the vulnerabilities attributed to insecure coding were fixed.
Why does it matter?
Zhang said the findings highlight two broader issues involving software development and AI security.
First, some developers lack the security expertise needed to integrate adequate protections into their software. Second, pressure to release products can result in security receiving less attention during the development process. Zhang argues that security should instead be incorporated from the beginning.
“When I train my students, I teach them that security is a critical component in their software design,” she said. “Every feature they develop, they have to keep security in mind.”
The rapid development of AI and the growth of AI-assisted software development, sometimes called “vibe coding,” introduce additional security concerns, Zhang said. People with limited software development experience may lack an understanding of software security, while AI-generated code does not necessarily address those vulnerabilities.
What can reduce the risk?
Zhang argues that security should be incorporated into the initial software design process rather than addressed primarily after vulnerabilities or breaches are discovered.
The research also identifies several areas for further work, including developing standards for protecting data used by AI systems and tools capable of analyzing AI agent skills for security vulnerabilities. Additional safeguards, whether through technical standards, contractual requirements or regulation, could also provide users with greater information about the security of the software they use.