When Your Repo Starts Talking: Agents.md and Agent Goal Hijack in VS Code Chat

Introduction

In recent times, the emergence of AI agents has revolutionized how we interact with technology and automation. Understanding the dynamics and potential risks associated with these agents is essential.

The Role of AGENTS.md Files

AGENTS.md files are critical in managing and defining the roles of various AI agents within a system. These files can illustrate how agents obtain authority and control within their operational environments.

Case Studies

Case 1: Unauthorized Authority in AI Agents

Three attack cases show how AGENTS.md files, marketplace skills, and dependencies give AI agents unauthorized authority, raising concerns about what should be tested before trusting one.

Case 2: Marketplace Skills

Marketplace skills enable AI agents to access additional functionalities that could be abused if not properly managed.

Case 3: Dependency Management

The dependencies an agent requires must be carefully controlled to prevent exploitation and vulnerabilities.

Conclusion

Understanding the implications of AI agents and the importance of AGENTS.md files is paramount for developers and security teams. Ensuring rigorous testing and validation can help mitigate potential risks associated with these evolving technologies.