Microsoft Copilot's vulnerability to prompt injection attacks has been exposed, revealing a hidden parameter that can be exploited to leak sensitive information. This security flaw highlights the importance of understanding the inner workings of AI assistants and the potential risks associated with URL parameters. The attack involves crafting a URL with a specific format, including an undocumented parameter, to bypass Copilot's security measures. By embedding a prompt within the URL, attackers can instruct Copilot to perform actions like searching an inbox for the latest sender's email address and extracting it into a variable. This information can then be used to construct a malicious URL that, when clicked, leaks sensitive data to an attacker-controlled server. The attack process involves several steps: the victim clicks the crafted URL, Copilot processes the injected prompt, and the sensitive information is automatically sent to the attacker's server. The researchers also discovered a separate prompt that could be used to search for and leak passwords or other credentials. This attack demonstrates the potential for AI assistants to inadvertently expose sensitive data if not properly secured. Additionally, Varonis uncovered a more sophisticated attack where a prompt injection in a webpage can poison Copilot's permanent memory store. This memory store saves user information, preferences, and instructions, and when a user instructs Copilot to summarize the page, the assistant updates this memory based on instructions hidden in the page metadata. This attack has far-reaching implications, as it can be used to manipulate outputs, filter information, and bias responses towards attacker-chosen narratives. The vulnerability in Copilot's security measures and the ease of exploiting prompt injection attacks underscore the need for robust security practices and ongoing vigilance in protecting sensitive data and user privacy.
