Security Challenges of AI Browsers: Vulnerabilities and Risks
Following the launch of the AI browser Comet by Perplexity, specialists began examining its security. Checks conducted by Brave indicated that such browsers are susceptible to malicious queries from fraudsters, jeopardizing user privacy. This has now been confirmed by OpenAI.
The company, which recently released the ChatGPT Atlas browser, published a new blog post detailing the identified vulnerability and measures to address it. OpenAI emphasizes that malicious queries are a persistent security concern for artificial intelligence, necessitating regular updates to product protections.
Malicious query attacks, or prompt injection, represent a specific type of attack on AI agents in browsers, where harmful instructions are intentionally embedded in content. These can lurk on websites, in emails, PDF files, or other materials processed by AI. The goal of such attacks is to manipulate the model into changing its behavior and executing the attacker’s commands instead of the user’s requests.
These attacks are particularly dangerous as they often do not require human involvement. Users may be completely unaware that the AI agent is transmitting their personal data to fraudsters or executing other actions programmed by malicious actors, such as sending harmful emails.
To combat these attacks, OpenAI has developed an "automated malicious actor based on LLM"—essentially an AI bot that simulates hacker actions and attempts to execute prompt injection. Initially, this AI tests attacks in a separate simulator to see how browser agents respond. By analyzing the results, the system continuously modifies and improves its attacks to learn how to better detect them in real-world conditions. The gathered data is later integrated into protective mechanisms.
OpenAI also showcased a demonstration of prompt injection that its AI detected and used to enhance the protection of ChatGPT Atlas. In one scenario, an attacker sent an email with a hidden instruction for the AI agent—a template for a resignation letter directed to the CEO. Later, when the user asked the agent to draft an absence message for the CEO, the agent could have utilized this instruction and sent out a resignation letter. However, thanks to training, the system recognized that the instruction was a harmful prompt injection and did not execute it without explicit user confirmation.
"The nature of prompt injection makes deterministic security guarantees challenging, but through scaling our automated security research, competitive testing, and strengthening our rapid response cycle, we can enhance the resilience and protection of the model before a real attack occurs," the company stated in its blog.
Despite implementing new tools and security measures, prompt injection remains a serious threat to AI-based browsers. This leads some industry experts to question the advisability of using such agent-based browsers, given the risks to personal data.
