Following the launch of the AI browser by Perplexity, experts have started examining its security aspects. Checks, including those from Brave, revealed that such browsers are susceptible to harmful queries from fraudsters, posing a threat to user data. OpenAI has now confirmed this.
The company, which recently released the ChatGPT Atlas browser, published a new blog detailing the identified vulnerability and the measures being taken to address it. OpenAI emphasizes that the implementation of malicious queries remains a persistent security issue, necessitating regular enhancements to product protection.
Prompt injection attacks are a particular concern, as they enable malicious actors to embed harmful instructions within content processed by AI. These instructions can be hidden in websites, emails, or PDF files. The goal of such attacks is to manipulate the model's behavior and execute commands from the attacker instead of the user's requests.
These attacks are especially dangerous because they often require no human involvement. Users might be unaware that their personal information is being sent to fraudsters or that the AI is performing unwanted actions.
To combat these threats, OpenAI has developed an "automated LLM-based adversary," which simulates hacker behavior and tests for vulnerabilities. Initially, this AI tests attacks in a controlled environment to observe how browser agents respond.
OpenAI also demonstrated a prompt injection example where an attacker sent an email containing a hidden instruction for the AI agent. When a user requested the AI to draft an absence notice, it could have used this instruction to send a resignation letter. However, thanks to training, the system recognized the instruction as a harmful prompt injection and did not execute it without explicit user confirmation.
In the company’s blog, it states: "The nature of prompt injections makes deterministic security guarantees challenging, but through scaling our automated security research, competitive testing, and strengthening our rapid response cycle, we can improve the model's resilience and protection before a real attack occurs." Despite the implementation of new security tools, prompt injection remains a significant threat to AI-based browsers, leading some industry experts to question the safety of using such agent-based browsers.
