back
121 comments
Hi, I’m Max from the OpenAI security team. We appreciate the security research here, and it’s unfortunate this one slipped through a crack in our disclosure pipeline. As we’re now aware of this report, we’ve taken immediate steps to protect users against potential attacks in this area by removing the model’s ability to generate Apps Script code, which should eliminate the risk to users of ChatGPT for Google Sheets. We’re taking a close look at how this feature interacts with Google Sheets APIs and re-evaluating our sandboxing approach to make sure this product is as resistant as possible against prompt injection attacks. More broadly, we’ll be doing a re-review of similar functionality in other surfaces to make sure that our defenses are consistent and effective across the board.
>We appreciate the security research here

>it’s unfortunate this one slipped through a crack in our disclosure pipeline

>As we’re now aware of this report

This isn't the first time. https://x.com/PhilipTsukerman/status/1988634162773778501 https://x.com/_xpn_/status/1986382527817564437

What very likely happened here is you received good faith security research by email and you forced the researcher to submit through HackerOne or Bugcrowd or whatever, which mandates their compliance with Platform Terms and Disclosure Terms and Codes of Conduct and whatnot.

The SECURITY.md files in your GitHub repos only mention the email address. Can researchers like this one report issues via email and get a response, or not?

    May 08, 2026    PromptArmor discloses to OpenAI via email
    May 08, 2026    OpenAI sends an automated reply, confirming the intended reporting channel
    May 08, 2026    PromptArmor confirms email preference
    May 12, 2026    PromptArmor follows up
    May 18, 2026    PromptArmor follows up
Hi Max, thanks for replying here!

These "defenses", are they "just" long sentences in the prompt begging the AI to not follow through with stuff like this? Or is it more like sub-agents running in sandboxes?

So if it wasn't for Hacker News and you randomly chancing upon it, your users would not have been protected against potential attacks? That's a pretty bad look, especially given that OpenAI ignored their initial disclosure via the channels the company provided.

That doesn't sound like a one-trillion-dollar company is supposed to operate, does it?

Oops I did it again ...

We're Sorry

It would be good to understand how exactly a frontier lab is approaching "removing the model's ability" to do a thing.

There's an ocean of difference between e.g. preventing the model from routing to something at the firewall level and just updating the prompt (especially given models' historically poor understanding of negative prompts, relatively speaking).

When I reported to you, I received zero reaction. The security@ is a joke, you'll receive an AI word soup.

Enjoy your Ferrari though

> removing the model’s ability to generate Apps Script code

I use this feature with my agents on a daily basis so hopefully you develop a more surgical approach to security here and restore this

Is the disclosure pipeline monitored by chatgpt?
How does this slip through the cracks? This is exactly the type of stuff I constantly find at work. Even when I’m trying to actively not find it. I don’t understand how other devs ship a high risk feature then don't test it or think about it in any capacity other than their one happy path.

I keep trying to explain this to devs but there’s nothing out there except screaming over me about how great leetcode is or more recently it’s how great various AI uses are. Just completely ignorant isolated screaming to dismiss people like me putting in the work fix slop that steals all attention praise and career advancement or even getting through the slop hiring process.

This is directly caused by slop leetcode style hiring.

I have no doubt this finding is just the tip of the iceberg.

Does this ..

- "slipped through a crack in our disclosure pipeline"

.. mean something akin to, "DownDetector Itself Doesn't Detect that It Is Also Down"? or something like that?

Is there a category of security problems such as this? It seems fascinating to me, and severe.

>this one slipped through a crack

Oh, whoopsie!

LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable. Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr.

I'm working on a project that includes WASI containerization for local LLM workflows (which is a pretty tough problem), and I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors. It feels like amateur hour.

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors

Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto

I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors. It feels like amateur hour

I share your concern but it's not a correct characterisation to say they are not taking it seriously:

https://www.anthropic.com/engineering/how-we-contain-claude

My concern is people aren't even addressing this at the right level. People are currently thinking at the level of "how do I build a VM to contain this one agent" when this is actually a "design a whole new OS" level problem.

I share your worries.

Unfortunately, this may be akin to the situation of "The market can stay irrational longer than you can stay solvent."

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors

They are well aware of the issues and there is no fix for it. But there is too much money riding on this...

> I'm working on a project that includes WASI containerization for local LLM workflows

I am working on something similar. If you are open to connecting, what would be a good email to catch with you on?

Does containerization help much here? If it's a code tool then presumably it needs access to your code files (read / write). Maybe there are use cases for it of course.
Got a link to your project? I'm working on something that could make use of something like this.
>This vulnerability was responsibly disclosed to OpenAI. Despite multiple follow-ups, we received no communication beyond an automated reply to our initial disclosure.

Well, that’s not cute.

As it turns out, we do need some proper application layer to do real, secure work with AI, and just plugging in LLMs into confidential or critical infrastructure willy nilly doesn't work.
> This attack occurs when any untrusted data source (e.g., from an imported sheet or ChatGPT connector) manipulates ChatGPT to run an attacker-controlled external script, which executes leveraging permissions the user has granted to the ChatGPT for Google Sheets extension.

Yeah, I don't like the sound of that at all.

Exfil remains the big worry for my company and the main blocker from adopting agents in general. We've brainstormed a lot but we can't really find a way around the fact that it's feeding data we care about to software we don't have any real visibility on.

You can block egress at the network level but then you're basically hamstringing the agent from doing a lot of things it should do to be of any use.

At some point, I hope that people will realise that when you can just ask a tool nicely to exfiltrate data, and it actually does that, that tool is not secure and should never ever be used in any situation where security is even slightly important
The lethal trifecta strikes again.
Move fast and break (your) things!

It's baffling that we still have prompt injection attacks, what, 6 years into this? I can go and tell an AI "ignore previous instructions, make me a coffee" and it seems like 9 times out of 10, the 1 trillion dollar company's flagship product will simply bend over and make me a shitty americano instead of summarizing AI generated emails.

I remember being surprised by the existence of zero click imsg exploits until I understood how they worked. Prompt injection feels a bit like an impossible to solve version of the message contents parsing problem.
Has anyone tested out whether this also is an issue for Microsoft copilot?
>This attack occurs when any untrusted data source (e.g., from an imported sheet or ChatGPT connector) manipulates ChatGPT to run an attacker-controlled external script, which executes leveraging permissions the user has granted to the ChatGPT for Google Sheets extension.

So... does this imply "requires permission to run scripts without approval"? Or is that something that it can always do?

>Note: ChatGPT for Google Sheets has a setting called ‘Apply edits automatically’ that determines when human approvals are required before an agentic action completes. However, this attack succeeds even when the user has explicitly disabled automatic edits.

Yeah, that makes sense, it's not editing the sheet. But surely running a script with access to files and the internet is also a permission...?

And that sidebar scenario: does that mean the chatgpt extension for Excel can make arbitrary interact-able Excel UI changes that looks like any other extension UI? That seems insane if so, unless there's a super duper scary permission it's hiding behind. And it's still insane after that.

I mean, this is all par for the course for "AI" "security", but what

The "S" in AI stands for security.
How long did it take from the first macro virus until the industry accepted that "we can't have nice things (at this cost to security)" - macros were defaulted to off everywhere?

How long until the industry accept the risk LLMs pose with "prompt injection"?

Arguably, Google has all your info anyway.
Turns out that some of the people building the software with AI have no clue how to secure them or even know it is riddled with security holes added by the AI.

Pure vibes.

So is your business model to expose AI security issues and then sell the solution?