An AI 'Security Fix' Introduced the Bug. GitHub's Own AI Review Called It Clean.

NativeFirst Team 7 min read
An open padlock hanging on a chain-link fence

Here’s a sentence that should not be possible: an AI tool designed to fix security bugs introduced one, another AI tool designed to catch security bugs approved it, and a third AI tool found it in the wild five days later and let itself into a company’s internal Jira.

That’s not a hypothetical. That’s the actual timeline of what happened to Snowflake this summer, and it’s public now — Wiz Research published the full writeup on August 17. I read it twice because I kept assuming I’d misunderstood a step. I hadn’t.

If you run any part of your iOS pipeline through GitHub Actions — and at this point, who doesn’t — this one is worth the ten minutes.


The PR that broke its own gate

On June 18, 2026, someone merged PR #1218 into snowflakedb/snowflake-connector-net, a public Snowflake repository. It touched a GitHub Actions workflow that files Jira tickets when someone opens a GitHub issue. The squash commit lists “Copilot Autofix powered by AI” as a co-author.

Before the change, the workflow took the issue title, passed it through an env: variable, and built the JSON payload with jq. That’s the safe pattern — GitHub’s own docs recommend it specifically because it stops untrusted text from ever touching the shell as code.

The autofix commit removed that and replaced it with this:

env:
  ISSUE_TITLE: ${{ github.event.issue.title }}
run: |
  TITLE=$(echo '${{ github.event.issue.title }}' | sed 's/"/\\"/g' | sed "s/'/\\\'/g")

See it? The issue title gets interpolated directly into the shell script before the sed escaping even runs. GitHub expands ${{ github.event.issue.title }} as raw text substitution, at parse time, before bash ever sees a quote to escape. Put a single quote in your issue title and you’ve broken out of the echo '...' and you’re writing your own shell command.

This is the textbook GitHub Actions script-injection pattern. It’s been documented for years. And an AI “autofix” tool put it back in, replacing code that had specifically been written to avoid it.

The kicker: there was an if: condition on the workflow that looked like a security gate — checking that the actor wasn’t a known bot. Except it checked github.event.pull_request.user.login, and on an issues trigger, github.event.pull_request doesn’t exist. It’s always null. So the condition always evaluated true. The gate was a prop. It had never once done anything.

GitHub’s own AI-assisted code review looked at the merged PR — the injection and the always-open gate together — and marked it clean.


The other AI found it in five days

Wiz runs something they call “Red Agent,” an autonomous AI security researcher that scans for exactly this class of bug across GitHub organizations it’s authorized to test. It flagged the workflow, tried to exploit it, and hit a bash syntax error on its first attempt — the # comment character it used to terminate the payload also ate the closing parenthesis of the $(...) subshell, which broke the script instead of hijacking it.

It didn’t stop and ask a human. It read the error, figured out the closing paren needed to be satisfied first, swapped to ; echo ' to close the block cleanly, and tried again. That attempt worked. Within seconds it had a callback from a GitHub Actions runner carrying a base64-encoded Jira API token, associated with qa@snowflake.net, with read access across Snowflake’s engineering, security compliance, and bug bounty tracking Jira projects.

Snowflake patched it the same day it was reported, rotated the credential, and confirmed via audit logs that nobody but Wiz ever touched it. Good outcome, real disclosure process, no actual harm. That’s not the part that should keep you up.

The part that should keep you up is the five-day gap between “AI-approved as safe” and “different AI walks in through the front door.”


Why this isn’t just a Snowflake problem

I went and checked my own GitHub Actions setup for this blog — the workflow that guarantees a post ships every day even if I’m asleep. It doesn’t touch github.event.issue.title or anything like it; it only runs on a schedule and manual dispatch, so there’s no untrusted input to inject into in the first place. Lucky, not virtuous — I just never had a reason to wire up issue-triggered automation.

But plenty of iOS teams do. Auto-labeling bots on new issues. Triage comments that echo back the PR title. A Slack notification step that includes github.event.pull_request.title in a run: block because it was the fastest way to get the string into the message. Fastlane pipelines wired to GitHub Actions for TestFlight uploads, often copy-pasted from a blog post or, increasingly, generated by an AI agent that has no memory of why the safe pattern existed in the first place — which is exactly what happened here. The autofix didn’t choose the vulnerable pattern out of malice. It didn’t have the context that the safe one was there on purpose.

That’s the actual lesson buried under the scary headline: AI code tools are pattern matchers without institutional memory. They don’t know your team removed direct string interpolation from that workflow eighteen months ago after an incident. They see input, they see a string, they produce plausible code that handles a string. Plausible and safe are different bars, and nothing in the generation step checks which one it cleared.

And the review step isn’t a backstop if it’s also AI, trained on the same corpus, prone to the same blind spot. Two models that share a training distribution can share a failure mode. That’s not a review — that’s the same guess, twice.


The one-line fix, if you want to check your own workflows right now

Grep your .github/workflows/*.yml for github.event inside a run: block. If you find ${{ github.event.issue.title }}, ${{ github.event.pull_request.title }}, ${{ github.event.comment.body }}, or anything else event-supplied sitting directly in a shell command, that’s the pattern. Move it into an env: block instead and reference it as a shell variable — $ISSUE_TITLE, not ${{ github.event.issue.title }} inline. The shell treats an environment variable as data. It treats the raw template expansion as code you just handed it, no questions asked.

It takes five minutes. It’s also exactly the fix Snowflake shipped the same day they found out, because it was already the pattern the AI tool had removed.


This isn’t a reason to stop using AI code review or AI autofix tools — Copilot catches plenty of real bugs, and I’ve said as much about Xcode’s own AI agent. It’s a reason to stop treating a green AI checkmark as the end of the review, the same way a code slop crisis isn’t fixed by adding another layer of unverified generation on top. The agentjacking research I covered a few weeks back showed AI agents can be hijacked through tool calls; this is the mirror case — an AI agent hijacked itself through its own blind spot, no attacker required. And it lines up with why OpenJDK banned AI-generated code outright rather than trust a review step to catch what the generation step missed.

If your CI pipeline touches untrusted input anywhere — an issue title, a PR description, a comment body — go check it today. Not because your AI reviewer is bad. Because it’s confident, and confident isn’t the same thing as right. That distinction is basically the entire thesis of our course lesson on reviewing AI-generated code before it ships, and this is as clean a real-world case study as you’ll find.

Share this post

Share on X LinkedIn

Comments

Leave a comment

0/1000

N

NativeFirst Team

Editorial

The NativeFirst team — engineers and designers building native Apple apps and writing the courses we wish we had when we started.