HAPPY FRIDAY,
Mid-August, which means half the office is on vacation and the other half is pretending they're not thinking about it. Good week to talk about AI making decisions before you even know it's been asked, and someone trying to cheat a legal proceeding via hidden instructions in a Word document. Usual slow news cycle stuff.
In This Week's AI Fridays:
- ChatGPT has already decided which brands to recommend before it runs a single search, and the gap between being in that list versus not is enormous
- Someone hid a prompt injection attack inside a legal filing to manipulate AI document review, which is either clever or alarming depending on how charitable you're feeling
- Shopify data puts real numbers on the AI traffic vs. Google debate, and the answer is: both, but differently
- LinkedIn now lets users flag content as AI slop, which is a sentence I did not expect to be typing in a professional newsletter
- Plus quick hits from the week
ChatGPT Already Knows Who It'll Recommend Before It Searches
New research shows that when ChatGPT formulates its internal search queries, it names specific brands before retrieving any external results. Being included in that pre-fetch query makes a brand 33 times more likely to end up in the final recommendation. This is not a small SEO tweak. It is a fairly significant structural shift in how brand visibility now works.
The Highlights:
- ChatGPT generates its own search queries internally before going out to retrieve information, and those queries often already contain brand names
- Being named in the pre-fetch query correlates with a 33x higher likelihood of appearing in the final recommendation
- This means brand presence in AI training data and model associations matters at a stage that happens before any traditional search retrieval kicks in
- GEO (generative engine optimisation) strategy needs to account for this pre-retrieval layer, not just the output
The Takeaway:
If your brand is not part of what the model already associates with a category, you are hoping to get picked up in the retrieval round, which is the consolation bracket. The interesting question now is what puts a brand into that pre-fetch layer, and no one has a fully clean answer yet.
Person Hides Prompt Injection in Legal Filing Telling AI to Side With Them
Someone embedded instructions directly into a legal document, written to be read by an AI rather than a human, telling the model to find in their favour if it reviewed the filing. It is the first documented case of prompt injection being used in a legal context, and the implications for any firm using AI-assisted document review are not comfortable reading. The attack only works if the AI processes the document without the right guardrails, which is a generous assumption about how most organisations have deployed these tools.
The Highlights:
- Hidden text in the legal filing contained instructions like "ignore previous instructions" and told the AI model to side with the filing party
- First known use of prompt injection in a legal document, though the technique itself has been around for years in other contexts
- The attack is only effective against AI systems reviewing documents without proper input sanitisation or injection detection
- Raises immediate questions for law firms, insurers, and any organisation running AI over third-party documents
The Takeaway:
The thing about prompt injection is that it has been a known vulnerability for years, and most deployments still treat it as someone else's problem. Putting it in a court filing is a reasonable escalation from someone who noticed that gap. Expect this to get much more common before it gets better.
AI in Marketing
Shopify: AI Referrals Up 197%, But Organic Search Still Leads Traffic
Shopify released platform data showing AI-driven referral traffic grew 197% year over year, with particularly strong conversion among first-time buyers. Google still accounts for the majority of overall traffic, so this is not a "search is dead" story, but the conversion quality gap is worth paying attention to. Someone arriving from an AI recommendation has already been through a layer of synthesis and filtering before they clicked, which may explain why they tend to buy more readily. For marketers still treating AI traffic as a rounding error, these numbers suggest it is worth its own line in the reporting. The channel is maturing faster than the measurement frameworks around it.
LinkedIn Is Policing AI Slop, So Time to Reset
LinkedIn has added a user-reportable flag for content that "seems like AI slop," which is a blunt label for a platform that has historically been tolerant of blunt content. The move puts social proof pressure on the problem in a way that algorithm tweaks alone have not managed, since now your connections can flag your posts and you will presumably know about it. For brands running high-volume AI content programs on LinkedIn, this is a signal worth taking seriously, less because of any immediate penalty and more because it tells you where platform norms are heading. The interesting design question is whether a crowdsourced slop flag will catch low-quality AI content or just penalise anyone who writes in a slightly formal register on a bad day.
ChatGPT Already Knows Who It'll Recommend Before It Searches
The pre-fetch finding from this week's hero story has direct marketing implications that are worth pulling out separately. If the model is naming your brand before it even retrieves external results, your GEO strategy needs to account for that pre-retrieval layer, which is shaped by training data, brand associations, and the kind of language that has historically described you at category level. Traditional SEO optimisation targets the retrieval stage. This research suggests there is an earlier decision happening that most brand strategies are not addressing. Worth running a few test prompts in your category to get a rough sense of whether you are showing up in the pre-fetch or waiting to be retrieved.
Person Hides Prompt Injection in Legal Filing Telling AI to Side With Them
The marketing angle here is less obvious but worth thinking through. Any organisation using AI to review contracts, proposals, briefs, or vendor documents is exposed to the same class of attack. A supplier could embed instructions in a scope of work document. A competitor could theoretically hide something in a public filing that your AI ingests as competitive research. Most marketing teams have not thought about prompt injection as a workflow risk, as it has tended to live in the security team's lane. The legal filing story is a useful prompt (apologies) to audit where your AI tools are reading third-party documents and what guardrails, if any, are in place.
Quick Hits
- AI referral traffic to Shopify merchants is up 197% year over year and converts better with new buyers, though Google still owns the majority of traffic overall. Read more
- LinkedIn now lets users report content as "AI slop," which is both a reasonable policy response and a mildly humiliating flag to receive from a colleague. Read more
- ChatGPT names brands in its internal queries before retrieving results, making pre-fetch inclusion worth 33x more in recommendation likelihood than showing up only in retrieval. Read more
- Someone hid a prompt injection attack in a legal filing instructing AI systems to rule in their favour, the first documented case of the technique being used in a legal context. Read more
- Shopify's AI traffic data is also a useful reminder that conversion rate, not just volume, is the metric worth tracking as the channel matures. Read more
- LinkedIn's slop flag is crowdsourced, which means it will probably be applied inconsistently, but the direction of travel for platform AI content policy is fairly clear. Read more
This article was obviously generated with AI but curated by a human, don't be weird about it.
Compiled by
Pete Bishop
Chief Innovation Officer, ZGM Modern Marketing Partners
Pete Bishop has spent the last two decades helping brands adopt new technology without losing the plot. He hosts Artificial Breakdown, a podcast and weekly newsletter that translates AI news into practical marketing decisions, and scans 50+ sources each week to write this issue.
Get this in your inbox
One email a week with the AI and marketing news worth your time.