Gen AI threat modelling - Old threats, new interfaces

OWASP was a great experience, it had been a while since I'd joined a security conference and I'd forgotten the electrifying energy.

My main reason for attending was to catch up on threat modelling and more specifically applying it to GenAI / LLM. Now, I'm not exactly new at threat modelling, my last paper on the subject is 11 years old, which justified some catching up.

Key takeaways for me are that:

• Applying standard STRIDE is surprisingly functional (STRIDE just won't die)
• Mappings do exist, but the paint is still fresh
• GenAI tools for threat modeling are also a thing, but your mileage may vary

The training provided by Toreon was extremely hands-on and as usual being three days in a room full of professionals fighting similar challenges is amazing.

At the same time, it's starting to feel like I've been around the block a few times, and that the same issues are back with a statistics twist.

• Prompt injection --> SQL injections
• Unbounded consumption --> Rate limiting
• ...

I can see some obvious challenges specific to GenAI:

• The explainability space will be a major challenge
• Human in the loop is not going to hold forever

So, for my day-to-day work my takeaway here is:

• Treat AI projects like you would any other, threat models, mitigations...
• Keep an eye on legislation changes (EU AI Act, UK AI Act) and new legal precedents, this is where assumptions will become reality in the near future.
• Document / ledger decisions related to these requirements as it's likely we will revisit most of them sooner rather than later.

But what about model poisoning, hidden artefacts...

Ok so I'm going to have to draw a line here, while this is a fascinating topic, it also isn't currently part of my scope and probably won't be for most of us.

By definition not everyone works in the AI model space. I'd even posit that for this to work economically there should be more consumers than providers.

Many of the attack vectors in the top 10 are beyond my current scope.

When working on internal tooling this reduces mostly to:

• Prompt injections
• Sensitive information disclosure
• Unbounded consumption

From a shared responsibility model, everything else shouldn't be my problem. I would argue that prompt injections shouldn't be either.

Wait this is too easy...

Ok so I did just brush of prompt injections with a, well let's just apply a control to it and it's fixed, magic wand... and yet that control doesn't really exist?

In a way I did, here's Microsoft's take on (mitigations for their products)[https://msrc.microsoft.com/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks/]. Spoilers, it's not there yet but it's taking shape.

I understand this is a new space and we are moving at unprecedented speed but at the end of the day companies are going to have to provide a functioning product to customers. At that point there is a reasonable expectation that we will be provided with adequate mitigations in place.

Anything else is insane, would you buy a hammer that might explode once every 10,000 hits?

But this wouldn't be fun at a conference without shenanigans...

Playing with QR codes and badges

During breaks at OWASP there was a question nagging me, QR codes attack surface.

Similarly to GenAI, QR codes have grown into a new way of getting users to click on links, If you're in contact with your SOC department, you've likely seen the surge of QR codes to bypass url / email security and we know we can't post a picture of a concert ticket / Amazon voucher as it will be redeemed faster than lighting.

So with my lab partner we came up with a quick test, I'd post a picture of me with the OWASP badge, modify the QR code and post on LinkedIn.

Make it crappy enough so that humans should be able to spot the swap and you've got a simple test.

And surprisingly, nothing much of anything! Tracking was pretty simple, tracking code on the page for human visitors and raw log outputs for direct page access.

Some friends scanned my badge, costing me a few beers but beyond that there wasn't a meaningful spike in traffic.

The original linked in post can be found here orginal post and comment on why you ended here if you want :)

Go back home.