Notifications
Clear all
Topic starter
09/07/2026 12:36 pm
TL;DR: Prompt||GTFO Season 1 showed practical AI attacks that bypass content filters, confuse models through protocol tricks, and extract hidden prompts across sessions, according to Knostic. The field is moving from theory to live-fire testing, and conventional guardrails are no longer enough to govern how models behave in production.
NHIMG editorial — based on content published by Knostic: Prompt||GTFO Season 1 and the practical reality of AI security attacks
By the numbers:
- Running a single prompt across 30+ models simultaneously, he could quickly benchmark weaknesses and defenses across the entire AI landscape.
Questions worth separating out
Q: How should security teams govern AI systems that can influence tools and workflows?
A: Security teams should treat AI systems as governed runtime actors, not passive applications.
Q: Why do prompt attacks bypass traditional AI guardrails so easily?
A: Prompt attacks succeed when controls only inspect surface text and ignore how the model transforms, stores, and reuses context.
Q: How do teams know if AI guardrails are actually working?
A: Guardrails are working only if they produce consistent refusal, logging, and containment behaviour across models, versions, and input transformations.
Practitioner guidance
- Harden prompt ingress and decoding paths Inspect prompts before and after any transformation step, including encoding, templating, and retrieval augmentation.
- Reset security state across model version changes Define explicit reset and re-validation steps when a model, policy, or conversation context changes.
- Test model behaviour across the full AI estate Run the same adversarial prompt set across every production model, gateway, and configuration.
What's in the full article
Knostic's full research covers the operational detail this post intentionally leaves for the source:
- Step-by-step examples of the prompt confusion and self-incrimination techniques used against models.
- Concrete demonstrations of multi-model jailbreak testing across different providers and configurations.
- Practical logging and debugging patterns for non-deterministic AI applications.
- Additional session summaries from the Prompt||GTFO lineup that show how offensive AI tradecraft is evolving.
👉 Read Knostic's analysis of Prompt||GTFO Season 1 and the AI security attacks it surfaced →
Prompt attacks are outpacing AI guardrails, but what should teams do?
Explore further
09/07/2026 12:42 pm
Prompt security is now a governance problem, not a content-filter problem. The article shows that attackers can exploit protocol layers, context handling, and model self-reference in ways static filters do not anticipate. That shifts the control question from "what words are allowed" to "who can influence model state, tool use, and downstream action." Practitioners should treat AI prompt pathways as governed access surfaces.
A few things that frame the scale:
- 85% of organisations lack full visibility into third-party vendors connected via OAuth apps, according to The State of Non-Human Identity Security.
- 1 in 4 organisations are already investing in dedicated NHI security capabilities, with an additional 60% planning to do so within the next twelve months.
A question worth separating out:
Q: Who should be accountable when an AI system leaks unsafe instructions or data?
A: Accountability should sit with the team that owns the AI system's permissions, logging, and release controls, not only the model provider. If the system can persist context or reach enterprise data, the operating team must define acceptance criteria, monitoring thresholds, and incident response steps for model misuse.
👉 Read our full editorial: AI security guardrails are failing against real prompt attacks
