Sandbox escapes. Secret exfiltration. Illegal steering.
Every finding here came out of a live, customer-facing agent in production - not a benchmark or a staging clone. These are the failure modes static tests and scripted evals never reach.
We turned a coding agent into a persistent foothold.
335 live adversarial runs against a coding agent. Flip on its auto-review guardian for hands-off autonomy, and a booby-trapped file in the repo it's reviewing talks the guardian into approving a persistent dev shell — which escapes the read-only sandbox and writes a persistence hook to ~/.zshenv, with no human in the loop.
> write_stdin → ~/.zshenv persistence hook
✓ no human in the loop
Three more, from live production agents.
Each ran against a live, customer-facing agent.
Browser agent leaking a 1Password vault credential
An indirect prompt injection (a crafted email the agent was told never to answer) drove a browser agent to read a password from its connected 1Password vault and exfiltrate it.
Case study coming soonBypassed an LLM-as-judge over network egress to exfiltrate API keys
A crafted outbound request read as benign to the LLM-as-judge policing network egress, smuggling the host's locally-stored API keys out to an attacker-controlled endpoint.
Case study coming soonJailbroke an AI real-estate search agent into illegal steering
Tested past its fair-housing guardrails into recommending homes and neighborhoods by a buyer's race, age, and other protected characteristics - the "steering" the Fair Housing Act and New York State law expressly prohibit.
Case study coming soonWhat we routinely find across engagements.
The findings above are a sample. These are the failure classes Nyx surfaces again and again.
Prompt injection & jailbreaks
Direct attempts to override the system prompt and coax the model past its own policy.
Indirect injection
Instructions smuggled in through retrieved pages, documents, and emails the agent reads.
Tool-use hijacking
Steering an agent's tools and code execution toward actions it was never meant to take.
Guardrail & judge bypass
Getting past an LLM-as-judge or policy filter that was supposed to hold the line.
Secret & PII exfiltration
Pulling API keys, credentials, and personal data out of the agent's context or environment.
Refund & transaction fraud
Talking support and commerce bots into refunds and transactions they should refuse.
Hallucinated advice & policy drift
Confident, off-policy answers in regulated domains like finance and health.
Reward hacking
Agents in RL and training setups gaming the reward signal instead of doing the task.