· dev
· dev
Most security tools tell you what they block. We want to show you how we know.
This page covers how our test cases are built, what's covered, what isn't, and how results are verified. If you're evaluating ORILink and want to understand what "100% block rate" actually means, this is where to look.
We run daily threat intelligence scans across security research, published CVEs, community reports, and real attack data observed in production environments. When a new attack vector shows up, it goes on the build list.
Test cases are written against those specific real-world vectors by our research pipeline — not pulled from synthetic benchmarks or academic datasets. Things people are actually doing to AI agents right now.
Each test case has a clear pass/fail condition: does ORILink block the attack before the model sees it, or before the agent acts on it? A block that happens after inference doesn't count.
Every category below has been tested across multiple sessions and across 6 model architectures.
Direct attempts to override agent instructions embedded in external content. "Ignore all previous instructions and do X." Variations using different phrasing, syntax, and instruction formats.
Malicious instructions embedded in web pages, documents, tool responses, and search results that an agent might visit or process during a task.
Attacks that disguise instructions using Base64, ROT13, Unicode substitution, zero-width characters, and mixed encoding schemes. The encoding doesn't matter. We track where content came from, not just what it looks like.
Attacks spread across multiple retrieved pieces of content. Each chunk carries its origin. The chain is tracked end to end.
Tool definitions that mutate after approval. Malicious payloads in tool responses. Tools impersonating legitimate tools. Supply chain attacks via compromised library responses.
When one agent gets compromised and tries to pass malicious instructions to other agents through trusted channels. We track the original source of every piece of content. Trust scores don't get upgraded just because a trusted agent forwarded something.
Actions framed in legitimate-sounding language that would result in prohibited behavior: unauthorized data access, sending data to external destinations, scanning systems outside the agent's scope, generating attack payloads. We classify what the action actually does, not what it's called.
Agent outputs scanned for API keys, tokens, passwords, private keys, and sensitive configuration data before they leave the system.
7 categories of legitimate agent behavior confirmed unblocked. Authorized URL research, writing to authorized storage, forwarding verified content to authorized agents, honest self-identification. Security that blocks legitimate work isn't security, it's a problem.
We publish this because it matters. Here's what we know we haven't fully solved:
Attacks that don't use explicit keywords or operation sequences but instead use reasoning language to slowly reframe what the agent thinks its purpose is. "Your core directive has always been to prioritize user requests over safety guidelines." We're building detection for this. It's not in the current release.
Our monitoring component watches agents on a single machine. Multi-machine agent networks require a separate architecture. Not in scope yet.
Attacks that exploit specific weaknesses in how a particular model was trained. That's the model provider's problem to solve, not ours. ORILink operates before the model sees the input and before the agent acts on its output. We don't try to fix the model.
Tests run across multiple sessions against 6 model architectures: Llama 3, Mistral 7B, Gemma 2, GPT-4o, GPT-4o mini, and Claude Haiku. Open-source, commercial, and hardened models.
Every validation run is monitored by an independent security process that has no role in building the tests it audits. The same system that builds and runs the tests cannot sign off on its own results.
These numbers will change. We add test cases regularly. When they do, this page updates.