Datadog builds agentic system to hunt exploitable vulnerabilities

Datadog has built an orchestration pipeline that gives AI agents the security context needed to assess whether potential vulnerabilities are exploitable. The system uses human-reviewed threat packs, model routing and a separate validation step to improve the quality of findings.
- Datadog’s pipeline supplies AI agents with system-specific security context.
- Threat packs contain invariants, architecture details, priorities and compensating controls.
- Agents route discovery and reproduction tasks to suitable models.
- A second agent validates whether a finding links a reachable entry point to a violated invariant.
- Datadog says the approach uncovered guest-to-host escapes in open source components.
Why vulnerability hunting needs more context
Datadog says current closed- and open-weight AI models can identify potential vulnerabilities through skills, tools and prompting. The more difficult task is determining whether an issue can actually be exploited after accounting for a service’s business logic and existing security controls.
That challenge is particularly significant for production services that depend on frameworks, infrastructure and other services maintained by many teams. An agent examining one repository may not see authentication checks or network restrictions implemented elsewhere, or understand the security requirements the service must uphold.
Datadog’s pipeline is designed to provide that missing context. It also allows the company to repeat hunts against known vulnerabilities and check whether agents continue to find them after a model or agent harness changes.
Threat packs give agents system-specific security requirements
The workflow centers on threat packs, which Datadog describes as compact, human-reviewed documents that turn security expertise into context for an agent-led hunt. Each pack combines a system summary, security invariants, review priorities and known compensating controls.
A threat pack uses a threat-pack.md file and a metadata.yaml file. The Markdown file covers the system summary, security invariants, review priorities and known compensating controls, while the YAML file identifies the pack’s scope, maintainer and code locations. Datadog versions both files so it can reconstruct the context used in a particular hunt and compare it with later versions.
The packs are usually around 300 lines, which Datadog says provides the required security context while using only a small share of an agent’s context window. The company says it has created hundreds of packs, prioritizing systems by risk. Agents draft initial versions using code, penetration test reports, risk reviews and design documents, while an interview skill guides questions for subject matter experts. Each pack receives human review before use.
Findings must connect an entry point to a violated invariant
Datadog’s pipeline separates vulnerability discovery and reproduction into different tasks, routing each task to the most cost-effective model capable of completing it. The workflow is intended to remain consistent across model providers and agent harnesses, which run the agents and connect them to suitable tools.
Before a report reaches triage, a second agent, acting as a judge, reviews the initial candidate finding against the relevant threat pack. The report must identify a declared invariant, name an entry point reachable by an attacker and explain how that path could violate the invariant. Findings that meet those conditions become qualified findings and can proceed to a proof-of-concept attempt.
Reports that cannot connect a reachable entry point to a violated invariant are marked disqualified and stop before reproduction, automated triage or remediation. Datadog retains those reports to identify missing context in a threat pack or evidence that the harness overlooked, then use that information to refine the pack or pipeline.
According to Datadog, hunts using the packs have uncovered guest-to-host escapes in open source components, including one through virtio-fs in Kata Containers’ Rust runtime and another caused by missing length validation in Dragonball’s virtio-blk implementation. The company says the packs have reduced noise and improved report quality, helping it prioritize findings more efficiently.
Datadog: How we built Datadog’s agentic vulnerability hunting system


