Traefik Labs has expanded Traefik Hub’s Triple Gate architecture, which combines API Gateway, Artificial Intelligence Gateway, and MCP Gateway, with broader runtime governance across the full Artificial Intelligence workflow. The release adds a composable multi-vendor safety pipeline with parallel guard execution, multi-provider failover routing, token-level cost controls, graceful error handling for agent-aware enforcement, IBM Granite Guardian integration, and a Regex Guard capability for custom guards. The company is positioning the update as a response to fragmented governance as enterprises adopt autonomous agents across varied environments.
The product addresses gaps left by tools that only inspect one layer of traffic. An LLM proxy sees the model interaction but not the agent actions that follow, while an MCP proxy sees tool calls but not the LLM conversation that triggered them. Traefik’s approach is to govern content safety, cost, resilience, and agent authorization from one infrastructure layer that organizations operate themselves, whether in public cloud or air-gapped environments. It is designed to work with any agent platform because it governs traffic rather than the runtime.
The new safety pipeline is built across four tiers. Regex guard lets teams create custom regex-based guards with sub-millisecond speed and zero external dependencies to block or mask known patterns such as Social Security numbers, credit card formats, API keys, or business-specific data. Content guard uses Microsoft Presidio for PII detection and masking. LLM guard with NVIDIA NIMs adds GPU-accelerated jailbreak detection, content safety across 22+ categories, and topic control. LLM guard with IBM Granite Guardian brings harm detection, jailbreak detection, topic control, hallucination detection, and RAG quality assessment.
Parallel guard execution changes how these checks run. LLM-based guards, the heavyweight tiers that can take multiple seconds to execute, now run in parallel rather than in series. Guards are classified as critical or optional, allowing failures in some checks to be logged while others can block and cancel a request. Multiple NVIDIA NIMs and IBM Granite guards can execute simultaneously, so total enforcement time equals the slowest guard, not the sum.
Traefik also introduced operational controls focused on resilience and spending. The failover router provides automatic failover across LLM providers and models via circuit breaker chain, allowing organizations to mix OpenAI, Anthropic, self-hosted models, and NVIDIA NIMs in a single failover chain while keeping safety policies enforced. Token rate limiting and quota management tracks input, output, and total tokens independently, with controls applied per-user, per-team, per-endpoint, or per-API-key through JWT claims. Guardrails can also return structured refusal responses over HTTP 200 instead of HTTP 403, helping agents continue multi-step workflows without breaking middleware chains or crashing task execution.
