Traffic Routing: Ingress Rules, Origin Services, and the Proxy Layer

In Part 3, we traced request handling down to the dispatchRequest call in the QUIC transport, which calls orchestrator.GetOriginProxy() and then ProxyHTTP or ProxyTCP. But what happens inside those proxy methods? How does cloudflared decide which of your origin services should handle a request for api.example.com/v2/users versus dashboard.example.com?

The answer lies in cloudflared's ingress system — a two-tier rule matching engine that evaluates internal rules before user-defined rules, supports hostname wildcards and path regex, and maps each match to a specific origin service type. This article covers the full routing pipeline from rule matching through origin delivery.

Two-Tier Ingress Rule Architecture

Cloudflared's Ingress struct maintains two separate rule lists:

type Ingress struct {
    InternalRules []Rule    // System-defined, negative indices
    Rules         []Rule    `json:"ingress"`  // User-defined, positive indices
    Defaults      OriginRequestConfig `json:"originRequest"`
}

flowchart TD
    Req[Incoming Request] --> Internal{Check InternalRules}
    Internal -->|Match at index i| IntResult[Return rule, index = -1-i]
    Internal -->|No match| User{Check User Rules}
    User -->|Match at index i| UserResult[Return rule, index = i]
    User -->|No match| CatchAll[Return last rule = catch-all]
    
    IntResult --> Service[Route to service]
    UserResult --> Service
    CatchAll --> Service
    
    style Internal fill:#dc2626,color:#fff
    style User fill:#2563eb,color:#fff

The FindMatchingRule method scans internal rules first, then user rules. Internal rules return negative indices (-1, -2, ...) to distinguish them from user rules in log output:

for i, rule := range ing.InternalRules {
    if rule.Matches(hostname, path) {
        return &rule, -1 - i
    }
}

Currently, the only internal rule is the management service. In StartServer, it's injected as:

internalRules := []ingress.Rule{ingress.NewManagementRule(mgmt)}

This design ensures that no user configuration can accidentally override the management endpoint. Even if a user creates a wildcard catch-all rule matching everything, the management rule is always checked first.

Tip: When debugging routing issues, check the rule index in the logs. Negative indices mean internal rules matched (likely the management service), while positive indices correspond to your config file rules (0-based). The catch-all rule is always the last index.

Rule Matching: Hostname, Path, and Wildcards

Each rule can specify a hostname pattern and a path regex. The matchHost function supports exact matches and wildcard subdomains:

flowchart TD
    Host[Request hostname] --> Exact{Exact match?}
    Exact -->|api.example.com = api.example.com| Match[✓ Match]
    Exact -->|No| Wild{Rule starts with '*.'?}
    Wild -->|Yes| Suffix{Request ends with suffix?}
    Wild -->|No| NoMatch[✗ No match]
    Suffix -->|*.example.com → .example.com suffix| Match
    Suffix -->|No| NoMatch
    
    Match --> Path{Path regex?}
    Path -->|None| FullMatch[Rule matches]
    Path -->|Defined| Regex{Regex matches path?}
    Regex -->|Yes| FullMatch
    Regex -->|No| NoMatch

The wildcard implementation is intentionally restrictive — only *. at the start of a hostname is allowed. * anywhere else is rejected during validation. The last rule in any ingress configuration must be a catch-all (no hostname, no path), which is enforced by validateHostname. Requests that don't match any explicit rule always hit this catch-all.

Path matching uses Go's standard regexp package. The path regex is compiled once during configuration parsing and stored as a *Regexp in the rule. An empty path means "match all paths."

Origin Service Type Hierarchy

The OriginService interface is the abstraction for anything cloudflared can proxy traffic to:

type OriginService interface {
    String() string
    start(log *zerolog.Logger, shutdownC <-chan struct{}, cfg OriginRequestConfig) error
    MarshalJSON() ([]byte, error)
}

classDiagram
    class OriginService {
        <<interface>>
        +String() string
        +start(log, shutdownC, cfg) error
        +MarshalJSON() ([]byte, error)
    }
    
    class HTTPOriginProxy {
        <<interface>>
        +RoundTrip(req) (resp, error)
    }
    
    class StreamBasedOriginProxy {
        <<interface>>
        +EstablishConnection(ctx, dest, log) (OriginConnection, error)
    }
    
    class HTTPLocalProxy {
        <<interface>>
        +ServeHTTP(w, r)
    }
    
    OriginService <|-- httpService : HTTP/HTTPS origins
    OriginService <|-- unixSocketPath : Unix sockets
    OriginService <|-- tcpOverWSService : TCP over WebSocket
    OriginService <|-- helloWorld : Built-in test server
    OriginService <|-- statusCode : Static status responses
    OriginService <|-- socksProxyOverWSService : SOCKS5 proxy
    OriginService <|-- bastionService : SSH bastion
    
    httpService ..|> HTTPOriginProxy
    unixSocketPath ..|> HTTPOriginProxy
    tcpOverWSService ..|> StreamBasedOriginProxy
    bastionService ..|> StreamBasedOriginProxy
    helloWorld ..|> HTTPLocalProxy

The origin services implement three proxy interfaces that the Proxy layer uses for dispatch:

Interface	Behavior	Services
HTTPOriginProxy	Standard HTTP round-trip via RoundTrip()	httpService, unixSocketPath
StreamBasedOriginProxy	Bidirectional byte stream via EstablishConnection()	tcpOverWSService, bastionService, socksProxyOverWSService
HTTPLocalProxy	Serve directly via ServeHTTP()	helloWorld, statusCode, ManagementService

The httpService at origin_service.go#L70-L94 is the most common type — it creates an http.Transport with configurable timeouts, TLS settings, and connection pooling for proxying to your HTTP origins.

The Proxy Layer: ProxyHTTP Dispatch

The Proxy.ProxyHTTP method is the central dispatch point. After finding the matching ingress rule, it performs a type-switch on the rule's service:

flowchart TD
    ProxyHTTP[ProxyHTTP called] --> Tags[Append tag headers]
    Tags --> Match[FindMatchingRule]
    Match --> MW[Apply ingress middleware]
    MW --> TypeSwitch{Service type?}
    
    TypeSwitch -->|HTTPOriginProxy| HTTP[proxyHTTPRequest]
    HTTP --> RT[RoundTrip to origin]
    RT --> Headers[Write response headers]
    Headers --> Body[Copy response body]
    
    TypeSwitch -->|StreamBasedOriginProxy| Stream[proxyStream]
    Stream --> Establish[EstablishConnection]
    Establish --> Ack[AckConnection]
    Ack --> Pipe[stream.Pipe bidirectional]
    
    TypeSwitch -->|HTTPLocalProxy| Local[proxyLocalRequest]
    Local --> ServeHTTP[proxy.ServeHTTP]

The middleware layer (called before dispatch) supports JWT validation for Cloudflare Access integration. If a rule has an access configuration with required: true, a JWT validator middleware is injected that rejects requests without valid Access tokens.

For HTTP origin proxying, proxyHTTPRequest handles WebSocket upgrades specially — it sets Connection: Upgrade and Upgrade: websocket headers, clears the body, and after the RoundTrip, checks for 101 Switching Protocols to initiate bidirectional streaming via stream.Pipe.

TCP Proxying and Flow Limiting

ProxyTCP handles WARP routing — private network TCP connections that arrive as raw TCP streams. Before dialing the origin, it acquires a flow limiter token:

if err := p.flowLimiter.Acquire(management.TCP.String()); err != nil {
    logger.Warn().Msg("Too many concurrent flows being handled, rejecting tcp proxy")
    return errors.Wrap(err, "failed to start tcp flow due to rate limiting")
}
defer p.flowLimiter.Release()

The flowLimiter is a mutex-protected counter with a configurable limit. When the limit is 0, it operates in unlimited mode. The limit can be hot-swapped via SetLimit() without restarting cloudflared — this is used when the Orchestrator receives a configuration update that changes warp-routing.maxActiveFlows.

sequenceDiagram
    participant Edge as Edge Request
    participant Proxy as ProxyTCP
    participant Limiter as flowLimiter
    participant Origin as Origin TCP

    Edge->>Proxy: ProxyTCP(ctx, conn, req)
    Proxy->>Limiter: Acquire("tcp")
    alt Limit reached
        Limiter-->>Proxy: ErrTooManyActiveFlows
        Proxy-->>Edge: Error response
    else Slot available
        Limiter-->>Proxy: nil (counter++)
        Proxy->>Origin: DialTCP(dest)
        Origin-->>Proxy: Connection
        Proxy->>Proxy: AckConnection
        Proxy->>Proxy: stream.Pipe(tunnelConn, originConn)
        Note over Proxy: Bidirectional copy until close
        Proxy->>Limiter: Release() (counter--)
    end

The flow limiter is shared between TCP (via ProxyTCP) and UDP (via SessionManager.RegisterSession) sessions, providing a unified backpressure mechanism across all tunnel traffic types.

The shouldFlush Heuristic for Streaming Responses

One of the subtlest pieces of cloudflared's proxy layer is the shouldFlush function. HTTP responses are typically buffered for efficiency, but streaming protocols like Server-Sent Events (SSE) and gRPC require immediate flushing after each write.

func shouldFlush(headers http.Header) bool {
    if contentLength := headers.Get(contentLengthHeader); contentLength == "" {
        return true
    }
    if transferEncoding := headers.Get(transferEncodingHeader); transferEncoding != "" {
        if strings.Contains(strings.ToLower(transferEncoding), "chunked") {
            return true
        }
    }
    if contentType := headers.Get(contentTypeHeader); contentType != "" {
        for _, c := range flushableContentTypes {
            if strings.HasPrefix(contentType, c) {
                return true
            }
        }
    }
    return false
}

The heuristic uses three signals:

1. No Content-Length — streaming responses typically don't know their length upfront
2. Chunked Transfer-Encoding — explicitly indicates streaming
3. Content-Type — text/event-stream (SSE), application/grpc, and application/x-ndjson are known streaming types

This is called in the HTTP/2 response writer's WriteRespHeaders method. Once shouldFlush returns true, every subsequent Write call is immediately followed by a Flush, preventing buffering delays that would break real-time protocols.

Tip: If your origin service uses a streaming protocol that's being buffered by cloudflared, check that it's setting one of these signals in its response headers. Adding Transfer-Encoding: chunked or removing Content-Length will trigger immediate flushing.

What's Next

We've traced the complete request path from transport to origin delivery. In Part 5, we'll examine cloudflared's most elegant engineering pattern: the Orchestrator's copy-on-write proxy swap that enables zero-downtime configuration updates, the feature flag system that uses DNS TXT records for gradual rollouts, and the full configuration hierarchy from CLI flags to edge-pushed remote config.