Anatomy of a Request: Kong's Runloop from Rewrite to Log

With Kong initialized and workers ready (as covered in Parts 1 and 2), we can now follow a single HTTP request through every phase of processing. This is where the architecture comes alive — the router matches a route, plugins authenticate and transform, the balancer selects an upstream target, and the response flows back through filters and loggers.

The central design pattern is the before/after sandwich: at each phase, the runloop handler executes infrastructure logic (before), then plugins run, then the runloop executes cleanup logic (after). Understanding this pattern is the key to understanding Kong's extensibility.

The Before/After Sandwich Pattern

Every request-handling phase in kong/init.lua follows the same structure:

function Kong.access()
  local ctx = ngx.ctx
  -- timing setup...
  ctx.KONG_PHASE = PHASES.access
  runloop.access.before(ctx)                                    -- infrastructure
  local plugins_iterator = runloop.get_plugins_iterator()
  execute_collecting_plugins_iterator(plugins_iterator, "access", ctx)  -- plugins
  -- delayed_response handling...
  runloop.access.after(ctx)                                     -- infrastructure
  -- timing finalization...
end

The before hook handles things plugins shouldn't worry about: router execution, X-Forwarded-* header setup, protocol negotiation. The after hook handles balancer preparation and internal redirects for gRPC. Plugins run in the middle, in priority order, and can short-circuit the pipeline via kong.response.exit().

This pattern is defined in the runloop handler's return table at kong/runloop/handler.lua:

return {
  -- ...
  init_worker = { before = function() ... end },
  rewrite = { before = function(ctx) ... end },
  access = { before = function(ctx) ... end, after = function(ctx) ... end },
  -- ...
}

sequenceDiagram
    participant Phase as Kong.<phase>()
    participant Before as runloop.<phase>.before()
    participant Plugins as plugins_iterator
    participant After as runloop.<phase>.after()

    Phase->>Phase: Set timing markers
    Phase->>Before: Infrastructure logic
    Before-->>Phase: Router match, context setup
    Phase->>Plugins: Iterate in priority order
    Plugins->>Plugins: Plugin 1 (PRIORITY=1250)
    Plugins->>Plugins: Plugin 2 (PRIORITY=910)
    Plugins->>Plugins: Plugin N (PRIORITY=...)
    Plugins-->>Phase: All plugins executed
    Phase->>After: Cleanup, balancer prep
    After-->>Phase: Ready for next phase
    Phase->>Phase: Finalize timing

Rewrite and Access: Routing and Service Resolution

The rewrite phase (Kong.rewrite()) is relatively lightweight. It initializes the request context, stashes a context reference for cross-phase retrieval, sets the workspace, and runs only global-scope plugins (plugins not bound to any specific route/service/consumer). The runloop's rewrite.before() at line 1177 captures the server port and initializes tracing.

Note that the rewrite phase uses execute_global_plugins_iterator — not the collecting iterator. This is because route matching hasn't happened yet, so Kong can't determine which route/service-scoped plugins apply.

The access phase (Kong.access()) is where the heavy lifting happens. The runloop's access.before() at lines 1184–1404 does the following:

1. Router execution: router:exec(ctx) matches the request against all configured Routes, returning a match_t table with the matched Route, Service, upstream URL, and URI transformations.
2. Workspace assignment: ctx.workspace = match_t.route.ws_id
3. X-Forwarded- setup*: Trusted IP detection and forwarded header propagation
4. Protocol enforcement: HTTPS redirect, gRPC/HTTP2 validation
5. Balancer preparation: balancer_prepare() at lines 805–856 creates the balancer_data table with retry count, timeouts, and the upstream host/port

After access.before(), plugins execute via the collecting iterator (more on this in Part 4). If a plugin calls kong.response.exit(403), the request is short-circuited via the delayed response mechanism.

flowchart TD
    A[access.before] --> B[router:exec]
    B --> C{Match found?}
    C -->|No| D[404 No Route matched]
    C -->|Yes| E[Set workspace]
    E --> F[Parse X-Forwarded-* headers]
    F --> G{HTTPS required?}
    G -->|Yes, HTTP request| H[Redirect/426]
    G -->|No| I[balancer_prepare]
    I --> J[Set upstream vars]
    J --> K{gRPC service?}
    K -->|Yes| L[Internal redirect @grpc]
    K -->|No| M[Continue to plugins]

Tip: The ctx.balancer_data table created in balancer_prepare is one of the most important data structures in the request lifecycle. It carries scheme, host, port, retry count, timeouts, and the tries array that records each balancer attempt. Plugins can modify these values before the balancer executes.

The Balancer Phase: Upstream Selection and Retries

Kong.balancer() is unique among Kong's phases: it may be called multiple times per request (once per upstream attempt), and it operates under severe Nginx restrictions — no yielding, no cosocket I/O.

The balancer phase is entered after Nginx's proxy_pass triggers the balancer_by_lua_block. On the first try, it sets the max retries via set_more_tries(retries) at line 1380. On subsequent tries (retries), it records the failure from the previous attempt and re-executes the balancer:

if try_count > 1 then
  local previous_try = tries[try_count - 1]
  previous_try.state, previous_try.code = get_last_failure()
  -- report to health checker ...
  local ok, err, errcode = balancer.execute(balancer_data, ctx)
  -- ...
end

The balancer.execute() function from kong/runloop/balancer/init.lua handles DNS resolution and target selection. For services using an Upstream entity with multiple targets, it applies the configured load-balancing algorithm (round-robin, consistent-hashing, least-connections).

Connection keepalive is managed at lines 1388–1451. The keepalive pool key is a composite of IP, port, SNI, TLS verification settings, and client certificate — ensuring connections are only reused when all security parameters match:

pool = fmt("%s|%s|%s|%s|%s|%s|%s",
  balancer_data_ip, balancer_data_port,
  var.upstream_host,
  service.tls_verify and "1" or "0",
  service.tls_verify_depth or "",
  service.ca_certificates and concat(service.ca_certificates, ",") or "",
  service.client_certificate and service.client_certificate.id or "")

flowchart TD
    A[Kong.balancer] --> B{First try?}
    B -->|Yes| C[set_more_tries]
    B -->|No| D[Record previous failure]
    D --> E[Report to health checker]
    E --> F[balancer.execute - re-resolve]
    C --> G[set_current_peer IP:port]
    F --> G
    G --> H[set_timeouts]
    H --> I{Keepalive enabled?}
    I -->|Yes| J[enable_keepalive with pool key]
    I -->|No| K[Skip]
    J --> L[Record timing]
    K --> L

Response Processing: header_filter and body_filter

Once Nginx receives the upstream response, Kong.header_filter() processes the response headers. This phase uses execute_collected_plugins_iterator — it replays the plugin list that was built during the access phase's collecting iteration, without re-resolving which plugins apply.

The runloop's header_filter.before() adds Kong-specific response headers like X-Kong-Proxy-Latency and X-Kong-Upstream-Latency. The header_filter.after() sets the Via and Server headers. Between these, plugins can modify, add, or remove any response header.

Kong.body_filter() handles the response body. A critical detail: this phase is called multiple times for chunked responses. Each call processes one chunk, and the arg[2] flag (eof) indicates whether this is the last chunk.

At line 1716, a special case handles plugins that set ctx.response_body:

if ctx.response_body then
  arg[1] = ctx.response_body
  arg[2] = true
end

This allows plugins like ai-proxy to replace the entire response body with a transformed version, collapsing multiple chunks into a single emission.

The Log Phase and Request Context Lifecycle

Kong.log() is the final phase. By this point, the response has been sent to the client — the log phase runs asynchronously after the connection is closed (or at least after the response is fully buffered).

The timing instrumentation throughout kong/init.lua uses KONG_*_START and KONG_*_ENDED_AT markers on ctx. The log phase at lines 1761–1842 fills in any gaps — if a previous phase didn't record its end time (because an error occurred), the log phase calculates it retroactively. Key computed values include:

• KONG_PROXY_LATENCY: Time from request start to balancer completion
• KONG_WAITING_TIME: Time waiting for upstream response
• KONG_RECEIVE_TIME: Time spent in header_filter + body_filter
• KONG_RESPONSE_LATENCY: Total time for non-proxied responses

After plugins execute their log handlers, the context is cleaned up at lines 1849–1856:

plugins_iterator.release(ctx)
runloop.log.after(ctx)
-- ...
release_table(CTX_NS, ctx)

The release_table call returns the ctx table to a pool (tablepool) for reuse, avoiding GC pressure on subsequent requests.

sequenceDiagram
    participant Access as access phase
    participant Balancer as balancer phase
    participant Upstream as upstream
    participant HF as header_filter
    participant BF as body_filter
    participant Log as log phase

    Note over Access: KONG_ACCESS_START
    Access->>Access: plugins execute
    Note over Access: KONG_ACCESS_ENDED_AT
    Balancer->>Upstream: connect & send
    Note over Balancer: KONG_BALANCER_ENDED_AT
    Upstream-->>HF: response headers
    Note over HF: KONG_WAITING_TIME = now - BALANCER_ENDED_AT
    HF->>BF: headers processed
    BF->>BF: body chunks
    Note over BF: KONG_RECEIVE_TIME computed
    BF->>Log: final chunk (eof)
    Log->>Log: fill timing gaps
    Log->>Log: execute plugin log handlers
    Log->>Log: release ctx to tablepool

The Delayed Response Mechanism

The delayed_response mechanism at lines 377 and 414 deserves special attention. During the collecting phase (access), ctx.delay_response is set to true. When a plugin calls kong.response.exit(), instead of immediately sending a response, the exit status and body are stashed in ctx.delayed_response:

ctx.delayed_response = {
  status_code = 403,
  content = { message = "Forbidden" },
}

Subsequent plugins in the iterator see ctx.delayed_response and are skipped. After the iterator completes, flush_delayed_response sends the actual response. This design ensures all plugins have a chance to run their downstream phase handlers (header_filter, body_filter, log) even when the request is short-circuited.

Tip: The coroutine wrapping in execute_collecting_plugins_iterator (line 399–400) is not just for error handling. By running each plugin in a coroutine, Kong can catch runtime errors without crashing the entire request pipeline. The error is logged, a 500 is queued via delayed_response, and the next plugin is skipped.

What's Next

We've traced a request through every phase and seen how the runloop and plugins interact. But we glossed over a critical question: which plugins run for a given request, and in what order? Part 4 dives into the plugin system — the collecting/collected iterator pattern, the 8-level configuration resolution, and the anatomy of a plugin handler.