From main() to Response: The Complete Request Lifecycle in workerd

In Part 1, we mapped out the directory structure and the six-layer architecture. Now we're going to walk through the code, tracing the exact path from the moment the workerd binary starts to the moment a JavaScript fetch() handler sends back a response. Along the way, we'll encounter a remarkably clever compiled-binary trick, a 6000-line "god file," and a threading model that bridges two fundamentally different memory worlds.

CLI Entry and Subcommands

The process begins in src/workerd/server/workerd.c++, which defines the CliMain class. This class uses KJ's MainBuilder pattern to register subcommands:

return kj::MainBuilder(context, getVersionString(),
                       "Runs the Workers JavaScript/Wasm runtime.")
    .addSubCommand("serve", KJ_BIND_METHOD(*this, getServe), "run the server")
    .addSubCommand("compile", KJ_BIND_METHOD(*this, getCompile),
                   "create a self-contained binary")
    .addSubCommand("test", KJ_BIND_METHOD(*this, getTest), "run unit tests")
    .addSubCommand("pyodide-lock", KJ_BIND_METHOD(*this, getPyodideLock),
                   "outputs the package lock file used by Pyodide")
    .build();

The four subcommands serve distinct purposes: serve runs the server, test executes worker test handlers, pyodide-lock outputs Python dependency information, and compile does something unusual — it creates a self-contained binary.

The Compiled Binary Trick

One of the most inventive features in workerd is the ability to compile a worker into a standalone binary. The compile subcommand serializes the Cap'n Proto config (including all embedded source code) and appends it to the end of the workerd binary itself, followed by a magic suffix.

When the binary starts, the constructor checks whether it is such a compiled binary by reading the tail bytes of its own executable (lines 706-728):

if (kj::arrayPtr(magic) == kj::asBytes(COMPILED_MAGIC_SUFFIX)) {
    // Oh! It appears we are running a compiled binary, it has a config appended.
    // ... read the config from the end of the binary
    config = capnp::readMessageUnchecked<config::Config>(
        reinterpret_cast<const capnp::word*>(mapping.begin()));
}

If the magic bytes match, the binary skips the subcommand menu entirely and goes straight to serving. This means you can distribute a single executable that contains both the runtime and your application — no config files, no JavaScript files, just one binary.

flowchart TD
    Start["workerd binary starts"] --> Check{"Magic bytes at end of file?"}
    Check -->|Yes| Compiled["Read embedded config → serve"]
    Check -->|No| SubCmd["Parse CLI subcommand"]
    SubCmd --> Serve["workerd serve config.capnp"]
    SubCmd --> Compile["workerd compile → append config to binary"]
    SubCmd --> Test["workerd test"]
    Serve --> ParseConfig["Parse Cap'n Proto config"]
    Compiled --> Run["Server::run()"]
    ParseConfig --> Run

Server::run() — The Bootstrap Sequence

Once the config is parsed, the serve subcommand calls Server::run(). This is the heart of the startup process:

kj::Promise<void> Server::run(
    jsg::V8System& v8System, config::Config::Reader config,
    kj::Promise<void> drainWhen) {
  // 1. Configure logging from config
  // 2. Build HTTP header table
  kj::HttpHeaderTable::Builder headerTableBuilder;
  globalContext = kj::heap<GlobalContext>(*this, v8System, headerTableBuilder);

  // 3. Set up fatal error handling
  auto [fatalPromise, fatalFulfiller] = kj::newPromiseAndFulfiller<void>();

  // 4. Set up drain handling
  auto forkedDrainWhen = handleDrain(kj::mv(drainWhen)).fork();

  // 5. Construct all services (including compiling all workers)
  co_await startServices(v8System, config, headerTableBuilder, forkedDrainWhen);

  // 6. Bind all sockets
  auto listenPromise = listenOnSockets(config, headerTableBuilder, forkedDrainWhen);

  // 7. Finalize the header table
  auto ownHeaderTable = headerTableBuilder.build();

  // 8. Wait for either socket closure or fatal error
  co_return co_await listenPromise.exclusiveJoin(kj::mv(fatalPromise));
}

The GlobalContext struct (defined at line 200) bundles the V8 system, the ThreadContext, and the HTTP header table — the shared infrastructure that all services need.

sequenceDiagram
    participant CLI as CliMain
    participant S as Server
    participant GC as GlobalContext
    participant Svc as Services
    participant Sock as Sockets

    CLI->>S: run(v8System, config, drainWhen)
    S->>GC: Create GlobalContext (V8System + HeaderTable)
    S->>Svc: startServices() — construct all Service objects
    Svc-->>S: Services linked and ready
    S->>Sock: listenOnSockets() — bind ports
    Sock-->>S: Accepting connections
    S->>S: co_await (listen ∥ fatalError)

Tip: Notice that startServices() is awaited before listenOnSockets(). This means all workers are compiled and all service cross-references are resolved before any sockets start accepting connections. However, the header table is finalized after socket setup begins — headers can be registered lazily during service construction.

The Service Hierarchy

The Service base class is defined as a private class within server.c++. It defines the contract every service must fulfill:

class Server::Service: public IoChannelFactory::SubrequestChannel {
 public:
  virtual void link(Worker::ValidationErrorReporter& errorReporter) {}
  virtual void unlink() {}
  virtual kj::Own<WorkerInterface> startRequest(
      IoChannelFactory::SubrequestMetadata metadata) override = 0;
  virtual bool hasHandler(kj::StringPtr handlerName) = 0;
};

The link()/unlink() lifecycle is crucial: after all services are constructed, link() is called to resolve cross-service references (e.g., a service binding from one worker to another). Before destruction, unlink() breaks these cycles so kj::Own<T> pointers can be safely destroyed.

The concrete implementations declared at the bottom of server.h (lines 294-305):

classDiagram
    class Service {
        +link()
        +unlink()
        +startRequest() WorkerInterface
        +hasHandler() bool
    }
    class WorkerService {
        -LinkedIoChannels ioChannels
        -Own~Worker~ worker
        +startRequest()
    }
    class NetworkService {
        +startRequest()
    }
    class ExternalHttpService {
        +startRequest()
    }
    class ExternalTcpService {
        +startRequest()
    }
    class DiskDirectoryService {
        +startRequest()
    }

    Service <|-- WorkerService
    Service <|-- NetworkService
    Service <|-- ExternalHttpService
    Service <|-- ExternalTcpService
    Service <|-- DiskDirectoryService

WorkerService executes JavaScript. NetworkService provides outbound network access (it backs the implicit "internet" service). ExternalHttpService and ExternalTcpService forward to a specific remote address. DiskDirectoryService serves files from a directory.

WorkerService — The God Object

WorkerService is the most complex class in the entire codebase. Its declaration tells the story:

class Server::WorkerService final: public Service,
                                   private kj::TaskSet::ErrorHandler,
                                   private IoChannelFactory,
                                   private TimerChannel,
                                   private LimitEnforcer {

It implements five interfaces simultaneously. This is not accidental — it's a deliberate design choice. A WorkerService is the complete I/O environment for a single worker. It provides:

• Service: request dispatch
• IoChannelFactory: subrequest routing, actor namespace lookups, cache access
• TimerChannel: timer implementation (setTimeout, etc.)
• LimitEnforcer: CPU/memory limits
• TaskSet::ErrorHandler: background task error handling

This consolidation means that server.c++ is enormous (~6000 lines), but it also means that all I/O concerns for a worker live in one place. When you need to understand how a worker's subrequest is routed, you don't need to chase through multiple abstraction layers — it's all in WorkerService.

WorkerEntrypoint and Per-Request Dispatch

When an HTTP request arrives on a socket, the Server's HTTP listener calls Service::startRequest(), which returns a WorkerInterface. For WorkerService, this creates a WorkerEntrypoint.

The static factory method WorkerEntrypoint::construct() is where the per-request machinery is assembled:

static kj::Own<WorkerInterface> construct(
    ThreadContext& threadContext,
    kj::Own<const Worker> worker,
    kj::Maybe<kj::StringPtr> entrypointName,
    Frankenvalue props,
    kj::Maybe<kj::Own<Worker::Actor>> actor,
    kj::Own<LimitEnforcer> limitEnforcer,
    kj::Own<void> ioContextDependency,
    kj::Own<IoChannelFactory> ioChannelFactory,
    kj::Own<RequestObserver> metrics,
    kj::TaskSet& waitUntilTasks,
    bool tunnelExceptions,
    kj::Maybe<kj::Own<BaseTracer>> workerTracer,
    ...);

Inside construct(), the init() method creates the IoContext — the per-request bridge that we'll explore deeply in Part 5. The IoContext binds a Worker instance to the current KJ event loop thread, providing the arena for all I/O objects this request will create.

sequenceDiagram
    participant HTTP as HTTP Listener
    participant WS as WorkerService
    participant WE as WorkerEntrypoint
    participant IoCtx as IoContext
    participant Lock as Worker::Lock
    participant JS as ServiceWorkerGlobalScope

    HTTP->>WS: startRequest(metadata)
    WS->>WE: construct(worker, ioChannelFactory, ...)
    WE->>IoCtx: Create IoContext
    Note over WE: IncomingRequest created
    HTTP->>WE: request(method, url, headers, body)
    WE->>Lock: takeAsyncLock()
    Lock-->>WE: Lock acquired
    WE->>JS: request(method, url, headers, body, response)
    JS-->>WE: Response sent
    WE->>IoCtx: drain() — wait for waitUntil() tasks

The request then flows to ServiceWorkerGlobalScope::request(), declared in global-scope.h. This method is the bridge between C++ and JavaScript — it invokes the worker's fetch() handler via JSG.

The drain/waitUntil Lifecycle

After the response is sent, the request isn't necessarily "done." Workers can call ctx.waitUntil(promise) to schedule background work that runs after the response. The IncomingRequest class manages this lifecycle:

1. delivered() — called when the request begins executing JavaScript
2. Response sent — the HTTP response body is transmitted to the client
3. drain() — waits for all waitUntil() tasks to complete (with soft timeout)
4. Destruction — the IoContext and all its I/O objects are torn down

For actors (Durable Objects), the lifecycle is more complex: an IoContext persists across multiple incoming requests, and drain() only returns when either all tasks complete, a new request arrives, or the actor shuts down.

Tip: The waitUntilTasks parameter in WorkerEntrypoint::construct() is a kj::TaskSet& owned by the WorkerService, not by the request. This means background tasks can outlive the individual request's scope, which is exactly what waitUntil() needs.

Putting It All Together

Let's trace the complete path for our Hello World example from Part 1:

1. workerd serve config.capnp — CliMain parses the config
2. Server::run() — creates GlobalContext, starts services
3. startServices() — constructs a WorkerService for "main", compiles worker.js into a Worker::Script, instantiates a Worker
4. listenOnSockets() — binds *:8080, creates HTTP listener
5. Request arrives — listener calls WorkerService::startRequest()
6. WorkerEntrypoint::construct() — creates IoContext
7. WorkerEntrypoint::request() — acquires async lock, enters V8
8. ServiceWorkerGlobalScope::request() — invokes export default { fetch() } via JSG
9. JavaScript runs: return new Response("Hello World\n")
10. Response sent — HTTP response streamed to client
11. IncomingRequest::drain() — no waitUntil tasks, returns immediately
12. IoContext destroyed — request complete

This entire flow happens in a single thread on a single KJ event loop. There are no thread switches for a normal stateless request — the only async points are KJ promise resolutions, which are single-threaded cooperative multitasking.

What's Next

We've now seen the complete request lifecycle at the C++ orchestration level. But we glossed over the most interesting part: how does ServiceWorkerGlobalScope::request() actually call into JavaScript? How does a C++ kj::String become a JS string, or a C++ jsg::Ref<Response> become a JavaScript Response object? In Part 3, we dive into JSG — the macro-driven binding system that is workerd's most architecturally distinctive subsystem.