From workerd serve to Accepting Requests: The Boot and Server Lifecycle

Every server has a story that starts before the first request arrives. For workerd, that story involves parsing a Cap'n Proto config file, initializing V8 exactly once, constructing a directed graph of interconnected services, binding sockets, and entering an accept loop — all in about 400 lines of startup code. This article traces every step. Along the way, we'll encounter two remarkable features: self-contained compiled binaries (where config is literally appended to the executable) and live-reload file watching with inotify/kqueue.

CliMain and KJ's MainBuilder Pattern

The entry point is the CliMain class at line 680 of workerd.c++:

src/workerd/server/workerd.c++#L680-L738

KJ provides a MainBuilder class that is similar to argparse or cobra — it defines subcommands, options, and positional arguments, then dispatches to handler methods. CliMain::getMain() registers four subcommands:

flowchart TD
    Main["workerd"] --> Serve["serve<br/>Run the server"]
    Main --> Compile["compile<br/>Create self-contained binary"]
    Main --> Test["test<br/>Run unit tests"]
    Main --> PyLock["pyodide-lock<br/>Print Pyodide package lock"]

Each subcommand shares a common set of config-parsing options (added by addConfigParsingOptions) and a serve-or-test option set (added by addServeOrTestOptions). The serve command adds socket-related flags (--socket-addr, --socket-fd), while test adds --compat-date and --predictable for deterministic testing.

The CLI_METHOD macro wraps method calls in a try/catch that converts CliError exceptions into KJ's validity-based error reporting — a pattern that avoids the verbosity of Result<T, E> types that the codebase comments openly mock:

// Honestly I do not know how people put up with patterns like
// Result<T, E>, it seems like such a slog.

Compiled Binary Self-Extraction

This is one of workerd's most creative features. The workerd compile command produces a single, self-contained binary — no config file needed at deployment time. The mechanism is elegantly simple: the compiled binary is just the workerd executable with a Cap'n Proto config blob and a magic suffix appended to its end.

At startup, the CliMain constructor reads its own binary's tail:

src/workerd/server/workerd.c++#L706-L738

flowchart LR
    subgraph "Compiled Binary Layout"
        A["workerd executable bytes"] --> B["Cap'n Proto config (N words)"]
        B --> C["uint64: config size (N)"]
        C --> D["Magic suffix: COMPILED_MAGIC_SUFFIX"]
    end

    subgraph "Detection at Startup"
        E["Read last bytes of own executable"] --> F{"Magic suffix<br/>present?"}
        F -->|Yes| G["mmap config from offset<br/>Skip normal CLI parsing"]
        F -->|No| H["Normal subcommand dispatch"]
    end

When the magic suffix is detected, the config is memory-mapped directly from the binary file and deserialized via readMessageUnchecked — a zero-copy Cap'n Proto operation. The constructor then sets config to the parsed result, and getMain() skips subcommand registration entirely, building only the serve options.

This means a compiled binary behaves like any other executable: ./my-worker --socket-addr main=*:8080 — no config file argument needed.

Tip: If you want to inspect what's inside a compiled binary, use capnp decode on the extracted config section. The offset math is straightforward: subtract the magic suffix size and the uint64 size field from the end, then read backward by the config word count.

Config Parsing and Validation

For non-compiled binaries, config parsing proceeds through parseConfigFile() and setConstName(). The code supports two formats: text Cap'n Proto (human-readable, the default) and binary Cap'n Proto (used by Wrangler, enabled with --binary).

Text parsing uses a capnp::SchemaParser that handles imports. workerd registers several built-in schemas — like workerd.capnp itself and compatibility-date.capnp — via tryImportBuiltin(). This is what allows config files to write using Workerd = import "/workerd/workerd.capnp" without those files being on disk.

The SchemaFileImpl class wraps the KJ filesystem to provide schema files to the parser, and also serves double duty as the file watcher's file collector — every file opened during parsing is registered for watching.

After parsing, the config is validated by the Server during service construction. Invalid configurations produce InvalidConfigService instances that throw descriptive errors when invoked, rather than failing the entire startup. This graceful degradation is particularly valuable in --watch mode.

V8 Initialization and the V8System Class

Before any JavaScript can execute, V8 must be initialized exactly once per process. This is the job of jsg::V8System:

src/workerd/jsg/setup.h#L45-L84

V8System wraps the process-wide V8 initialization dance: creating a v8::Platform, calling v8::V8::Initialize(), and setting up any V8 flags from the config's v8Flags list. The class also manages the V8 platform wrapper, which hooks into KJ's event loop for background task scheduling.

sequenceDiagram
    participant CLI as CliMain
    participant V8S as V8System
    participant V8 as V8 Engine
    participant Server as Server

    CLI->>V8S: Construct (once per process)
    V8S->>V8: v8::V8::Initialize()
    V8S->>V8: SetFlagsFromString(config.v8Flags)
    CLI->>Server: server->run(v8System, config)
    Server->>V8S: Create Isolates via V8System

The V8System instance is created in CliMain's serve() method (or equivalent) and passed by reference to Server::run(). Individual V8 isolates are created later, when WorkerService instances are constructed. Each isolate is typed — meaning it knows the complete set of API types it supports — via the JsgWorkerdIsolate type declared in workerd-api.c++.

The source comments include a delightful rant about glibc's approach to detecting processor count (reading /proc/cpuinfo instead of calling sched_getaffinity), explaining why V8System explicitly accepts a thread pool size parameter.

Service Graph Construction

The heart of the boot sequence is Server::run():

src/workerd/server/server.h#L100-L113

This method iterates over config.getServices() and creates a concrete Service implementation for each:

• Worker → WorkerService (line 1882): Compiles JavaScript, manages isolate, handles request dispatch
• Network → NetworkService (line 937): Wraps kj::Network as an HTTP client service
• External → ExternalHttpService or ExternalTcpService: Proxies to a fixed remote address
• Disk → DiskDirectoryService: Serves files from a local directory

src/workerd/server/server.c++#L937-L960

After construction comes the link phase. Each service's link() method is called, resolving inter-service references. For WorkerService, this means connecting service bindings (like env.MY_SERVICE) to actual Service instances, and setting up Durable Object actor namespaces:

src/workerd/server/server.c++#L1934-L1958

flowchart TD
    Config["Config::Reader"] --> |"iterate services"| Create["Create Service instances"]
    Create --> WS["WorkerService<br/>(compiles JS)"]
    Create --> NS["NetworkService<br/>(internet access)"]
    Create --> ES["ExternalHttpService"]
    Create --> DS["DiskDirectoryService"]

    WS --> Link["link() phase"]
    NS --> Link
    ES --> Link
    DS --> Link

    Link --> |"resolve bindings"| Graph["Connected Service Graph"]
    Graph --> Sockets["Bind Sockets"]

The link/unlink pattern deserves attention. Because services can reference each other (a Worker's binding points to another Service), there are potential reference cycles. The unlink() method, called in Server's destructor, breaks these cycles before the services are destroyed. The destructor even verifies that unlinking eliminated all cycles — a leak in this system would be a serious bug.

Socket Binding and the HttpListener Accept Loop

With services constructed and linked, the server binds sockets. For each socket in the config, the server resolves the address (or accepts an override from --socket-addr / --socket-fd), optionally configures TLS, and creates an HttpListener:

src/workerd/server/server.c++#L5171-L5246

The HttpListener::run() method is a coroutine-style accept loop: it calls listener->acceptAuthenticated(), extracts the peer identity (network address for TCP, or PID/UID for Unix sockets), constructs a cf blob with client metadata, creates an HttpConnection, and dispatches to kj::HttpServer::listenHttp().

sequenceDiagram
    participant L as HttpListener
    participant S as Socket
    participant Conn as Connection
    participant Svc as Service

    loop Accept Loop
        L->>S: acceptAuthenticated()
        S-->>L: AuthenticatedStream (stream + peerIdentity)
        L->>L: Build cfBlobJson from peer identity
        L->>Conn: Create Connection(cfBlobJson)
        L->>Conn: listenHttp(stream)
        Conn->>Svc: startRequest(metadata)
        Svc-->>Conn: WorkerInterface
    end

The peer identity extraction is sophisticated. For TLS connections, the code peels off the TLS identity layer to access the underlying network identity. For local Unix socket connections, it extracts Linux credentials (PID, UID) and includes them in the cf blob — the same request.cf object that Workers access in production.

Each accepted connection runs independently in the server's global task set, meaning the accept loop continues even while previous connections are being processed. Connection-level exceptions are logged but don't crash the server.

File Watching and Live Reload

The --watch flag enables live reload, powered by OS-native file watching. The FileWatcher class has two implementations — one for Linux using inotify, and one for macOS/BSD using kqueue:

src/workerd/server/workerd.c++#L174-L250

The Linux implementation watches directories (not individual files) because inotify can handle this efficiently. It tracks which filenames within each directory are of interest, and only triggers on events matching those filenames. The kqueue implementation watches individual file descriptors (duplicated via dup() because closing the original FD would unregister the watch).

When a change is detected, the entire server restarts: all services are torn down and rebuilt from the re-parsed config. In watch mode, config errors are logged but don't terminate the process — the server continues running with the last valid config while waiting for further changes.

The file watcher also monitors the workerd binary itself. If you rebuild workerd and the binary changes on disk, the server restarts with the new binary — a convenience for development.

Tip: In watch mode, config errors are printed to stderr but the server keeps running. This means you can make a typo in your config, see the error immediately, fix it, and the server picks up the correction — all without manually restarting.

With the server now accepting connections, the next question is: what happens when a request actually arrives? In Part 3, we'll trace a request from socket acceptance through WorkerEntrypoint, IoContext, and all the way to JavaScript handler invocation and response generation.