Read OSS

Serving the World: JSON-RPC, Engine API, and API Architecture

Intermediate

Prerequisites

  • Articles 1-2: Architecture and Boot Process
  • JSON-RPC basics
  • Understanding of Ethereum's execution/consensus layer split

Serving the World: JSON-RPC, Engine API, and API Architecture

Every dApp, wallet, block explorer, and development tool interacts with Geth through its JSON-RPC interface. Geth serves thousands of RPC methods across multiple namespaces, over HTTP, WebSocket, and IPC transports — plus a separate authenticated endpoint for the consensus layer. This article covers the full RPC architecture: the reflection-based registration framework, the transport layer, the Backend abstraction that decouples API handlers from internals, namespace registration, and the Engine API.

The RPC Framework: Reflection-Based Registration

Geth's RPC framework, implemented in the rpc/ package, uses Go reflection to automatically map struct methods to JSON-RPC endpoints. The service.go file contains the registration machinery:

type serviceRegistry struct {
    mu       sync.Mutex
    services map[string]service
}

type service struct {
    name          string
    callbacks     map[string]*callback
    subscriptions map[string]*callback
}

type callback struct {
    fn          reflect.Value
    rcvr        reflect.Value
    argTypes    []reflect.Type
    hasCtx      bool
    errPos      int
    isSubscribe bool
}

When you register a struct with server.RegisterName("eth", myService), the framework uses reflection to discover all exported methods, determine their parameter types, and create callable wrappers. The naming convention is automatic: a method GetBalance on a service registered under namespace "eth" becomes eth_getBalance (camelCase with lowercase first letter).

flowchart LR
    REG["RegisterName('eth', EthereumAPI{})"] --> REFLECT["Reflect on EthereumAPI"]
    REFLECT --> DISCOVER["Find exported methods"]
    DISCOVER --> MAP["GetBalance → eth_getBalance<br/>BlockNumber → eth_blockNumber<br/>Subscribe → eth_subscribe"]
    MAP --> STORE["Store in serviceRegistry"]

    REQ["JSON-RPC Request<br/>{method: 'eth_getBalance'}"] --> LOOKUP["Lookup service + method"]
    LOOKUP --> INVOKE["Reflect.Call(args...)"]
    INVOKE --> RESP["JSON-RPC Response"]

Methods that accept a context.Context as their first parameter get the request context injected automatically. Methods returning (Subscription, error) are treated as subscription endpoints. The framework handles all the JSON marshaling and error wrapping.

Tip: When adding a new RPC method, you just need to add an exported method to the right API struct. The framework discovers it automatically — no registration boilerplate, no code generation. The struct name becomes the namespace prefix and the method name becomes the RPC method suffix.

Transport Layer: HTTP, WebSocket, IPC

The Node creates and manages multiple transport endpoints during construction, as we saw in Part 2. The RPC framework supports four transports:

flowchart TD
    subgraph "Unauthenticated"
        HTTP["HTTP Server<br/>:8545<br/>Stateless requests"]
        WS["WebSocket Server<br/>:8546<br/>Subscriptions"]
        IPC["IPC Server<br/>Unix socket<br/>Local access only"]
    end
    subgraph "Authenticated (JWT)"
        HTTP_AUTH["Auth HTTP<br/>:8551<br/>Engine API"]
        WS_AUTH["Auth WebSocket<br/>:8551<br/>Engine API subscriptions"]
    end
    subgraph "In-Process"
        INPROC["InProc Handler<br/>Direct function calls<br/>No serialization"]
    end
    HTTP --> RPC_SERVER["rpc.Server<br/>serviceRegistry"]
    WS --> RPC_SERVER
    IPC --> RPC_SERVER
    HTTP_AUTH --> RPC_SERVER
    WS_AUTH --> RPC_SERVER
    INPROC --> RPC_SERVER

The authenticated endpoints (httpAuth, wsAuth) are specifically for the Engine API. They use JWT authentication — the consensus layer and execution layer share a secret, and every request must include a valid JWT token. The Node's startRPC() method sets up all transports, and authenticated endpoints are only created when there are API methods that require authentication.

The in-process handler (inprocHandler) deserves special mention. It's used by node.Attach() to create an RPC client that calls methods directly without any network serialization. This is how the geth console command works — it connects to its own node via an in-process client.

The Backend Interface and EthAPIBackend

The ethapi.Backend interface is the massive abstraction that decouples all RPC handlers from blockchain internals. It has over 40 methods spanning chain access, transaction pool operations, configuration, and log filtering:

type Backend interface {
    // General Ethereum API
    SyncProgress(ctx context.Context) ethereum.SyncProgress
    SuggestGasTipCap(ctx context.Context) (*big.Int, error)
    ChainDb() ethdb.Database
    AccountManager() *accounts.Manager
    RPCGasCap() uint64
    RPCEVMTimeout() time.Duration

    // Blockchain API
    HeaderByNumber(ctx context.Context, number rpc.BlockNumber) (*types.Header, error)
    BlockByNumber(ctx context.Context, number rpc.BlockNumber) (*types.Block, error)
    StateAndHeaderByNumber(ctx context.Context, number rpc.BlockNumber) (*state.StateDB, *types.Header, error)

    // Transaction pool API
    SendTx(ctx context.Context, signedTx *types.Transaction) error
    GetPoolTransaction(txHash common.Hash) *types.Transaction
    // ... many more
}
classDiagram
    class Backend {
        <<interface>>
        +SyncProgress() SyncProgress
        +HeaderByNumber(number) Header
        +BlockByNumber(number) Block
        +StateAndHeaderByNumber(number) StateDB, Header
        +SendTx(tx) error
        +GetPoolTransaction(hash) Transaction
        +ChainConfig() ChainConfig
        +Engine() Engine
    }
    class EthAPIBackend {
        -eth *Ethereum
        -gpo *OracleBackend
        +HeaderByNumber() Header
        +SendTx() error
    }
    class EthereumAPI {
        -b Backend
        +GetBalance(addr, block)
        +GetTransactionByHash(hash)
    }
    EthAPIBackend ..|> Backend
    EthereumAPI --> Backend : uses

The EthAPIBackend implementation (in eth/api_backend.go) simply delegates each method to the appropriate component of the Ethereum struct — eth.blockchain for chain queries, eth.txPool for transaction pool operations, eth.engine for consensus engine access.

This indirection is intentional. It means RPC handlers never directly touch the blockchain or transaction pool — they always go through the Backend interface. This makes the handlers testable with mock backends and theoretically allows different node types (full, light) to share the same handler code.

API Namespaces and Registration

The Ethereum service registers its API namespaces via the APIs() method:

func (s *Ethereum) APIs() []rpc.API {
    apis := ethapi.GetAPIs(s.APIBackend)  // eth, txpool, debug namespaces
    return append(apis, []rpc.API{
        {Namespace: "miner",  Service: NewMinerAPI(s)},
        {Namespace: "eth",    Service: downloader.NewDownloaderAPI(...)},
        {Namespace: "admin",  Service: NewAdminAPI(s)},
        {Namespace: "debug",  Service: NewDebugAPI(s)},
        {Namespace: "net",    Service: s.netRPCService},
    }...)
}

The GetAPIs() function in ethapi creates the core API structs:

func GetAPIs(apiBackend Backend) []rpc.API {
    return []rpc.API{
        {Namespace: "eth",    Service: NewEthereumAPI(apiBackend)},
        {Namespace: "eth",    Service: NewBlockChainAPI(apiBackend)},
        {Namespace: "eth",    Service: NewTransactionAPI(apiBackend, nonceLock)},
        {Namespace: "txpool", Service: NewTxPoolAPI(apiBackend)},
        // ...
    }
}

Multiple services can register under the same namespace — all eth_* methods come from three different structs (EthereumAPI, BlockChainAPI, TransactionAPI), but they all appear under the eth namespace to callers.

flowchart TD
    subgraph "eth namespace"
        EA["EthereumAPI<br/>eth_chainId, eth_syncing,<br/>eth_gasPrice"]
        BCA["BlockChainAPI<br/>eth_getBalance, eth_getCode,<br/>eth_call, eth_estimateGas"]
        TXA["TransactionAPI<br/>eth_sendRawTransaction,<br/>eth_getTransactionByHash"]
    end
    subgraph "Other namespaces"
        MINER["MinerAPI<br/>miner_setGasPrice,<br/>miner_setExtra"]
        ADMIN["AdminAPI<br/>admin_peers,<br/>admin_nodeInfo"]
        DEBUG["DebugAPI<br/>debug_traceTransaction,<br/>debug_dumpBlock"]
        NET["NetAPI<br/>net_version,<br/>net_peerCount"]
    end

Engine API: Consensus Layer Drives Execution

The most critical RPC interface in post-Merge Ethereum isn't the public eth_* API — it's the Engine API. This authenticated JSON-RPC interface is how the consensus layer client tells Geth what to do.

The Engine API has three core method families:

  1. engine_forkchoiceUpdatedV* — The CL tells the EL which block is the head, which is safe, and which is finalized. Optionally includes a payload request to start building a new block.

  2. engine_newPayloadV* — The CL sends a new execution payload (block) for validation. Geth executes it and returns whether it's valid.

  3. engine_getPayloadV* — The CL retrieves a previously requested payload that the EL has been building.

sequenceDiagram
    participant CL as Consensus Layer
    participant Engine as Engine API (authenticated)
    participant Miner as Miner
    participant BC as BlockChain

    CL->>Engine: engine_forkchoiceUpdatedV3<br/>(headHash, safeHash, finalHash, payloadAttrs)
    Engine->>BC: Update fork choice head
    Engine->>Miner: BuildPayload(payloadAttrs)
    Miner-->>Engine: payloadId

    Note over Miner: Miner builds block in background...

    CL->>Engine: engine_getPayloadV4(payloadId)
    Engine->>Miner: GetPayload(payloadId)
    Miner-->>Engine: ExecutionPayload + BlobsBundle
    Engine-->>CL: Return payload

    Note over CL: CL proposes block to network

    CL->>Engine: engine_newPayloadV4(payload)
    Engine->>BC: InsertBlockWithoutSealVerification
    BC-->>Engine: valid/invalid/syncing
    Engine-->>CL: PayloadStatus

The Miner struct, despite its legacy name from the proof-of-work era, is now a payload builder. When the CL requests a payload via forkchoiceUpdated, the Miner starts assembling a block — selecting transactions from the pool, executing them, computing the state root, and preparing the execution payload. The CL can then retrieve it with getPayload.

type Miner struct {
    confMu      sync.RWMutex
    config      *Config
    chainConfig *params.ChainConfig
    engine      consensus.Engine
    txpool      *txpool.TxPool
    prio        []common.Address  // Prioritized senders
    chain       *core.BlockChain
    pending     *pending
}

The Engine API endpoints are registered as authenticated APIs — they only appear on the JWT-protected transport. The Node's getAPIs() method separates authenticated from unauthenticated APIs, ensuring Engine API methods can never be called on the public HTTP/WS endpoints.

Tip: The Engine API's catalyst package (in eth/catalyst/) contains the actual method implementations. If you need to understand how the CL-EL interaction works in detail, start with catalyst/api.go — it contains ForkchoiceUpdatedV*, NewPayloadV*, and GetPayloadV* implementations.

With the RPC and API layer covered, we've walked through every major subsystem of Geth — from boot to block execution to storage to networking to external APIs. The final article brings it all together with practical guidance on building, testing, and contributing to the project.