protocolbuffers/protobuf
10 articles
Prerequisites
- ›Basic understanding of what Protocol Buffers are and their purpose as a serialization format
Navigating the Protobuf Monorepo: Architecture and Directory Guide
A high-level map of the protobuf monorepo covering the dual-world architecture, directory structure, compilation pipeline, language runtimes, and build system.
Protocol Buffers Source Code: A Map of the Territory
A high-level orientation to the protobuf monorepo covering its three-layer architecture, dual-runtime strategy, directory structure, and the protoc entry point.
Inside protoc: From .proto File to Generated Code
A deep dive into the protoc compiler's internals: lexing, parsing, descriptor building, and the code generator/plugin dispatch system.
Inside protoc: From .proto Files to Type-Safe Code
A deep dive into the protoc compilation pipeline: tokenizer, parser, DescriptorPool, CodeGenerator interface, and the plugin protocol.
The Descriptor System and Message Hierarchy: Protobuf's Type Universe
Exploring protobuf's central type system: the Descriptor hierarchy, DescriptorPool internals, the Message class hierarchy, DynamicMessage, Reflection, and the Editions system.
The C++ Runtime: Messages, Reflection, and Arena Allocation
An in-depth examination of the C++ runtime: MessageLite/Message hierarchy, PROTOBUF_CUSTOM_VTABLE, DynamicMessage, and the three-layer Arena system.
High-Performance Serialization: Arena Allocation and Tail-Call Table Parsing
Diving into protobuf's performance-critical internals: zero-copy I/O, Arena allocation, and the TcTable tail-call parsing system.
μpb: The Lightweight C Runtime Powering Python, Ruby, and PHP
Examining μpb's architecture: MiniTable compact schemas, arena fusing, the upb_Message representation, and language bindings for Python, Ruby, and PHP.
The Rust Integration and C++ Code Generator: A Study in Code Generation Patterns
Exploring code generation patterns: Rust's dual-kernel proxy-based API, the C++ generator's strategy pattern for field types, and HPB as a new C++ API on upb.
Conformance Testing and CI: Keeping 10+ Languages in Sync
How protobuf maintains correctness across 10+ language implementations through conformance testing, failure tracking, and CI infrastructure.