Buf blog

Today we’re announcing public availability of hyperpb, a fully-dynamic Protobuf parser that is 10x faster than dynamicpb, the standard Go solution for dynamic Protobuf. In fact, it’s so efficient that it’s 3x faster than parsing with generated code! It also matches or beats vtprotobuf’s generated code at almost every benchmark, without skimping on correctness.

The first 15 field numbers are special: most runtimes will decode them much faster than the other field numbers. When designing a message type for decoding performance, it’s good to use these field numbers on fields that are almost always present.

Instead of using oneofs, you can now use the new (buf.validate.message).oneof Protovalidate annotation. As long as you're validating your messages with Protovalidate, (buf.validate.message).oneof does exactly what you'd expect, with none of the pain.

We’re announcing a major step toward our vision of a single schema definition in Protobuf that governs your entire data flow — from API contracts to streaming topics to analytical tables. Bufstream now supports Databricks Managed Iceberg Tables in private preview, bringing together Buf's schema-first approach with Databricks' industry-leading data governance and optimization capabilities. This integration represents more than just technical compatibility—it's a fundamental shift toward treating schemas as the foundation of your entire data architecture.

Don’t use 'required' to modify fields—you won’t be able to get rid of it later when you realize it was a bad idea.

buf convert and protoscope are powerful tools for examining and debugging wire format dumps.

Enum values are a poorly designed feature and shouldn’t be used.

Avoid import public and import weak.

We're excited to announce that Bufstream is now available on the AWS Marketplace. Enterprise customers can purchase through their AWS account and accelerate their deployment of Bufstream.

If you're looking for a modern drop-in replacement for Apache Kafka to save costs and complexity, Bufstream is probably your best bet. However, we've got a bigger mission here. Buf wants to bring schema-driven development across your entire stack, from your network APIs, to your streaming data, to your lakehouse, unified behind one schema language that can do it all.

Protobuf’s distributed nature introduces evolution risks that make it hard to fix some types of mistakes. Sometimes the best thing to do is to just let it be.

Enums inherit some unfortunate behaviors from C++. Avoid this problem by using the Buf lint rules ENUM_VALUE_PREFIX and ENUM_ZERO_VALUE_SUFFIX.

Compression is everywhere. This pervasiveness means that wire size tradeoffs matter less than they used to twenty years ago, when Protobuf was designed.

Don’t rename fields. Even though there are a slim number of cases you can get away with it, it’s rarely worth doing, and is a potential source of bugs.

Bufstream — Buf’s drop-in replacement for Apache Kafka® — now supports business-critical workloads with multi-region, active-active clusters on GCP.

At less than $1M/month for 100 GiB/s of writes and 300 GiB/s of reads, Bufstream using Spanner is 25x cheaper than Confluent Cloud, 1.5x cheaper than WarpStream, and 3x cheaper than self-hosted Apache Kafka®.

Today, we’re announcing the 2.0 release of the Connect-ES project, the TypeScript implementation of Connect for Web browsers and Node.js. This release introduces improved support for major frameworks and simplified code generation. Connect-ES 2.0 now uses Protobuf-ES 2.0 APIs to leverage reflection, extension registries, and Protobuf custom options. The 2.0 release is a major version bump and comes with breaking changes. Read on to learn what’s changed and how to migrate to the 2.0 release.

Jepsen's Bufstream report bolsters enterprise use of Buf’s elastic Kafka-compatible streaming platform to enable data quality, enforce governance policies, and cut costs 8x

We’re excited to share that Connect-Swift has officially reached v1.0—its first stable release! With this milestone, the library’s battle-tested APIs will remain stable until its next major release. Projects can rely on Connect-Swift without worrying that future releases will cause breakages or require migrations.

Buf is introducing custom lint and breaking change plugins via the Bufplugin framework. Check it out to see how easy it is to author, test, and consume your own lint and breaking change rules.