Buf's goal is to shift API development toward a schema-driven paradigm and thus pave the way for a future in which APIs are defined in a way that service owners and clients can depend on.
Defining APIs using an IDL provides a number of benefits over simply exposing REST/JSON services, and today, Protobuf is the most stable, widely adopted IDL in the industry. But as things stand today, using Protobuf is much more difficult than using JSON as your data transfer format.
Buf is building tooling to make Protobuf reliable and user friendly for service owners and clients, while keeping it the obvious choice on the technical merits. Your organization should not have to reinvent the wheel to create, maintain, and consume Protobuf APIs efficiently and effectively. We'll handle your Protobuf management strategy for you, so you can focus on what matters.
The problems we aim to solve
Traditionally, adopting Protobuf presents a number of challenges across the API lifecycle. These are the problems we aim to solve:
API designs are often inconsistent: Writing maintainable, consistent Protobuf APIs isn't as widely understood as writing maintainable REST/JSON-based APIs. With no standards enforcement, inconsistency can arise across an organization's Protobuf APIs, and design decisions can inadvertently affect your API's future iterability.
npm, Rust without
cargo, Go without modules, and all of the other programming language dependency managers we've all grown so accustomed to.
Forwards and backwards compatibility is not enforced: While forwards and backwards compatibility is a promise of Protobuf, actually maintaining backwards-compatible Protobuf APIs isn't widely practiced, and is hard to enforce.
Stub distribution is a difficult, unsolved process: Organizations have to choose to either centralize their
protocworkflow and distribute generated code, or require all service clients to run
protocindependently. Because there is a steep learning curve to using
protoc(and the associated
protocplugins) in a reliable manner, organizations often struggle with distributing their Protobuf files and stubs. This creates substantial overhead, and often requires a dedicated team to manage the process. Even when using a build system like Bazel, exposing APIs to external customers remains problematic.
The tooling ecosystem is limited: Many user-friendly tools exist for REST/JSON APIs today. On the other hand, mock server generation, fuzz testing, documentation, and other daily API concerns are not widely standardized or user friendly for Protobuf APIs. As a result, teams regularly reinvent the wheel and build custom tooling to replicate the JSON ecosystem.
Buf is building a modern Protobuf ecosystem
Our tools address many of the problems above, ultimately allowing you to redirect much of your time and energy from managing Protobuf files to implementing your core features and infrastructure.
buf CLI enables you to create consistent Protobuf APIs that preserve compatibility and comply with best practices.
The tool is currently available on an open-source basis. The
buf CLI incorporates these components to help you create
consistent Protobuf APIs:
- A new developed high-performance Protobuf compiler.
- A linter that enforces good API design choices and structure.
- A breaking change detector that enforces compatibility at the source code or wire level.
- A generator that invokes your
protocplugins based on a configurable template.
The Buf Schema Registry (BSR)
The Buf Schema Registry (BSR) is a hosted SaaS platform that serves as your organization’s source
of truth for your Protobuf APIs. The BSR enables you to centrally maintain compatibility and manage dependencies, while
enabling your clients to consume APIs reliably and efficiently. Similar to
pip for Python, or
cargo for Rust, the BSR finally brings dependency management to your Protobuf APIs.
Where to go from here
See the installation page to install the
Next, we recommend completing the tour. The tour provides an overview of most of the existing functionality of Buf and takes approximately 20 minutes to complete.
After completing the tour, check out the remainder of the documentation for your specific areas of interest. We've aimed to provide as much documentation as we can for the various components of Buf to give you a full understanding of Buf's surface area.
Finally, follow the project on GitHub, and contact us if you'd like to get involved.