Breaking change detection – Overview
This document provides an overview of Buf's breaking change detection. At every step of the development process, starting from developer IDEs and culminating in the Buf Schema Registry's server-wide enforcement, breaking change detection ensures that your organization can evolve Protobuf schemas quickly and safely.
Protobuf has many ways to evolve schemas without breaking existing code. Many of Buf's breaking change rules are designed to maximize your evolution options to do exactly that. However, sometimes it's a better choice to make a breaking change rather than go to the extra effort of backwards compatibility. If you have few clients and can easily update and deploy them, it may be perfectly okay to break your schemas. And if you have a public API or too many clients to easily update, you should probably avoid breaking them.
Whichever path you choose, Buf's breaking change detection allows you to make informed decisions, while removing the need for constant vigilance during code review. It reliably and mechanically identifies breaking changes so you and your team can focus on making an informed decision about whether to allow them or not.
Key concepts
Buf's breaking change detection evaluates your schemas' compatibility at three phases of development:
- During development:
You can spot-check in your local environment by running
buf breaking
. - In code review: You can integrate with your CI/CD workflows (like GitHub Actions) to ensure that breaking changes get flagged directly in your review flow.
- When shipping to the Buf Schema Registry (BSR): This makes them available to other teams and downstream systems like Kafka. The BSR lets your organization enforce policy checks that prevent unintended breaking changes from being committed to the BSR. Instead, they go to a review flow so that the repository owners can review the changes and approve or reject them before they enter your production environment.
buf breaking
runs a set of compatibility checks comparing the current version of your Protobuf schema to a past version.
The past version can be any type of input that the Buf CLI accepts, such as a BSR module, a GitHub repo, or a Buf image.
The checker has a built-in set of rules and categories and can also accept rules and categories via Buf plugins. They can be used alongside or in place of Buf's defaults.
Rules and categories
The checker's rules are split up into logical categories depending on the nature of the breaking changes, so choosing a strictness level is straightforward. Rules can also be selected individually to more closely match your organization's policies. See the rules and categories page for detailed information.
The configuration categories, from strictest to most lenient, are:
FILE
: Default. Detects generated source code breakage on a per-file basisPACKAGE
: Detects generated source code breakage on a per-package basisWIRE_JSON
: Detects breakage to wire (binary) or JSON encodingWIRE
: Detects breakage to wire (binary) encoding
Changes that pass breaking change detection under a stricter policy also pass with all less-strict policies.
For example, passing the FILE
rules means you will pass the PACKAGE
rules.
Defaults and configuration
You configure breaking change detection in the buf.yaml
configuration file, which is placed at the root of the Protobuf source files it defines.
If the input doesn't contain a buf.yaml
file, the Buf CLI operates as if there's a buf.yaml
file with these default values:
You can skip the Buf checker's built-in rules and categories entirely by omitting them and listing categories or rules provided by Buf plugins instead.
If any configured Buf plugins have rules where default
is set to true
, those rules are automatically checked if none of the plugin's rules or categories are listed in the breaking
sections of buf.yaml
.
Below is an example of each buf breaking
configuration option.
For more information on specific options and Buf's rules and categories, see the buf.yaml
reference and the rules and categories page.
version: v2
breaking:
use:
- FILE
except:
- RPC_NO_DELETE
ignore:
- foo/bar.proto
ignore_only:
FIELD_SAME_JSON_NAME:
- baz
ignore_unstable_packages: true
plugins:
- plugin: buf-plugin-foo
Using Buf plugins
To use rules and categories from a Buf plugin, you first need to install the plugin locally (ideally on your $PATH
), then configure it in the buf.yaml
file.
As with Buf's built-in rules and categories, you can specify combinations of rules and categories using use
, except
, ignore
, and ignore_only
.
For example, if you want to use the CATEGORY_ID_FROM_PLUGIN
category from buf-plugin-foo
but don't want to check against its RULE_ID_FROM_PLUGIN
rule, you'd define it like this:
version: v2
breaking:
use:
- FILE
- CATEGORY_ID_FROM_PLUGIN # Applies all rules contained in this buf-plugin-foo category
except:
- RULE_ID_FROM_PLUGIN # Omits this buf-plugin-foo rule
plugins:
- plugin: buf-plugin-foo
You can get a list of all of your workspace's rules and categories (including those from configured Buf plugins) by running buf config ls-breaking-rules
.
Integration with CI/CD workflows
Because buf breaking
is part of a CLI, you can easily integrate it into CI/CD workflows.
For instructions, see the General CI/CD setup and GitHub Action pages.
Usage examples
The most basic usage of buf breaking
is to check your local files against your local Git repository.
However, both the input and the schema you're checking against can be specified in multiple ways, so you can use buf breaking
in many types of workflows.
Git and GitHub
Note
Check out Buf's dedicated GitHub Action to seamlessly add breaking change detection into your CI/CD pipeline.
Local repositories
You can directly compare against the .proto
files at the head of a git
branch or a git
tag.
See the inputs documentation for details on git
branches and git
tags.
It's especially useful for iterating on your schema locally.
Remote repositories
Note that many CI services like Travis CI don't do a full clone of your repo, instead cloning a certain number of commits (typically around 50) on the specific branch that's being tested.
In this scenario, other branches aren't present in your clone within CI, so the local example above doesn't work.
You can fix this by giving the remote path directly to the Buf CLI so it can clone the repo itself, for example against https://github.com/foo/bar.git
:
It only clones the single commit at the HEAD
of the branch, so even for large repositories, this should be quick.
For remote locations that require authentication, see HTTPS Authentication and SSH Authentication for details.
Using tags
You can compare against a git
tag, for example v1.0.0
:
Within subdirectories
You can compare against a subdirectory in your git repository.
For example, if your buf.yaml
is stored in the subdirectory proto
:
Buf Schema Registry repository
If you're already using the BSR in your Protobuf workflow, then comparing your local module against the latest version stored in the BSR is a common use case.
Archives (.tar
and .zip
)
You can compare against tarballs and zip archives of your .proto
files as well.
This is especially useful for GitHub where they can be retrieved for any commit or branch.
This example assumes your repo is github.com/foo/bar
and COMMIT
is a variable storing the commit to compare against:
$ buf breaking --against "https://github.com/foo/bar/archive/${COMMIT}.tar.gz#strip_components=1"
$ buf breaking --against "https://github.com/foo/bar/archive/${COMMIT}.zip#strip_components=1"
Output as JSON
You can also output the errors as JSON.
The output defaults to a single-line comma-separated message, but can be piped to other tools for formatting.
For example, you can send the output to jq
:
$ buf breaking --against '.git#branch=main' --error-format=json | jq .
{
"path":"acme/pet/v1/pet.proto",
"start_line":18,
"start_column":3,
"end_line":18,
"end_column":9,
"type":"FIELD_SAME_TYPE",
"message":"Field \"1\" on message \"Pet\" changed type from \"enum\" to \"string\"."
}
Limit to specific files
By default, the Buf CLI builds all files under the buf.yaml
configuration file.
Instead, you can manually specify the file or directory paths to check.
This is an advanced feature intended to be used for editor or Bazel integration.
In general, it's better to let the Buf CLI discover all files under management and handle this for you, especially when using the FILE
category.
Breaking change detection is limited to the given files if the --path
flag is specified, as in this command:
Advanced use cases
Due to the nature of inputs, buf breaking
happily compares just about anything.
You may have an advanced use case, so this example demonstrates its ability to compare a git
repository against a remote archive.
Copy/paste this into your terminal:
$ buf breaking \
"https://github.com/googleapis/googleapis.git" \
--against "https://github.com/googleapis/googleapis/archive/b89f7fa5e7cc64e9e38a59c97654616ad7b5932d.tar.gz#strip_components=1" \
--config '{"version":"v2","breaking":{"use":["PACKAGE"]}}'
google/cloud/asset/v1/assets.proto:27:1:File option "cc_enable_arenas" changed from "false" to "true".
To explicitly target a branch , you can adapt the command to include branch=<branch_name>
in the git
input:
$ buf breaking \
"https://github.com/googleapis/googleapis.git#branch=main" \
--against "https://github.com/googleapis/googleapis/archive/b89f7fa5e7cc64e9e38a59c97654616ad7b5932d.tar.gz#strip_components=1" \
--config '{"version":"v2","breaking":{"use":["PACKAGE"]}}'
google/cloud/asset/v1/assets.proto:27:1:File option "cc_enable_arenas" changed from "false" to "true".
Related docs
- Try out breaking change detection with the tutorial
- Get detailed explanations of the breaking change rules and categories
- Learn about the breaking change policy check and the review flow
- Browse the buf.yaml configuration file reference and
buf breaking
command reference - See more about the types of inputs that the Buf CLI accepts