Buf's breaking change detection is configurable for a wide range of scenarios, offering rules and thoughtful categories that make it easier to enforce exactly the right policy for your team.
Categories
We categorize breaking rules into four categories: FILE
, PACKAGE
, WIRE_JSON
, and WIRE
. From strictest to most
lenient, they are:
FILE
: Default. Detects changes that move generated code between files, breaking generated source code on a per-file basis. This breaks generated stubs in some languages—for example, it's safe to move code between files in Go but not in Python.PACKAGE
: Detects changes that break generated source code changes on a per-package basis. It detects changes that would break the generated stubs, but only accounting for package-level changes.WIRE_JSON
: Detects changes that break wire (binary) or JSON encoding. Because JSON is ubiquitous, we recommend this as the minimum level.WIRE
: Detects changes that break wire (binary) encoding.
Unlike lint rules, you shouldn't mix and exclude specific breaking change rules, although we do allow it. Instead it's
best to choose one of the four categories. If there's any doubt, choose FILE
. buf breaking
is feedback that your
changes may break your program or others' programs. You always have the option of being less strict later.
See the rules section below for details about individual rules and what categories they're in.
FILE
and PACKAGE
The FILE
and PACKAGE
categories protect compatibility in generated code. For example, deleting an enum or message
often removes the corresponding type in generated code. Any code that refers to that enum or message will then fail to
compile.
As an example imagine you had an Arena
enum and marked ARENA_FOO
as deprecated:
enum Arena {
ARENA_UNSPECIFIED = 0;
ARENA_FOO = 1 [deprecated = true];
ARENA_BAR = 2;
}
Later you remove the field, because it's no longer supported by the server:
enum Arena {
ARENA_UNSPECIFIED = 0;
ARENA_BAR = 2;
}
This change is perfectly wire compatible but all code that referred to ARENA_FOO
will fail to compile:
resp, err := service.Visit(
ctx,
connect.NewRequest(&visitv1.VisitRequest{
Arena: visitv1.Arena_ARENA_FOO, // !!!
}),
)
In some cases this is desirable, but more commonly you're sharing your .proto
files or generated code to clients that
you don't control. You should choose FILE
or PACKAGE
breaking detection if you want to know when you'll break your
client's code.
Though these rules are code generator specific, you should use FILE
to protect all generated languages. FILE
is
absolutely necessary for C++ and Python.
You can use PACKAGE
to protect languages that are less sensitive to types moving between files within the same
package, like Go.
WIRE
and WIRE_JSON
WIRE
and WIRE_JSON
detect breakage of encoded messages. For example:
- Changing an optional field into a required one. Old messages which don't have that field encoded will fail to read in the new definition.
- Reserving deleted types for which reuse in the future could cause wire incompatibilities.
WIRE
and WIRE_JSON
don't check for breakage in generated source code. This is advantageous when:
- You control all of your clients for your service. You're fixing it if it breaks anyway.
- You want your client's build to break instead of getting errors at runtime. (Hopefully your clients are equally happy to immediately stop what they're doing to fix your service.)
- All of your clients are in a monorepo. You want to determine who's depending on deprecated features by a broken build instead of at runtime.
- You're your own client. For example, you're trying to detect issues reading Protobuf encoded messages from older versions of your program that were persisted to disk or other non-volatile storage.
We recommend using WIRE_JSON
instead of WIRE
because Protobuf's JSON encoding breaks when field names change.
- Use
WIRE_JSON
if you're using Connect, gRPC-Gateway, or gRPC JSON. - Use the less strict
WIRE
when you can guarantee only binary encoded messages are decoded.
Rules
The rules are grouped below based on the kind of breaking change they check for: deletions, sameness, and changes to Protobuf file options. Each rule lists the categories that include it.
Deletion checks
ENUM_NO_DELETE
MESSAGE_NO_DELETE
SERVICE_NO_DELETE
Category: FILE
These check that no enums, messages, or services are deleted from a given file. Deleting an enum, message or service deletes the corresponding generated type, which could be referenced in source code. Instead of deleting these types, deprecate them:
enum Foo {
option deprecated = true;
FOO_UNSPECIFIED = 0;
...
}
message Bar {
option deprecated = true;
}
service BazService {
option deprecated = true;
}
PACKAGE_ENUM_NO_DELETE
PACKAGE_MESSAGE_NO_DELETE
PACKAGE_SERVICE_NO_DELETE
Category: PACKAGE
These have the same effect as their non-prefixed counterparts above, except that they verify that types aren't deleted
from a given package, while letting them move between files in the same package. As an example of how this works,
consider the rules ENUM_NO_DELETE
and PACKAGE_ENUM_NO_DELETE
.
ENUM_NO_DELETE
is in theFILE
category, and checks that for each file, no enum is deleted.PACKAGE_NO_DELETE
is in thePACKAGE
category, and checks that for a given package, no enum is deleted. However, enums are allowed to move between files within a package.- Given these definitions, and given that a file doesn't change its package (which is checked by
FILE_SAME_PACKAGE
, also included in every category), it's clear that passingENUM_NO_DELETE
implies passingPACKAGE_ENUM_NO_DELETE
.
FILE_NO_DELETE
Category: FILE
This checks that no file is deleted. Deleting a file results in its generated header file being deleted as well, which could break source code.
PACKAGE_NO_DELETE
Category: PACKAGE
This checks that every package that existed in your previous version still exists in the current schemas. Deleting a package usually deletes other types that break generated code.
ENUM_VALUE_NO_DELETE
FIELD_NO_DELETE
Categories: FILE
, PACKAGE
These check that no enum value or message field is deleted. Deleting an enum value or message field results in the corresponding value or field being deleted from the generated source code, which could be referenced. Instead of deleting these, deprecate them:
enum Foo {
FOO_UNSPECIFIED = 0;
FOO_ONE = 1 [deprecated = true];
}
message Bar {
string one = 1 [deprecated = true];
}
ENUM_VALUE_NO_DELETE_UNLESS_NUMBER_RESERVED
FIELD_NO_DELETE_UNLESS_NUMBER_RESERVED
Categories: WIRE
, WIRE_JSON
These check that no enum value or message field is deleted without reserving the number. Though deleting an enum value or message field isn't directly a wire-breaking change, reusing these numbers in the future is likely to result in either bugs (in the case of enums) or actual wire incompatibilities (in the case of messages, if the type differs). This is also a JSON breaking change for enum values if they are serialized as integers (which is an option). Protobuf provides the ability to reserve numbers to prevent them from being reused in the future. For example:
enum Foo {
// We have deleted FOO_ONE = 1
reserved 1;
FOO_UNSPECIFIED = 0;
}
message Bar {
// We have deleted string one = 1
reserved 1;
}
Note that deprecating a field instead of deleting it has the same effect as reserving the field (as well as reserving the name for JSON), so this is what we recommend.
ENUM_VALUE_NO_DELETE_UNLESS_NAME_RESERVED
FIELD_NO_DELETE_UNLESS_NAME_RESERVED
Category: WIRE_JSON
These check that no enum value or message field is deleted without reserving the name. This is the JSON equivalent of reserving the number—JSON uses field names instead of numbers (optional for enum fields, but allowed). We recommend reserving both the number and the name in most cases. Here's an example:
enum Foo {
// We have deleted FOO_ONE = 1
reserved 1;
reserved "FOO_ONE";
FOO_UNSPECIFIED = 0;
}
message Bar {
// We have deleted string one = 1
reserved 1;
reserved "one";
}
Note that it's usually better to deprecate enum values and message fields than to reserve them in advance.
RPC_NO_DELETE
Categories: FILE
, PACKAGE
This checks that no RPC is deleted from a service. Doing so isn't a wire-breaking change (although client calls fail if a server doesn't implement a given RPC)—however, existing source code may reference a given RPC. Instead of deleting an RPC, deprecate it.
service BazService {
rpc Bat(BatRequest) returns (BatResponse) {
option deprecated = true;
}
}
ONEOF_NO_DELETE
Categories: FILE
, PACKAGE
This checks that no oneof is deleted from a message. Various languages generate types for oneofs, which would no longer be present if deleted.
MESSAGE_NO_REMOVE_STANDARD_DESCRIPTOR_ACCESSOR
Categories: FILE
, PACKAGE
This checks that the
no_standard_descriptor_accessor
message option isn't changed from false
or unset to true
. Changing this option to true
results in the
descriptor()
accessor not being generated in certain languages, which is a generated source code breaking change.
Protobuf has issues with fields that are named "descriptor", with any capitalization and with any number of underscores
before and after "descriptor". Don't name fields this.
Sameness checks
FILE_SAME_PACKAGE
Categories: FILE
, PACKAGE
, WIRE_JSON
, WIRE
This checks that a given file has the same package
value. Changing the package value results in a ton of issues
downstream in various languages, and for the FILE
category, this effectively results in any types declared within that
file being considered deleted.
FILE_SAME_SYNTAX
Categories: FILE
, PACKAGE
This checks that a file doesn't switch between proto2
and proto3
, including going to or from unset (which assumes
proto2
) to set to proto3
. Changing the syntax results in differences in generated code for many languages.
ENUM_VALUE_SAME_NAME
Categories: FILE
, PACKAGE
, WIRE_JSON
This checks that a given enum value has the same name for each enum value number. For example You can't change
FOO_ONE = 1
to FOO_TWO = 1
. Doing so results in potential JSON incompatibilities and broken source code.
Note that for enums with allow_alias
set, this verifies that the set of names in the current definition covers the set
of names in the previous definition. For example, the new definition // new
is compatible with // old
, but // old
isn't compatible with // new
:
// old
enum Foo {
option allow_alias = 1;
FOO_UNSPECIFIED = 0;
FOO_BAR = 1;
FOO_BARR = 1;
}
// new
enum Foo {
option allow_alias = 1;
FOO_UNSPECIFIED = 0;
FOO_BAR = 1;
FOO_BARR = 1;
FOO_BARRR = 1;
}
FIELD_SAME_CTYPE
Categories: FILE
, PACKAGE
This checks that a given field has the same value for the
ctype
option. This
affects the C++ generator. This is a Google-internal field option, so generally you won't have this set, and this rule
should have no effect.
FIELD_SAME_JSTYPE
Categories: FILE
, PACKAGE
This checks that a given field has the same value for the
jstype
option.
This affects JavaScript generated code.
FIELD_SAME_TYPE
Categories: FILE
, PACKAGE
This checks that a field has the same type. Changing the type of a field can affect the type in the generated source code, wire compatibility, and JSON compatibility. Note that technically, it's possible to interchange some scalar types—however, most of these result in generated source code changes anyway, and affect JSON compatibility. Instead of worrying about this, just don't change your field types.
Note that with maps, you may get slightly confusing error messages when changing a field to or from a map and some other
type, denoting that the label of the field changed from repeated
to optional
or the message changed type from
message
to another type. This is because of the way maps are implemented in Protobuf, where every map is actually just
a repeated
field of an implicit message. Buf still properly detects this change and outputs an error, so the pass/fail
decision remains the same.
FIELD_WIRE_COMPATIBLE_TYPE
Categories: WIRE
This rule replaces FIELD_SAME_TYPE
for the WIRE
category. The consequences of this rule are:
- If the type changed between int32, uint32, int64, uint64, and bool, the check passes.
- If the type changed between sint32 and sint64, the check passes.
- If the type changed between fixed32 and sfixed32, the check passes.
- If the type changed between fixed64 and sfixed64, the check passes.
- If the type changed from string to bytes, the check passes.
- If the type changed from bytes to string, the check produces an error about string and bytes compatibility. Per the Protobuf docs, you can change between string and bytes IF the data is valid UTF-8, but because we're only concerned with the API definition and can't know how a user actually uses the field, the check fails.
- If the previous and current types are both enums, Buf checks them to see if (1) the short names are equal, and (2) the previous enum is a subset of the current enum. A subset is defined as having a subset of the name/number enum values. If the previous enum is a subset, the check passes. This covers the case where someone moves where an enum is defined, but still allows values to be added to this enum in the same change, because adding values to an enum isn't a breaking change.
- A link to https://developers.google.com/protocol-buffers/docs/proto3#updating is added to failures produced from
FIELD_WIRE_COMPATIBLE_TYPE
.
FIELD_WIRE_JSON_COMPATIBLE_TYPE
Categories: WIRE_JSON
This rule replaces FIELD_SAME_TYPE
for the WIRE_JSON
category.
JSON allows for some exchanging of types, but due to the way various fields are serialized, the rules are stricter (see
the Protocol Buffer docs). For example, int32, sint32, and
uint32 can be exchanged, but 64-bit numbers have a different representation in JSON. Since sint32 isn't compatible with
int32 or uint32 in WIRE
, we have to limit this to allowing int32 and uint32 to be exchanged in JSON.
The consequences of this rule are:
- If the type changes between int32 and uint32, the check passes.
- If the type changes between int64 and uint64, the check passes.
- If the type changes between fixed32 and sfixed32, the check passes.
- If the type changes between fixed64 and sfixed64, , the check passes.
- If the previous and current types are both enums, Buf checks them to see if (1) the short names are equal, and (2) the previous enum is a subset of the current enum. A subset is defined as having a subset of the name/number enum values. If the previous enum is a subset, the check passes. This covers the case where someone moves where an enum is defined, but still allows values to be added to this enum in the same change, because adding values to an enum isn't a breaking change.
- Links to https://developers.google.com/protocol-buffers/docs/proto3#updating and
https://developers.google.com/protocol-buffers/docs/proto3#json are added to failures produced from
FIELD_WIRE_JSON_COMPATIBLE_TYPE
.
FIELD_SAME_LABEL
Categories: FILE
, PACKAGE
, WIRE_JSON
, WIRE
This checks that no field changes its label. The available labels are optional
, required
, and repeated
. Changing
to or from optional
/required
and repeated
means a generated source code and JSON breaking change. Changing to or
from optional
and repeated
is actually not a wire-breaking change—however, changing to or from optional
and
required
is. Given that it's unadvisable in almost any situation to change your label, and that there is only one
exception, we find it best to forbid this entirely.
FIELD_SAME_ONEOF
Categories: FILE
, PACKAGE
, WIRE_JSON
, WIRE
This checks that no field moves into or out of a oneof or changes the oneof it's a part of. Doing so is almost always a generated source code breaking change. Technically there are exceptions with regard to wire compatibility, but the rules are complex enough that it's safer to never change a field's presence inside or outside a given oneof.
FIELD_SAME_NAME
Categories: FILE
, PACKAGE
, WIRE_JSON
This checks that the field name for a given field number doesn't change. For example, you can't change int64 foo = 1;
to int64 bar = 1;
. This affects generated source code, but also affects JSON compatibility because JSON uses field
names for serialization.
FIELD_SAME_JSON_NAME
Categories: FILE
, PACKAGE
, WIRE_JSON
This checks that the json_name
field option doesn't change, which would break JSON compatibility. Though it's not a
generated source code breaking change in general, some Protobuf plugins may generate code based on this option, and
having this as part of the FILE
and PACKAGE
groups also fulfills that the FILE
and PACKAGE
categories are
supersets of the WIRE_JSON
category.
RESERVED_ENUM_NO_DELETE
RESERVED_MESSAGE_NO_DELETE
Categories: FILE
, PACKAGE
, WIRE_JSON
, WIRE
These check that no reserved number range or reserved name is deleted from any enum or message. Deleting a reserved value means that future versions of your Protobuf schema could use names or numbers in those ranges, and if the ranges were reserved, it was probably because an enum value or field was deleted.
Note that moving from reserved 3 to 6;
to reserved 2 to 8;
, for example, would technically be fine—however, the
check still fails in this case. Making sure all ranges are covered is truly a pain—we have no other excuse. For now,
just do reserved 3 to 6, 2, 7 to 8;
to pass breaking change detection.
EXTENSION_MESSAGE_NO_DELETE
Categories: FILE
, PACKAGE
This checks that no extension range is deleted from any message. Though this won't have any effect on your generated source code, deleting an extension range can result in compile errors for downstream Protobuf schemas, and is generally not recommended. Note that extensions are a proto2-only construct, so this has no effect for proto3.
MESSAGE_SAME_MESSAGE_SET_WIRE_FORMAT
Categories: FILE
, PACKAGE
, WIRE_JSON
, WIRE
This checks that the
[message_set_wire_format``](https://github.com/protocolbuffers/protobuf/blob/v24.2/src/google/protobuf/descriptor.proto#L528) message option is the same. Since this is a
proto1` construct, we congratulate you if you are using this for any
current Protobuf schema, as you are a champion of maintaining backwards compatible APIs over many years. Instead of
failing breaking change detection, perhaps you should get an award. 🏆
RPC_SAME_REQUEST_TYPE
RPC_SAME_RESPONSE_TYPE
RPC_SAME_CLIENT_STREAMING
RPC_SAME_SERVER_STREAMING
Categories: FILE
, PACKAGE
, WIRE_JSON
, WIRE
These check that RPC signatures don't change. Doing so would break both generated source code and over-the-wire RPC calls.
RPC_SAME_IDEMPOTENCY_LEVEL
Categories: FILE
, PACKAGE
, WIRE_JSON
, WIRE
This checks that the
idempotency_level
RPC option doesn't change. Doing so can result in different HTTP verbs being used.
Protobuf option checks
FILE_SAME_CC_ENABLE_ARENAS
FILE_SAME_CC_GENERIC_SERVICES
FILE_SAME_CSHARP_NAMESPACE
FILE_SAME_GO_PACKAGE
FILE_SAME_JAVA_GENERIC_SERVICES
FILE_SAME_JAVA_MULTIPLE_FILES
FILE_SAME_JAVA_OUTER_CLASSNAME
FILE_SAME_JAVA_PACKAGE
FILE_SAME_JAVA_STRING_CHECK_UTF8
FILE_SAME_OBJC_CLASS_PREFIX
FILE_SAME_OPTIMIZE_FOR
FILE_SAME_PHP_CLASS_PREFIX
FILE_SAME_PHP_GENERIC_SERVICES
FILE_SAME_PHP_METADATA_NAMESPACE
FILE_SAME_PHP_NAMESPACE
FILE_SAME_PY_GENERIC_SERVICES
FILE_SAME_RUBY_PACKAGE
FILE_SAME_SWIFT_PREFIX
Categories: FILE
, PACKAGE
These check that each of these file options don't change values between versions of your Protobuf schema. Changing any of these values results in differences in your generated source code. You may not use any or all of these languages in your development—if you don't, none of these rules should ever break.
What we left out
We think the rules above represent a complete view of what is and isn't compatible with respect to Protobuf schemas. We
cover every available field within a
FileDescriptorSet
as of Protobuf v3.11.4, as well as additional fields as added. If we've missed something, let us know.
We did leave out custom options, though. There's no way for us to know the effects of your custom options, so we can't reliably determine their compatibility.