Protolock is a widely-used Protobuf tool that primarily concentrates on breaking change detection. It deserves a lot of praise - in the OSS world, it largely pioneered the breaking change detection effort, and has been very well maintained. We can't heap enough praise on this effort, it's helped the Protobuf ecosystem move forward in a big way.
In this document, we'll discuss the pros and cons of Protolock vs Buf's breaking change detector, as well as Buf equivalent commands and migration.
- Protolock has a plugin interface allowing you to create external binaries that Protolock then calls itself to verify rules. The equivalent way to do this in Buf is to ask us to add a lint or breaking checker, which we're more than happy to do in most scenarios. It's our feeling that calling out to external binaries for individual lint rules leads to issues with tool distribution and management, but this use case may be something you want, and Buf does not support it.
- Protolock uses a third-party Protobuf parser that is not tested to cover every edge case
of the Protobuf grammar, and has had such issues in the past. Additionally, this parser
does not verify that what it is parsing is actually valid Protobuf, meaning that Protolock
can both have breakages for valid Protobuf file, and happily parse Protobuf files that are
not valid. Instead, Buf lets you use either the internal compiler that is tested to
cover every edge case and will only parse valid files, or use
bufinput. See our compiler discussion for more details.
- Protolock uses a custom structure, represented in JSON, to store your Protobuf schema state. This structure is populated based on the results of the third-party Protobuf parser, meaning that file data can be corrupted for an invalid parse. This structure also does not cover all known elements of a Protobuf schema, especially Protobuf options that can have an effect on your API compatibility. Instead, Buf uses FileDescriptorSets, extended to Images, which are the primitive of the Protobuf ecosystem, and have been stable for over a decade. Buf's equivalent to lock files are just serialized FileDescriptorSets.
- Protolock only enforces 8 rules related to API compatibility in strict mode, and 5
with strict mode disabled. Buf enforces 46 rules related to API compatibility
in it's strictest mode (
FILE), and 15 rules related to wire-only compatibility in it's weakest mode (
WIRE). We believe that the additional rules that Buf enforces are critical to API compatibility.
- Breaking change rules are not a binary proposition - there are different kinds of breaking changes that you may care about. Buf provides four categories of breaking change rules to select - per-file generated stub breaking changes, per-package generated stub breaking changes, wire breaking changes, and wire + JSON breaking changes. Within these categories, you can go further and enable or disable individual checkers through configuration.
- Buf provides
file:line:column:messagereferences for breaking change violations, letting you know where a violation occurred, including potentially integrating this into your editor in the future. These reference your current Protobuf schema, including if types move across files between versions of your Protobuf schema. The error output can be outputted as text or JSON, with other formats coming in the future. Protolock prints out unreferenced messages.
- Protolock relies on
proto.lockfiles as the only way to store the representation of your previous Protobuf schema, and these files are represented by a custom structure. Buf allows you to use lock files through
buf image build, but also allows other methods to store and retrieve your previous Protobuf schema, including:
- Cloning the head of a branch of a git repository, either local or remote, and compiling on the fly.
- Reading a tarball, either local or remote, gzipped or uncompressed, and compiling on the fly.
- Reading a "lock file", represented as an Image, from either a local location or a remote http/https location.
- Both Protolock and Buf run file discovery for your Protobuf files, however Buf allows you to skip file discovery and specify your files manually for use cases that require this, such as Bazel.
- Since Buf can process FileDescriptorSets as input, Buf provides a
protoc plugin to allow you to use Buf's
breaking change detection functionality with your current
See the breaking configuration documentation for more details.
Note that configuration can be provided via the flag
--input-config on the command line
if you do not want to have a configuration file.
Protolock rules to Buf configured checkers
See the breaking checkers documentation for an overview of all available breaking checkers.
While we recommend using one of Buf's preset breaking categories, the below configuration selects the same checkers as the rules enforced by Protolock:
breaking: use: - ENUM_VALUE_NO_DELETE_UNLESS_NAME_RESERVED - ENUM_VALUE_NO_DELETE_UNLESS_NUMBER_RESERVED - FIELD_NO_DELETE_UNLESS_NAME_RESERVED - FIELD_NO_DELETE_UNLESS_NUMBER_RESERVED - FIELD_SAME_NAME - FIELD_SAME_TYPE - RESERVED_ENUM_NO_DELETE - RESERVED_MESSAGE_NO_DELETE - RPC_NO_DELETE - RPC_SAME_CLIENT_STREAMING - RPC_SAME_REQUEST_TYPE - RPC_SAME_RESPONSE_TYPE - RPC_SAME_SERVER_STREAMING
This roughly corresponds to the
WIRE_JSON group, with some checkers added and
some deleted. The below configuration is equivalent to the above configuration:
breaking: use: - WIRE_JSON - RPC_NO_DELETE except: - ENUM_VALUE_SAME_NAME - FIELD_SAME_JSON_NAME - FIELD_SAME_LABEL - FIELD_SAME_ONEOF - MESSAGE_SAME_MESSAGE_SET_WIRE_FORMAT - RPC_SAME_IDEMPOTENCY_LEVEL
Protolock flags that are Buf configuration options
The Protolock flag
--ignore can be handled by the
The Protolock flag
--protoroot is effectively handled by your build
The Protolock flag
--lockdir is handled by your against input, as Buf can take multiple types
of input to compare against. The equivalent in Buf would be to specify your image location with
There are multiple methods to compare versions in Buf, see the breaking usage documentation for more details.
This section assumes you are using stored Image files as your method of comparing versions of your Protobuf schema.
$ buf image build -o lock.bin
This writes a binary Image of your current Protobuf schema. If you prefer this to be stored as JSON,
as Protolock does, instead write to a file with a
.json extension, such as
buf image build -o lock.json. Note that by default,
buf image build include source code info,
which makes the resulting file significantly larger. If this is not a concern, we recommend
keeping the source code info for usage with other parts of Buf, but if you are only using Buf
for breaking change detection, you can safely suppress source code info with the
$ buf check breaking --against-input lock.bin
This checks for breaking changes against the
lock.bin Image file.
buf check breaking --against-input lock.json if you wrote a JSON file.
$ buf check breaking --against-input lock.bin && buf image build -o lock.bin
docker pull bufbuild/buf docker run --volume "$(pwd)/workspace" --workdir "/workspace" bufbuild/buf check lint