Bazel

Buf provides official support for the Bazel build tool with rules_buf, which enables you to:

Lint Protobuf sources using the buf_lint_test rule.
Perform breaking change detection for Protobuf Inputs using the buf_breaking_test rule.
Use the Gazelle extension to generate Bazel rules.

Setup

To get started, add a series of imports to your Bazel WORKSPACE,

WORKSPACE

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_buf",
    sha256 = "523a4e06f0746661e092d083757263a249fedca535bd6dd819a8c50de074731a",
    strip_prefix = "rules_buf-0.1.1",
    urls = [
        "https://github.com/bufbuild/rules_buf/archive/refs/tags/v0.1.1.zip",
    ],
)

load("@rules_buf//buf:repositories.bzl", "rules_buf_dependencies", "rules_buf_toolchains")

rules_buf_dependencies()

rules_buf_toolchains()

load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")

rules_proto_dependencies()

rules_proto_toolchains()

Using a specific version of the `rules_proto`

rules_proto is required to use rules_buf. By default, rules_buf automatically loads rules_proto, but you can use a specific version of it by loading it before rules_buf:

WORKSPACE

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_proto",
    sha256 = "66bfdf8782796239d3875d37e7de19b1d94301e8972b3cbd2446b332429b4df1",
    strip_prefix = "rules_proto-4.0.0",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_proto/archive/refs/tags/4.0.0.tar.gz",
        "https://github.com/bazelbuild/rules_proto/archive/refs/tags/4.0.0.tar.gz",
    ],
)

load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")

rules_proto_dependencies()

rules_proto_toolchains()

http_archive(
    name = "rules_buf",
    sha256 = "523a4e06f0746661e092d083757263a249fedca535bd6dd819a8c50de074731a",
    strip_prefix = "rules_buf-0.1.1",
    urls = [
        "https://github.com/bufbuild/rules_buf/archive/refs/tags/v0.1.1.zip",
    ],
)

# Load the other rules_buf assets

Pinning the `buf` version

By default, rules_buf_toolchains loads the latest buf version. For hermetic builds, pin the buf version using the version attribute.

WORKSPACE

# rules_buf fetches the sha based on the version number--the version is enough for hermetic builds.
-rules_buf_toolchains()
+rules_buf_toolchains(version = "v1.47.2")

Rules

The rules work alongside proto_library rules. You can configure rules_buf using a buf.yaml configuration file. Export the buf.yaml using exports_files(["buf.yaml"]) to reference it. For repositories that contain a [buf.work.yaml][buf_work_yaml] that references multiple buf.yaml files, you need to export and reference each buf.yaml file independently.

We recommend using the Gazelle extension to generate the following rules.

`buf_dependencies`

buf_dependencies is a repository rule that downloads one or more modules from the Buf Schema Registry (BSR) and generates build files using Gazelle. Set up Gazelle to use this rule. To also use Gazelle to generate this rule and update deps in proto_library targets, see the Dependencies section.

Attributes

Name	Description	Type	Mandatory	Default
name	A unique name for this repository.	Name	required
modules	The module pins `remote/owner/repo:revision`	List of strings	required

Example

WORKSPACE

load("@rules_buf//buf:defs.bzl", "buf_dependencies")

buf_dependencies(
    name = "buf_deps",
    modules = [
        "buf.build/envoyproxy/protoc-gen-validate:dc09a417d27241f7b069feae2cd74a0e",
        "buf.build/acme/petapis:84a33a06f0954823a6f2a089fb1bb82e",
    ],
)

BUILD

load("@rules_proto//proto:defs.bzl", "proto_library")

# imports "validate/validate.proto"
proto_library(
    name = "foo_proto",
    srcs = ["pet.proto"],
    deps = ["@buf_deps//validate:validate_proto"],
)

We recommend using a single buf_dependencies rule for each buf.yaml file. The Gazelle extension does this by default.

`buf_lint_test`

buf_lint_test is a test rule that lints one or more proto_library targets.

Note

Unused imports can't be detected due to the way the lint plugin captures warnings (Issue #32).

Attributes

Name	Description	Type	Mandatory	Default
`name`	A unique name for this target.	Name	required
`config`	The `buf.yaml` file.	Label	optional	Applies the default `buf.yaml`
`targets`	`proto_library` targets to lint	List of labels	required

Example

load("@rules_buf//buf:defs.bzl", "buf_lint_test")
load("@rules_proto//proto:defs.bzl", "proto_library")

proto_library(
    name = "foo_proto",
    srcs = ["pet.proto"],
    deps = ["@go_googleapis//google/type:datetime_proto"],
)

buf_lint_test(
    name = "foo_proto_lint",
    targets = [":foo_proto"],
    config = "buf.yaml",
)

This can be run as:

$ bazel test :foo_proto_lint

We recommend having a single buf_lint_test for each proto_library target. The Gazelle extension can generate them in the same pattern.

`buf_breaking_test`

buf_breaking_test is a test rule that checks one or more proto_library targets for breaking changes. It requires an image file to check against.

Attributes

Name	Description	Type	Mandatory	Default
`name`	A unique name for this target.	Name	required
`against`	The image file to check against.	Label	required
`config`	The `buf.yaml` file.	Label	optional	Applies the default `buf.yaml`
`exclude_imports`	Exclude imports from breaking change detection.	Boolean	optional	`False`
`limit_to_input_files`	Only run breaking checks against the files in the targets. This has the effect of filtering the against image to only contain the files in the input.	Boolean	optional	`True`
`targets`	`proto_library` targets to check for breaking changes	List of labels	required	`[]`

Example

load("@rules_buf//buf:defs.bzl", "buf_breaking_test")
load("@rules_proto//proto:defs.bzl", "proto_library")

proto_library(
    name = "foo_proto",
    srcs = ["foo.proto"],
)

buf_breaking_test(
    name = "foo_proto_breaking",
    against = "//:image.binpb", # The Image file to check against.
    targets = [":foo_proto"], # The Protobuf library
    config = ":buf.yaml",
)

This can be run as:

$ bazel test :foo_proto_breaking

We recommend having a single buf_breaking_test for each buf.yaml. For repositories that contain a buf.work.yaml that references multiple buf.yaml files, there needs to be exactly one buf_breaking_test for each buf.yaml file.

Alternatively, a single buf_breaking_test can be used against each proto_library target. For this to work, limit_to_input_files attribute must be set to True as the against image file may contain other Protobuf files. Although this is closer to how Bazel operates, for this particular use case it isn't recommended. See the module vs package mode example for a concrete example of the differences.

The Gazelle extension can generate buf_breaking_test in either levels of granularity.

Image inputs

You can generate a Buf image file like this:

$ buf build --exclude-imports -o image.binpb <input>

The <input> is often a directory containing a buf.yaml file, but all of the other Input formats are also supported.

We recommend storing the Image file in a testdata directory and checking it in to version control and updating it as needed. In the case of repositories that follow a versioning scheme like semver, you can update it on each new release either manually or with a post-release hook.

As an alternative to checking the Image file into version control, you can use CI artifacts. Many CI servers, like Travis CI, enable you to upload build artifacts to a backend like S3. In CI, you can set up a pipeline to build the Image on each commit and then add those artifacts to your WORKSPACE:

WORKSPACE

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file")

# Assuming you're using s3 and bucket is at http://s3-us-east-1.amazonaws.com/bucket/foo/bar
# and COMMIT is a variable storing the commit to compare against
http_file(
    name = "buf_module",
    urls = ["http://s3-us-east-1.amazonaws.com/bucket/foo/bar/images/${COMMIT}/image.binpb"],
    sha256 = "...",
)

This file can be referenced from buf_breaking_test. The commit and sha256 need to be updated as needed.

A single image file should be maintained for each buf.yaml file. This is true for both module and package level granularity of buf_breaking_test.

Gazelle

Gazelle is a build file generator for Bazel projects that natively supports Protobuf. rules_buf includes a Gazelle extension for generating buf_breaking_test and buf_lint_test rules out of buf.yaml configuration files.

Setup

Start by setting up rules_buf, then set up Gazelle using the official instructions. Once Gazelle is set up, add the following snippet at the end of the WORKSPACE file:

WORKSPACE

load("@rules_buf//gazelle/buf:repositories.bzl", "gazelle_buf_dependencies")

gazelle_buf_dependencies()

Now modify the BUILD file with the gazelle target to include the buf extension:

BUILD

-load("@bazel_gazelle//:def.bzl", "gazelle")
+load("@bazel_gazelle//:def.bzl", "gazelle", "gazelle_binary")

+gazelle_binary(
+    name = "gazelle-buf",
+    languages = [
+        # Loads the native proto extension
+        "@bazel_gazelle//language/proto:go_default_library",
+        # Loads the Buf extension
+        "@rules_buf//gazelle/buf:buf",
+        # NOTE: This needs to be loaded after the proto language
+    ],
+)

gazelle(
    name = "gazelle",
+    gazelle = ":gazelle-buf",
)

Export the buf.yaml file by adding exports_files(["buf.yaml"]) to the BUILD file.

Inside Buf [workspaces][workspaces], make sure to export each buf.yaml file.

Now run Gazelle:

$ bazel run //:gazelle

This takes care of updating your Protobuf build files—just run //:gazelle whenever Protobuf files are added/removed.

Dependencies

Gazelle can also be used to generate buf_dependencies rules. It imports dependencies from buf.lock files.

Add the following code to the BUILD file,

BUILD

gazelle(
    name = "gazelle-update-repos",
    args = [
        # This can also be `buf.yaml` and `buf.lock`.
        "--from_file=buf.work.yaml",
        # This is optional but recommended, if absent gazelle
        # will add the rules directly to WORKSPACE
        "-to_macro=buf_deps.bzl%buf_deps",
        # Deletes outdated repo rules
        "-prune",
    ],
    command = "update-repos",
    gazelle = ":gazelle-buf",
)

Add the following line of code anywhere after rules_buf_toolchains in the WORKSPACE file:

WORKSPACE

load("@rules_buf//buf:defs.bzl", "buf_dependencies")

Now run Gazelle update-repos command:

$ bazel run //:gazelle-update-repos

This creates the file buf_deps.bzl with the buf_deps macro that loads the buf_dependencies rules. It also calls the macro from the WORKSPACE file.

Arguments

Argument	Mandatory	Default
`-from_file file` Must be one of `buf.yaml` or `buf.lock`. When using `buf.yaml`, the rule imports from the associated `buf.lock` file.	Required
`-to_macro macroFile%defName` Tells Gazelle to write new repository rules into a `.bzl` macro function rather than the `WORKSPACE` file.	Optional	''
`-prune true`	False When true, Gazelle removes `buf_dependencies` rules that no longer have equivalent `buf.yaml` files.	Optional

Lint

By default, a buf_lint_test rule is generated for each of the proto_library rule generated by Gazelle. It picks up the buf.yaml that the Protobuf package belongs to.

Run this command to list the generated lint rules:

$ bazel query 'kind(buf_lint_test, //...)'

Breaking change detection

To run breaking change detection against Protobuf sources, you need to add a Gazelle directive that points to an Image target to generate breaking change detection rules. Gazelle directives are top-level comments in Bazel BUILD files that provide Gazelle with configuration.

See the Image inputs section for instructions on maintaining image files themselves.

Add this Gazelle directive to the top of the BUILD file at the root of the Buf Module, which is the directory with a buf.yaml file.

BUILD

# gazelle:buf_breaking_against //:against_image_file

You can generate buf_breaking_test in two different modes: module mode (preferred) and package mode.

Module mode (preferred)

This is the default and preferred mode. buf_breaking_test is generated for each buf module. The rule references all the proto_library rules that are part of a buf module. This way the test can detect if any files are deleted.

Once the buf_breaking_against directive is added, run gazelle:

$ bazel run //:gazelle

Run this command to list the generated breaking rules:

$ bazel query 'kind(buf_breaking_test, //...)'

This mimics running buf breaking on a module. This is the most accurate way to check for breaking changes. However, depending on multiple targets at once is an anti-pattern in Bazel, so that's why we've provided package mode as an alternative.

Package mode

Package mode generates a buf_breaking_test rule for each of the proto_library rule, which lets you test only the proto_library that has changed.

Add this Gazelle directive to switch to package mode:

# gazelle:buf_breaking_mode package

Now run Gazelle again:

$ bazel run //:gazelle

Running this command shows buf_breaking_test rules generated in multiple packages:

$ bazel query 'kind(buf_breaking_test, //...)'

Example: Module vs. Package mode

Let's consider a Buf module with this directory structure:

├── buf.yaml
├── BUILD
├── foo
│   └── v1
│       ├── foo.proto
│       └── BUILD
└── bar
    └── v1
        ├── bar.proto
        └── BUILD

Module mode

A single buf_breaking_test rule is generated in BUILD. If a breaking change occurs in either foo.proto or bar.proto this test detects it (even if foo.proto is deleted entirely).

Let's break down this scenario,

A typical CI setup would be to use bazel test //... to run tests.
When foo.proto is deleted, Gazelle needs to be run again to update the build files.
This time when Gazelle runs, buf_breaking_test rule has one less target: ["bar/v1:bar_proto"].
The CI runs bazel test //....
The buf_breaking_test detects the missing file and fail.

Package mode

buf_breaking_test rules are generated in both foo/v1/BUILD and bar/v1/BUILD against their respective proto_library targets. If a breaking change occurs in either foo.proto or bar.proto these tests detects it. However, if either foo.proto or bar.proto is deleted the tests fail to detect it.

Let's break down this scenario,

A typical CI setup would be to use bazel test //... to run tests.
When foo.proto is deleted, Gazelle needs to be run again to update the build files.
This time when Gazelle runs, foo/v1/BUILD no longer contains a buf_breaking_test rule.
The CI runs bazel test //....
This always passes as long as the test is removed along with the file.

Examples

Check out some of the sample workspaces that demonstrate usage in various scenarios.

Bazel

Setup

Using a specific version of the rules_proto

Pinning the buf version

Rules

buf_dependencies

Attributes

Example

buf_lint_test

Attributes

Example

buf_breaking_test

Attributes

Example

Image inputs

Gazelle

Setup

Dependencies

Arguments

Lint

Breaking change detection

Module mode (preferred)

Package mode

Example: Module vs. Package mode

Module mode

Package mode

Examples

Using a specific version of the `rules_proto`

Pinning the `buf` version

`buf_dependencies`

`buf_lint_test`

`buf_breaking_test`