Overview

A key component of maintaining a consistent Protobuf schema is utilizing a linter to enforce a common set of rules. This is a good practice whether you're building a personal project, maintaining a large set of Protobuf definitions across a major organization, or anything in-between. It's especially important for users and organizations that continually onboard new engineers without a wealth of experience in Protobuf schema design to make sure that APIs remain consistent and follow patterns that allow easy maintainability, but is useful even if you have years of Protobuf experience as a forcing function on your API decisions.

Buf provides lint functionality through buf check lint, which runs a set of lint checkers across your entire Protobuf schema. By default, Buf uses a carefully curated set of lint checkers designed to guarantee consistency and maintainability across a Protobuf schema of any size and any purpose, but without being so opinionated as to restrict organizations from making the design decisions they need to make for their individual APIs.

Buf is designed to be extremely simple to use, while providing functionality for advanced use cases. Features of Buf's linter include:

  • Automatic file discovery. By default, Buf will build your .proto files by walking your file tree and building them per your build configuration. This means you no longer need to manually specify your --proto_paths and files every time you run the tool. However, Buf does allow manual file specification through command-line flags if you want no file discovery to occur, for example in Bazel setups.

  • Selectable configuration of the exact lint checkers you want, including categorization of lint checkers into logical categories. While we recommend using the DEFAULT set of lint checkers, Buf allows you to easily understand and select the exact set of checkers your organization needs.

  • Selectable error output. By default, Buf outputs file:line:col:message information for every lint error, with the file path carefully outputted to match the input location, including if absolute paths are used. JSON output that includes the end line and end column of the lint error is also available, and JUnit output is coming soon.

  • Editor integration. The default error output is easily parseable by any editor, making the feedback loop for lint errors very short. Currently, we provide Vim integration, but will extend this in the future to include other editors such as Emacs, VS Code, and Intellij IDEs.

  • Lint anything from anywhere. Buf allows you to not only lint a Protobuf schema stored as .proto files, but allows you to lint from many different Inputs:

    • Tar or zip archives containing .proto files, both local and remote.
    • Git repository branches or tags containing .proto files, both local and remote.
    • Pre-built Images or FileDescriptorSets from protoc, from both local and remote (http/https) locations.
  • Speed. Buf's internal Protobuf compiler utilizes all available cores to compile your Protobuf schema, while still maintaining deterministic output. Additionally files are copied into memory before processing. As an unscientific example, Buf can compile all 2,311 .proto files in googleapis in about 0.8s on a four-core machine, as opposed to about 4.3s for protoc on the same machine. While both are very fast, this allows for instantaneous feedback, which is especially useful with Editor integration. Buf's speed is directly proportional to the input size, so linting a single file only takes a few milliseconds.

  • Use protoc as your compiler. Existing lint and breaking change detection tools produce an internal representation of your Protobuf schema in one of two ways:

    • By using a third-party Protobuf parser, which is usually error-prone and almost never covers every edge case of the Protobuf grammar.
    • By shelling out to protoc itself and parsing the result, which not only requires specific management of protoc in relation to the lint/breaking change detection tool, but can be cumbersome and error-prone itself, especially if the tool parses error output from protoc.

    Buf tackles this issue by using FileDescriptorSets (which we extend into Images.md) internally for all operations, and allowing these FileDescriptorSets to be produced in one of two ways:

    • By using a newly-developed Golang Protobuf compiler that is continuously tested against thousands of known Protobuf definitions, including all known edge cases of the Protobuf grammar.
    • By allowing users to provide protoc output as buf input, thereby bypassing any compiling or parsing on the part of buf entirely, and instead using protoc, the gold standard of Protobuf compilation.

    See the Image and compiler documentation for more details.

    In short, we don't expect you to natively trust the internal compiler is actually equivalent to protoc - we would want to verify this claim ourselves. There are also cases (such as Bazel setups) where you may already have infrastructure around calling protoc, and may want to just use artifacts from protoc as input to buf.

  • Use buf as a protoc plugin instead of a standalone tool. You can go a step further and use Buf's lint functionality as a protoc plugin with the provided protoc-gen-buf-check-lint plugin.