The Buf CLI's buf generate
command generates code from your Protobuf files. It uses a buf.gen.yaml
configuration
file to configure input, plugin, and output options, and is a direct replacement for code generation in protoc
. It can
accept many input types—for this tutorial, you'll use a module,
which is Buf's key working unit.
The tutorial will take you through various ways to set up your generation, from fully local to managed mode.
Prerequisites
We recommend completing the Buf CLI tour to get an overview of the Buf CLI first.
This tutorial assumes you already have Protocol Buffers installed.
- Install the Buf CLI
- Install the
protoc-gen-go
plugin, or have the correspondingprotoc
plugin for your output language of choice installed and in your$PATH
. The code examples use the Go plugin.$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.31.0 $ export PATH="$PATH:$(go env GOPATH)/bin"
- Have at least 1 directory containing
.proto
files
1. Define a module
Modules represent a collection of files that are configured, built, and versioned as a logical unit when performing
buf
operations. Define a module by adding a buf.yaml
config file to the directory of the .proto
files you want to
group together.
$ cd /path/to/your/directory
$ buf mod init
For example:
.
└── acme
└── weather
└── v1
├── buf.yaml
└── weather.proto
buf mod init
creates a basic boilerplate buf.yaml
file with all of the required elements:
version: v1
breaking:
use:
- FILE
lint:
use:
- DEFAULT
This new module is your input for the buf generate
commands in the rest of the tutorial.
For more information about specific fields, see the buf.yaml
reference.
2. Configure your buf.gen.yaml
file
To generate code with the Buf CLI, you use a buf.gen.yaml
configuration file to specify the languages you want to
output, the plugins you want to use, and so on. This file replaces the various command-line flags that are required by
protoc
.
Create a new buf.gen.yaml
file one directory above your .proto
files, and copy/paste the following code into it.
.
├── buf.gen.yaml
└── acme
└── weather
└── v1
├── buf.yaml
└── weather.proto
version: v1
plugins:
- plugin: go
out: gen/go
opt: paths=source_relative
For more information about the available fields, see the
buf.gen.yaml
reference.
3. Generate code using local plugins
Now that your configuration is set up, all you need to do is run the command:
$ buf generate
You should see a new gen
directory appear in your tree, containing the generated client code. The file structure under
the gen
directory corresponds to the structure of your Protobuf files:
.
├── buf.gen.yaml
├── acme
│ └── weather
│ └── v1
│ ├── buf.yaml
│ └── weather.proto
└── gen
├── go
└── acme
└── weather
└── v1
└── weather.pb.go
Any errors are printed out in a file:line:column:message
format by default. For example:
acme/pet/v1/pet.proto:5:8:acme/payment/v1alpha1/payment.proto: does not exist
4. Generate code using remote plugins
Now you'll regenerate the code, this time using the same plugin hosted on the Buf Schema Registry (BSR).
First, go to the directory where the buf.gen.yaml
file is and remove the gen
directory.
$ rm -rf gen
Then modify your buf.gen.yaml
file to point the plugin:
fields to the remote plugin. Note that you can specify the
version (and revision number, if one exists).
version: v1
plugins:
- plugin: buf.build/protocolbuffers/go:v1.31.0
out: gen/go
Regenerate the code:
$ buf generate
The gen
directory reappears with the same structure and files as before. You've now removed the necessity for
locally-installed protoc
plugins for this set of .proto
files.
See Using remote plugins for more information about the advantages of remote plugins and where to find them.
4. Generate code using managed mode
Managed mode is Buf's way of clearly separating API producer concerns from consumer concerns, and reducing toil and error across organizations:
- Producers are free to publish clean API definitions without including Protobuf options like language-specific package
and class prefixes in their
.proto
files. - Consumers can enable managed mode with two lines of code and generate code with thoughtful default settings for these options, while still having the flexibility to override them if needed. There's no need to remember or share text files of arcane invocation flags.
Because your project may not include these Protobuf options, we'll use the files below to demonstrate the concept.
syntax = "proto3";
package acme.weather.v1;
// Messages, enums, service, etc.
version: v1
managed:
enabled: true
go_package_prefix:
default: github.com/acme/weather/gen/proto/go
plugins:
- plugin: buf.build/protocolbuffers/go:v1.31.0
out: gen/go
When you run buf generate
with this setup, the compiler applies the managed mode defaults and
the specified go_package_prefix
on the fly, creating an invisible .proto
file that includes the options you would
have had to hard-code:
syntax = "proto3";
package acme.weather.v1;
option go_package = "github.com/acme/weather/gen/proto/go/acme/weather/v1;weatherv1"
// Messages, enums, service, etc.
and generates the Go code in the specified structure:
.
├── buf.gen.yaml
├── acme
│ └── weather
│ └── v1
│ ├── buf.yaml
│ └── weather.proto
└── gen
└── go
└── github.com
└── acme
└── weather
└── gen
└── proto
└── go
└── acme
└── v1
└── weather.pb.go
For more information about managed mode's defaults, usage, and fields, see Managed mode and the buf.gen.yaml reference.
Related docs
- See detailed usage examples in the overview
- Learn how to find and consume remote plugins
- Browse the
buf generate
command reference
To see complete file sets illustrating these code generation methods, visit these example projects: