Using generated SDKs – Tutorial
The Buf CLI's buf generate
command generates code from your Protobuf files. It uses a buf.gen.yaml
configuration
file to configure input, plugin, and output options, and is a direct replacement for code generation in protoc
. It can
accept many input types—for this tutorial, you'll use a single-module
workspace.
The tutorial will take you through various ways to set up your generation, from fully local to managed mode.
Prerequisites
We recommend completing the Buf CLI tour to get an overview of the Buf CLI first.
This tutorial assumes you already have Protocol Buffers installed.
- Install the Buf CLI
- Install the
protoc-gen-go
plugin, or have the correspondingprotoc
plugin for your output language of choice installed and in your$PATH
. The code examples use the Go plugin.$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.31.0 $ export PATH="$PATH:$(go env GOPATH)/bin"
- Have at least 1 directory containing
.proto
files
1. Define a module
Modules represent a collection of files that are configured, built, and versioned as a logical unit when performing
Buf operations. Workspaces are collections of modules and are configured by the buf.yaml
configuration file, which
should generally be put above the directories that contain the modules within it.
For example, a workspace with a single module would be structured like this (this workspace will be the example throughout):
workspace_root
├── buf.yaml
└── proto
└── acme
└── weather
└── v1
└── weather.proto
Create a basic boilerplate buf.yaml
file with all of the required elements by running buf config init
in your
workspace root:
$ cd /path/to/workspace/directory
$ buf config init
The command generates a minimal configuration file to define your module:
version: v2
lint:
use:
- STANDARD
breaking:
use:
- FILE
Next, add the module
paths for the Protobuf file directories within the workspace (one per module):
version: v2
modules:
- path: proto
lint:
use:
- STANDARD
breaking:
use:
- FILE
This new module is your input for the buf generate
commands in the rest of the tutorial. For more information about
the specific fields, see the buf.yaml
reference.
2. Configure your buf.gen.yaml
file
To generate code with the Buf CLI, you use a buf.gen.yaml
configuration file to specify the languages you want to
output, the plugins you want to use, and so on. This file replaces the various command-line flags that are required by
protoc
.
Create a new buf.gen.yaml
file in the workspace root, and copy/paste the following code into it.
workspace_root
├── buf.gen.yaml
├── buf.yaml
└── proto
└── acme
└── weather
└── v1
└── weather.proto
version: v2
plugins:
- local: protoc-gen-go
out: gen/go
opt: paths=source_relative
inputs:
- directory: proto
The file defines which plugins to use to generate code, where to output it, and what the inputs are. For more
information about the available fields, see the buf.gen.yaml
reference.
buf generate
can take many types of input beyond a local directory. See the inputs reference
for details about how to specify other types of input to Buf CLI commands.
3. Generate code using local plugins
Now that your configuration is set up, all you need to do is run the command:
$ buf generate
You should see a new gen
directory appear in your tree, containing the generated client code. The file structure under
the gen
directory corresponds to the structure of your Protobuf files:
workspace_root
├── buf.gen.yaml
├── buf.yaml
├── gen
│ └── go
│ └── acme
│ └── weather
│ └── v1
│ └── weather.pb.go
└── proto
└── acme
└── weather
└── v1
└── weather.proto
4. Generate code using remote plugins
Now you'll regenerate the code, this time using the same plugin hosted on the Buf Schema Registry (BSR).
First, remove the gen
directory.
$ rm -rf gen
Then modify your buf.gen.yaml
file to point the plugins
keys to the remote plugin. Note that you can specify the
version (and revision number, if one exists).
version: v2
plugins:
- remote: buf.build/protocolbuffers/go:v1.31.0
out: gen/go
Regenerate the code:
$ buf generate
The gen
directory reappears with the same structure and files as before. You've now removed the necessity for
locally-installed protoc
plugins for this set of .proto
files.
See Using remote plugins for more information about the advantages of remote plugins and where to find them.
4. Generate code using managed mode
Managed mode is Buf's way of clearly separating API producer concerns from consumer concerns, and reducing toil and error across organizations:
-
Producers are free to publish clean API definitions without including Protobuf options like language-specific package and class prefixes in their
.proto
files. -
Consumers can enable managed mode with two lines of code and generate code with thoughtful default settings for these options, while still having the flexibility to override them if needed. There's no need to remember or share text files of arcane invocation flags.
Because your project may not include these Protobuf options, we'll use the files below to demonstrate the concept.
Given the Protobuf file below and the requirement that the go_package
file option needs to be prepended with
github.com/acme/weather/gen/go
, the corresponding managed mode settings would be as shown in the buf.gen.yaml
file below.
syntax = "proto3";
package acme.weather.v1;
option go_package = "acme/weather/v1";
// Messages, enums, services, etc.
version: v2
managed:
enabled: true
override:
- file_option: go_package_prefix
value: github.com/acme/weather/gen/go
plugins:
- remote: buf.build/protocolbuffers/go:v1.31.0
out: gen/go
inputs:
- directory: proto
When you run buf generate
with this setup, the compiler applies the managed mode defaults and
the specified go_package_prefix
on the fly, creating a temporary .proto
file to generate code that includes the options you would
have had to hard-code into each of your .proto
files:
syntax = "proto3";
package acme.weather.v1;
option go_package = "github.com/acme/weather/gen/go/acme/weather/v1"
// Messages, enums, services, etc.
which then generates the Go code in the specified structure:
workspace_root
├── buf.gen.yaml
├── buf.yaml
├── gen
│ └── go
│ └── github.com
│ └── acme
│ └── weather
│ └── gen
│ └── go
│ └── acme
│ └── v1
│ └── weather.pb.go
└── proto
└── acme
└── weather
└── v1
└── weather.proto
For more information about managed mode's defaults, usage, and fields, see Managed mode and the buf.gen.yaml reference.
Related docs
- See detailed usage examples in the overview
- Learn how to find and consume remote plugins
- Browse the
buf generate
command reference