Go packages
The Buf Schema Registry provides generated SDKs for Go from modules and plugins using go get
, just like any other Go
library. It generates SDKs automatically when you push schema changes, which eliminates the need to manage a Protobuf
toolchain or generate code locally.
See the tutorial for instructions on how to access generated SDKs from the BSR directly.
The BSR Go module proxy implements the GOPROXY protocol for Buf modules by generating SDKs on the
fly—Go code stubs aren't generated until you request them using go get
. Once generated, they're cached, and the cached
SDKs are returned from subsequent requests.
Using Go modules
The key to consuming from the BSR Go module proxy is choosing the right Go module path. The import path for generated Go code has this format:
In this example, buf.build/gen/go/connectrpc/eliza/connectrpc/go
, the BSR generates code for the
connectrpc/eliza
module using the
connectrpc/go
plugin. This SDK depends on
buf.build/gen/go/connectrpc/eliza/protocolbuffers/go
as well—connect-go
has a dependency on code generated by the
protocolbuffers/go
plugin, and generated SDKs automatically include plugin
dependencies.
Installing SDKs
To generate the connectrpc/eliza
Protobuf module using the connect-go
plugin, you
could install the generated SDK like this:
$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go
Outputgo: downloading buf.build/gen/go/connectrpc/eliza/connectrpc/go v1.11.0-20230727062025-d8fbf2620c60.1 go: added buf.build/gen/go/connectrpc/eliza/connectrpc/go v1.11.0-20230727062025-d8fbf2620c60.1
Import paths
If you're consuming a publicly available generated SDK, an easy way to get the import paths for your packages is to load up the module on pkg.go.dev (for example, https://pkg.go.dev/buf.build/gen/go/connectrpc/eliza/connectrpc/go). From there, you can drill down into directories to get the import path for a particular package.
If you're not consuming a publicly available SDK, you can generally construct an import path by combining the elements
that go into the import path: remote, module, plugin and proto package(s). Continuing with the connectrpc
example:
- remote:
buf.build
- module:
connectrpc/eliza
- plugin:
connectrpc/go
- proto package(s): The
connectrpc/eliza
module only has a single proto package, connectrpc.eliza.v1
From here, you can build up the package you'd expect,using the following syntax:
Certain plugins, like Connect and gRPC, generate code into subpackages (see Connect's reasoning). So, we tack on Connect's generated subpackage name and get this path, which contains our importable code:
buf.build/gen/go/connectrpc/eliza/connectrpc/go/connectrpc/eliza/v1/elizav1connect
Note the apparent duplication in the import path. In the example above, connectrpc/eliza
is included in the path
twice in separate locations, which may seem odd. However, this is a consequence of following good Protobuf package
hygiene—a well-named package commonly shares some elements with the name of the module.
Versions
By default, when you go get
a Buf module, the BSR generates code from the most recent
reference for the module. However, you can also install a specific module
version by referencing an explicit version or a label with the @
syntax. To discover SDK versions, you can
browse a repository's SDKs page, which has installation instructions and an interactive UI.
Latest
To get the generated SDK for the latest version of the module, using the latest plugin version:
$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@latest
Commit
To get the generated SDK for the module at a commit, using the latest plugin version:
$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@<COMMIT_ID>
Label
The BSR supports commits on labels. This feature enables you to push unreleased Protobuf file changes and consume generated code without affecting the default label.
To get the generated SDK for the module at a label, using the latest plugin version:
$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@<LABEL>
References with names that contain /
are not compatible with the Go generated SDKs versioning scheme.
Full syntax
With SDK versions (valid semver):
- The version core is the plugin version 1.11.0
- The semver pre-release version is composed of:
- module commit timestamp (YYYYMMDDHHMMSS) 20230727062025
- module commit short name (12 characters) d8fbf2620c60
- The final identifier is the plugin revision 1 for the plugin version
Most users will likely use @latest
, but if you need to reference a version explicitly, you can do so like this:
$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@v1.11.0-20230727062025-d8fbf2620c60.1
If you need a more specific version than the @latest
, @<COMMIT_ID>
or @<LABEL>
shorthands,
such as needing to install a specific plugin version,
you can use the buf registry sdk version
command.
Only commits that are on the default label at the time they're pushed to the BSR have populated timestamps. Timestamps
on commits pushed to other labels are zeroed out with 00000000000000
to easily distinguish them as changes in labels
that are still in development.
Private generated SDKs
When using SDKs generated from private BSR repositories, you'll need to authenticate by including a personal API token for local use or a Bot user's API token for CI workflows.
The go
tool authenticates with proxies using .netrc
credentials or credentials
directly specified in GOPROXY URLs. Luckily, buf registry login
updates your .netrc
for you. See the
Authentication page for instructions on authenticating with buf registry login
.
Private modules are only accessible to authorized users and can't be pulled through the default module proxy at
https://proxy.golang.org or have their checksums verified with the public checksum database at https://sum.golang.org/.
To access private modules, you need to specify GOPRIVATE
:
export GOPRIVATE="buf.build/gen/go,${GOPRIVATE}"
This implicitly sets GONOPROXY
/GONOSUMDB
to the same value, causing Go to use the direct
strategy for fetching
directly from BSR without going through the public proxy.
Modify the Go environment using the export
command in your shell or with the go env -w
command to persist the
change:
$ export GOPRIVATE="buf.build/gen/go,${GOPRIVATE}"
$ go env -w GOPRIVATE="buf.build/gen/go,${GOPRIVATE}"
For more information, see the official private modules documentation.
Private generated SDKs in CI
Taking all of the above into consideration, to use private generated SDKs in CI, we recommend the following:
- Create a new BSR token.
- Add a secret environment variable
BUF_TOKEN
to your GitHub repository (or repository on another VCS provider) using your new BSR token.- For GitHub Actions, go to Settings -> Secrets and variables -> Actions -> New repository secret.
- Add a secret environment variable
BUF_USER
with the BSR username that the token is associated with. - Set
GOPRIVATE=buf.build/gen/go
as an environment variable in your CI job.- For GitHub Actions, see this doc.
- Alternatively, if you'd like this to just be active for all GitHub actions jobs, feel free to just set
GOPRIVATE
as a repository secret with the valuebuf.build/gen/go
. - For other CI workflows, you'll likely just need to run
export GOPRIVATE=buf.build/gen/go
.
- Run
echo "${BUF_TOKEN}" | buf registry login --username "${BUF_USER}" --token-stdin
as part of CI.
Using GOPROXY for private generated SDKs
While not recommended, if you would prefer not to authenticate via buf registry login
, you can also set GOPROXY
directly.
After creating a new BSR token, export the following environment variables:
export BUF_TOKEN="<your_token>"
export BUF_USER="<username_associated_with_token>"
export GOPROXY="https://${BSR_USER}:${BSR_TOKEN}@buf.build/gen/go,${GOPROXY}"
export GONOSUMDB="https://${BSR_USER}:${BSR_TOKEN}@buf.build/gen/go,${GONOSUMDB}"
As mentioned in the Go docs on using direct credentials, use caution when taking this approach, as environment variables may appear in shell history and in logs.
Because GONOPROXY
includes the Buf registry, use of GOPRIVATE
not compatible with specifying auth directly in GOPROXY
.
Only netrc-based auth via buf registry login
is supported in this configuration.
Available plugins
For a full list of supported plugins, navigate to the BSR plugins page and search for Go.
To learn more about how these plugins are packaged and distributed, go to the bufbuild/plugins repository. If you find a useful plugin that should be added, file an issue.
Related docs
- Try the generated SDKs tutorial to learn how to generate SDKs from the BSR.