Skip to content

Go packages#

The Buf Schema Registry provides generated SDKs for Go from modules and plugins using go get, just like any other Go library. It generates SDKs automatically when you push schema changes, which eliminates the need to manage a Protobuf toolchain or generate code locally.

See the quickstart for instructions on how to access generated SDKs from the BSR directly.

The BSR Go module proxy implements the GOPROXY protocol for Buf modules by generating SDKs on the fly—Go code stubs aren't generated until you request them using go get. Once generated, they're cached, and the cached SDKs are returned from subsequent requests.

Using Go modules#

The key to consuming from the BSR Go module proxy is choosing the right Go module path. The import path for generated Go code has this format:

Go import path syntax
BSR_INSTANCE/gen/go/MODULE_OWNER/MODULE_NAME/PLUGIN_OWNER/PLUGIN_NAME

# Examples
buf.build/gen/go/connectrpc/eliza/connectrpc/go           # Path for public BSR
example.buf.dev/gen/go/connectrpc/eliza/connectrpc/go     # SDK for Pro plan BSR instance
buf.example.com/gen/go/connectrpc/eliza/connectrpc/go     # SDK for Enterprise BSR instance

The module path contains these elements:

  • BSR_INSTANCE is the domain name of your BSR instance. (Default: buf.build)
  • MODULE_OWNER is the owner of the module.
  • MODULE_NAME is the name of the module.
  • PLUGIN_OWNER is the owner of the plugin.
  • PLUGIN_NAME is the name of the plugin.

In this example, the BSR generates code for the connectrpc/eliza module using the connectrpc/go plugin. This SDK depends on BSR_INSTANCE/gen/go/connectrpc/eliza/protocolbuffers/go as well—connect-go has a dependency on code generated by the protocolbuffers/go plugin, so the SDK automatically includes the dependency also.

Installing SDKs#

To generate the connectrpc/eliza Protobuf module using the connect-go plugin, you could install the generated SDK like this:

$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go

go: downloading buf.build/gen/go/connectrpc/eliza/connectrpc/go v1.11.0-20230727062025-d8fbf2620c60.1
go: added buf.build/gen/go/connectrpc/eliza/connectrpc/go v1.11.0-20230727062025-d8fbf2620c60.1

Availability delay for Go module proxy#

When you push a module change to the BSR, the public Go module proxy (and the docs) can take ~30 min to update. From their docs:

it may take up to 30 minutes for the mirror's cache to expire and fresh data about the version to become available.

This can also be true when you buf push and immediately do a go get ...@latest. In this case, it won't pick up the latest commit for a bit because the go env GOPROXY for most users is https://proxy.golang.org. To overcome this, you can fetch directly from the BSR Go module proxy and bypass the Google module proxy:

GOPRIVATE=buf.build/gen/go go get ... @latest

Import paths#

If you're consuming a publicly available generated SDK, an easy way to get the import paths for your packages is to load up the module on pkg.go.dev (for example, https://pkg.go.dev/buf.build/gen/go/connectrpc/eliza/connectrpc/go). From there, you can drill down into directories to get the import path for a particular package.

If you're not consuming a publicly available SDK, you can generally construct an import path by combining the elements that go into the import path, as in the preceding example: BSR instance, module, plugin, and Protobuf package.

Go import path syntax
BSR_INSTANCE/gen/go/MODULE_OWNER/MODULE_NAME/PLUGIN_OWNER/PLUGIN_NAME/PROTO_PACKAGE

# Example
buf.build/gen/go/connectrpc/eliza/connectrpc/go/connectrpc/eliza/v1

Certain plugins, like Connect and gRPC, generate code into subpackages (see Connect's reasoning). So, we tack on Connect's generated subpackage name and get this path, which contains our importable code:

buf.build/gen/go/MODULE_OWNER/MODULE_NAME/PLUGIN_OWNER/PLUGIN_NAME/PROTO_PACKAGE/SUBPACKAGE

# Example
buf.build/gen/go/connectrpc/eliza/connectrpc/go/connectrpc/eliza/v1/elizav1connect

Note the apparent duplication in the import path. In the example above, connectrpc/eliza is included in the path twice in separate locations, which may seem odd. However, this is a consequence of following good Protobuf package hygiene—a well-named package commonly shares some elements with the name of the module.

Versions#

By default, when you go get a Buf module, the BSR generates code from the most recent reference for the module. However, you can also install a specific module version by referencing an explicit version or a label with the @ syntax. To discover SDK versions, you can browse a repository's SDKs page, which has installation instructions and an interactive UI.

Latest#

To get the generated SDK for the latest version of the module, using the latest plugin version:

$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@latest

Commit#

To get the generated SDK for the module at a commit, using the latest plugin version:

$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@COMMIT_ID

Label#

The BSR supports commits on labels. This feature enables you to push unreleased Protobuf file changes and consume generated code without affecting the default label.

To get the generated SDK for the module at a label, using the latest plugin version:

$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@LABEL

References with names that contain / are not compatible with the Go generated SDKs versioning scheme.

Full syntax#

Go module version syntax
PLUGIN_VERSION-MODULE_TIMESTAMP-COMMIT_ID.PLUGIN_REVISION

# Example (this comes after 'go@' in the preceding examples)
v1.11.0-20230727062025-d8fbf2620c60.1

With SDK versions (valid semver):

  • The version core is the plugin version 1.11.0
  • The semver pre-release version is composed of:
    • module commit timestamp (YYYYMMDDHHMMSS) 20230727062025
    • module commit short name (12 characters) d8fbf2620c60
  • The final identifier is the plugin revision 1 for the plugin version

Most users likely want to use @latest, but if you need to reference a version explicitly, you can do so like this:

$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@v1.11.0-20230727062025-d8fbf2620c60.1

If you need a more specific version than the @latest, @COMMIT_ID or @LABEL shorthands, such as needing to install a specific plugin version, you can use the buf registry sdk version command.

Only commits that are on the default label at the time they're pushed to the BSR have populated timestamps. Timestamps on commits pushed to other labels are zeroed out with 00000000000000 to easily distinguish them as changes in labels that are still in development.

Private generated SDKs#

When using SDKs generated from private BSR repositories, you'll need to authenticate by including a personal API token for local use or a Bot user's API token for CI workflows.

The go tool authenticates with proxies using .netrc credentials or credentials directly specified in GOPROXY URLs. Luckily, buf registry login updates your .netrc for you. See the Authentication page for instructions on authenticating with buf registry login.

Private modules are only accessible to authorized users and can't be pulled through the default module proxy at https://proxy.golang.org or have their checksums verified with the public checksum database at https://sum.golang.org/. To access private modules, you need to specify GOPRIVATE:

export GOPRIVATE="buf.build/gen/go,${GOPRIVATE}"

This implicitly sets GONOPROXY/GONOSUMDB to the same value, causing Go to use the direct strategy for fetching directly from BSR without going through the public proxy.

Modify the Go environment using the export command in your shell or with the go env -w command to persist the change:

$ export GOPRIVATE="buf.build/gen/go,${GOPRIVATE}"

$ go env -w GOPRIVATE="buf.build/gen/go,${GOPRIVATE}"

For more information, see the official private modules documentation.

Private generated SDKs in CI#

Taking all of the above into consideration, to use private generated SDKs in CI, we recommend the following:

  • Create a new BSR token.
  • Add a secret environment variable BUF_TOKEN to your GitHub repository (or repository on another VCS provider) using your new BSR token.
    • For GitHub Actions, go to Settings -> Secrets and variables -> Actions -> New repository secret.
  • Add a secret environment variable BUF_USER with the BSR username that the token is associated with.
  • Set GOPRIVATE=buf.build/gen/go as an environment variable in your CI job.
    • For GitHub Actions, see this doc.
    • Alternatively, if you'd like this to just be active for all GitHub actions jobs, feel free to just set GOPRIVATE as a repository secret with the value buf.build/gen/go.
    • For other CI workflows, you'll likely just need to run export GOPRIVATE=buf.build/gen/go.
  • Run echo "${BUF_TOKEN}" | buf registry login --username "${BUF_USER}" --token-stdin as part of CI.

Using GOPROXY for private generated SDKs#

While not recommended, if you prefer not to authenticate via buf registry login, you can also set GOPROXY directly. After creating a new BSR token, export the following environment variables:

export BUF_TOKEN="<your_token>"
export BUF_USER="<username_associated_with_token>"
export GOPROXY="https://${BSR_USER}:${BSR_TOKEN}@buf.build/gen/go,${GOPROXY}"
export GONOSUMDB="https://${BSR_USER}:${BSR_TOKEN}@buf.build/gen/go,${GONOSUMDB}"

As mentioned in the Go docs on using direct credentials, use caution when taking this approach, as environment variables may appear in shell history and in logs.

Because GONOPROXY includes the Buf registry, use of GOPRIVATE not compatible with specifying auth directly in GOPROXY. Only netrc-based auth via buf registry login is supported in this configuration.

Available plugins#

For a full list of supported plugins, navigate to the BSR plugins page and search for Go.

To learn more about how these plugins are packaged and distributed, go to the bufbuild/plugins repository. If you find a useful plugin that should be added, file an issue.

Troubleshooting#

Why do I get a 404 error when trying to install the generated Go SDK for buf.build/googleapis/googleapis?#

All types in googleapis/googleapis have a go_package option that points to the canonical package: google.golang.org/genproto/googleapis. We recommend that you import the canonical packages instead of generating googleapis yourself.