Sign inSign up
Buf Schema Registry (BSR)
Buf Schema Registry (BSR) / Generated SDKs

Go packages

The Buf Schema Registry provides generated SDKs for Go from modules and plugins using go get, just like any other Go library. It generates SDKs automatically when you push schema changes, which eliminates the need to manage a Protobuf toolchain or generate code locally.

See the tutorial for instructions on how to access generated SDKs from the BSR directly.

The BSR Go module proxy implements the GOPROXY protocol for Buf modules by generating SDKs on the fly—Go code stubs aren't generated until you request them using go get. Once generated, they're cached, and the cached SDKs are returned from subsequent requests.

Using Go modules

The key to consuming from the BSR Go module proxy is choosing the right Go module path. The import path for generated Go code has this format:

Go module path syntax
{remote}/gen/go/{moduleOwner}/{moduleName}/{pluginOwner}/{pluginName}
Examplebuf.build/gen/go/connectrpc/eliza/connectrpc/go
Legend:
constant{variable}

In this example, buf.build/gen/go/connectrpc/eliza/connectrpc/go, the BSR generates code for the connectrpc/eliza module using the connectrpc/go plugin. This SDK depends on buf.build/gen/go/connectrpc/eliza/protocolbuffers/go as well—connect-go has a dependency on code generated by the protocolbuffers/go plugin, and generated SDKs automatically include plugin dependencies.

Installing SDKs

To generate the connectrpc/eliza Protobuf module using the connect-go plugin, you could install the generated SDK like this:

$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go

Output
go: downloading buf.build/gen/go/connectrpc/eliza/connectrpc/go v1.11.0-20230727062025-d8fbf2620c60.1
go: added buf.build/gen/go/connectrpc/eliza/connectrpc/go v1.11.0-20230727062025-d8fbf2620c60.1

Import paths

If you're consuming a publicly available generated SDK, an easy way to get the import paths for your packages is to load up the module on pkg.go.dev (for example, https://pkg.go.dev/buf.build/gen/go/connectrpc/eliza/connectrpc/go). From there, you can drill down into directories to get the import path for a particular package.

If you're not consuming a publicly available SDK, you can generally construct an import path by combining the elements that go into the import path: remote, module, plugin and proto package(s). Continuing with the connectrpc example:

  • remote: buf.build
  • module: connectrpc/eliza
  • plugin: connectrpc/go
  • proto package(s): The connectrpc/eliza module only has a single proto package, connectrpc.eliza.v1

From here, you can build up the package you'd expect,using the following syntax:

Go SDK import syntax
{remote}/gen/go/{moduleOwner}/{moduleName}/{pluginOwner}/{pluginName}/{protoPackage}
Examplebuf.build/gen/go/connectrpc/eliza/connectrpc/go/connectrpc/eliza/v1
Legend:
constant{variable}

Certain plugins, like Connect and gRPC, generate code into subpackages (see Connect's reasoning). So, we tack on Connect's generated subpackage name and get this path, which contains our importable code:

buf.build/gen/go/connectrpc/eliza/connectrpc/go/connectrpc/eliza/v1/elizav1connect

Note the apparent duplication in the import path. In the example above, connectrpc/eliza is included in the path twice in separate locations, which may seem odd. However, this is a consequence of following good Protobuf package hygiene—a well-named package commonly shares some elements with the name of the module.

Versions

By default, when you go get a Buf module, the BSR generates code from the most recent reference for the module. However, you can also install a specific module version by referencing an explicit version or a label with the @ syntax. To discover SDK versions, you can browse a repository's SDKs page, which has installation instructions and an interactive UI.

Latest

To get the generated SDK for the latest version of the module:

$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@latest

Commit

To get the generated SDK for the module at a commit:

$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@<COMMIT_ID>

Label

The BSR supports commits on labels. This feature enables you to push unreleased Protobuf file changes and consume generated code without affecting the default label.

To get the generated SDK for the module at a label:

$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@<LABEL>
Caveats

References with names that contain / are not compatible with the Go generated SDKs versioning scheme.

Full syntax

Go module version syntax
{pluginVersion}-{moduleTimestamp}-{moduleCommit}.{pluginRevision}
Examplev1.11.0-20230727062025-d8fbf2620c60.1
Legend:
{variable}

With SDK versions (valid semver):

  • The version core is the plugin version 1.11.0
  • The semver pre-release version is composed of:
    • module commit timestamp (YYYYMMDDHHMMSS) 20230727062025
    • module commit short name (12 characters) d8fbf2620c60
  • The final identifier is the plugin revision 1 for the plugin version

Most users will likely use @latest, but if you need to reference a version explicitly, you can do so like this:

$ go get buf.build/gen/go/connectrpc/eliza/connectrpc/go@v1.11.0-20230727062025-d8fbf2620c60.1

Only commits that are on the default label at the time they're pushed to the BSR have populated timestamps. Timestamps on commits pushed to other labels are zeroed out with 00000000000000 to easily distinguish them as changes in labels that are still in development.

Private generated SDKs

When using SDKs generated from private BSR repositories, you'll need to authenticate by including a personal API token for local use or a Bot user's API token for CI workflows.

The go tool authenticates with proxies using .netrc credentials or credentials directly specified in GOPROXY URLs. Luckily, buf registry login updates your .netrc for you. See the Authentication page for instructions on authenticating with buf registry login.

Private modules are only accessible to authorized users and can't be pulled through the default module proxy at https://proxy.golang.org or have their checksums verified with the public checksum database at https://sum.golang.org/. To access private modules, you need to specify GOPRIVATE:

export GOPRIVATE="buf.build/gen/go,${GOPRIVATE}"

This implicitly sets GONOPROXY/GONOSUMDB to the same value, causing Go to use the direct strategy for fetching directly from BSR without going through the public proxy.

Modify the Go environment using the export command in your shell or with the go env -w command to persist the change:

$ export GOPRIVATE="buf.build/gen/go,${GOPRIVATE}"

$ go env -w GOPRIVATE="buf.build/gen/go,${GOPRIVATE}"

For more information, see the official private modules documentation.

Private generated SDKs in CI

Taking all of the above into consideration, to use private generated SDKs in CI, we recommend the following:

  • Create a new BSR token.
  • Add a secret environment variable BUF_TOKEN to your GitHub repository (or repository on another VCS provider) using your new BSR token.
    • For GitHub Actions, go to Settings -> Secrets and variables -> Actions -> New repository secret.
  • Add a secret environment variable BUF_USER with the BSR username that the token is associated with.
  • Set GOPRIVATE=buf.build/gen/go as an environment variable in your CI job.
    • For GitHub Actions, see this doc.
    • Alternatively, if you'd like this to just be active for all GitHub actions jobs, feel free to just set GOPRIVATE as a repository secret with the value buf.build/gen/go.
    • For other CI workflows, you'll likely just need to run export GOPRIVATE=buf.build/gen/go.
  • Run echo "${BUF_TOKEN}" | buf registry login --username "${BUF_USER}" --token-stdin as part of CI.

Using GOPROXY for private generated SDKs

While not recommended, if you would prefer not to authenticate via buf registry login, you can also set GOPROXY directly. After creating a new BSR token, export the following environment variables:

export BUF_TOKEN="<your_token>"
export BUF_USER="<username_associated_with_token>"
export GOPROXY="https://${BSR_USER}:${BSR_TOKEN}@buf.build/gen/go,${GOPROXY}"
export GONOSUMDB="https://${BSR_USER}:${BSR_TOKEN}@buf.build/gen/go,${GONOSUMDB}"

As mentioned in the Go docs on using direct credentials, use caution when taking this approach, as environment variables may appear in shell history and in logs.

Because GONOPROXY includes the Buf registry, use of GOPRIVATE not compatible with specifying auth directly in GOPROXY. Only netrc-based auth via buf registry login is supported in this configuration.

Available plugins

For a full list of supported plugins, navigate to the BSR plugins page and search for Go.

To learn more about how these plugins are packaged and distributed, go to the bufbuild/plugins repository. If you find a useful plugin that should be added, file an issue.

  1. Using Go modules
  2. Installing SDKs
  3. Import paths
  4. Versions
    1. Latest
    2. Commit
    3. Label
    4. Full syntax
  5. Private generated SDKs
    1. Private generated SDKs in CI
    2. Using GOPROXY for private generated SDKs
  6. Available plugins
  7. Related docs