Custom plugins
All public plugins on the BSR are maintained by Buf for security purposes—we want to make sure that the code generators everyone uses are verified by us. However, organizations often write custom Protobuf plugins to generate logic specific to their business. Custom plugins are available on our Pro and Enterprise plans—reach out if you're interested in adding this feature.
- Pro and Enterprise: Can upload unlimited custom plugins, which can be public or private within your private
BSR instance:
- Public—All user accounts with access to the instance will have access to the plugin
- Private—Only user accounts who are members of the organization that owns a plugin will have access to the plugin
The instructions below cover how to build, package, and push a custom plugin.
Prerequisites
- A minimum Buf CLI version of v1.21.0 or higher.
- Docker must be installed and running.
- A BSR organization must exist, and the user pushing the plugin must have at least an Admin role.
An organization can be created through the BSR UI (see Manage organizations) or the Buf CLI (see the buf beta registry command reference).
buf beta registry organization create <{private_bsr_server}/{organization_name}>
Creating a custom plugin
The following are the steps we'll cover to build and push a custom plugin:
- Build Docker image (optional)
- Prepare
buf.plugin.yaml
file - Push the plugin
Build a Docker image
If you already have a Docker image built with a Protobuf plugin that accepts a CodeGeneratorRequest
from standard
input and writes a CodeGeneratorResponse
to standard output, then you can skip this step.
We'll use a Go-based plugin (protoc-gen-go-json) as an example. If you have a more advanced setup and need help packaging a plugin, don't hesitate to get in touch.
# syntax=docker/dockerfile:1.6
FROM golang:1.21-bullseye AS builder
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
go install -ldflags "-s -w" -trimpath github.com/mitchellh/protoc-gen-go-json@v1.1.0
# When building a Docker image on a host that does not match linux/amd64 (such as an M1),
# go install will put the binary in $GOPATH/bin/$GOOS_$GOARCH/. The mv command copies
# the binary to /go/bin so subsequent steps do not fail when copying from the builder.
RUN mv /go/bin/linux_amd64/* /go/bin || true
FROM scratch
COPY --link /etc/passwd /etc/passwd
COPY /go/bin/ /
USER nobody
ENTRYPOINT [ "/protoc-gen-go-json" ]
- Make sure the OS/arch targets
linux/amd64
(hard coded above). This is important, especially if you're building Docker images on an ARM-based machine, such as an Apple M1 computer. - If possible, use minimal images such as scratch or distroless and leverage multi-stage builds.
- Set a non-root
USER
, such as nobody, and make sure that user exists.
Once you've prepared the Dockerfile, build and tag the image:
docker build --platform linux/amd64 -t buf.example.com/acme/go-json:v1.1.0 .
Prepare buf.plugin.yaml
At a minimum, the buf.plugin.yaml
file defines the plugin name and plugin version.
version: v1
name: buf.example.com/acme/go-json
plugin_version: v1.1.0
- The
version
key is the YAML configuration version, always set tov1
. - The
plugin_version
key has the formatv{semver}
—the v prefix is required and the version must be valid semantic versioning. - The
name
key format is{private_bsr_server}/{organization_name}/{plugin_name}
There is additional metadata that may be captured by the buf.plugin.yaml
configuration file, such as the plugin source
URL, the output language, description, dependencies, etc. See the buf.plugin.yaml
file
documentation.
Push plugin to the BSR
Make sure you have authenticated to the BSR.
Once you have a Docker image and a buf.plugin.yaml
file, run the following command:
$ buf beta registry plugin push \
--visibility [public,private] \
--image buf.example.com/acme/go-json:v1.1.0
The --image
flag is referencing an image built in an earlier step, but this can also be an image from an external
source, such as Docker Hub or Quay. Make sure to docker pull
the image so it is available locally.
Setting the visibility to private makes the plugin accessible to organization members only.
A successful push outputs details about the plugin, and the plugin is immediately available in the Plugins section within the organization.
Owner Name Version Revision
acme go-json v1.1.0 1
Upload a custom plugin for generated SDKs
Generated SDKs allow schema consumers to fetch pre-generated packages in their native ecosystem so they don't need to generate code manually. You can take any module on the BSR and a supported plugin and fetch bundled code. However, this means that we need to specify configuration and plugin defaults in advance.
Occasionally, users want different code generation behavior in generated SDKs. Because there is no way to change configuration or plugin options, the only solution is to upload (and maintain) a different flavor of the plugin to force a different behavior during code generation. The example below shows how to add CommonJS support to the bufbuild/es plugin (which is hard-coded to output ESM) by making a copy and changing the defaults, but the process is similar for any plugin you want to modify.
1. Create a BSR organization for custom plugins
Keeping the custom plugins together makes maintenance simpler, but it's not required. For this example we'll use a
sample Pro BSR instance and organization at https://jsexample.buf.dev/custom-plugins
.
2. Clone and modify the base plugin
The BSR's plugins are publicly accessible, so you can start from the existing plugin code and make modifications to
suit your needs. You can find general best practices and a list of the buf.plugin.yaml
fields in the
plugin repo documentation.
-
Clone the following repository:
https://github.com/bufbuild/plugins
. -
Go to the directory of the version you want to use—we'll use the current version,
v1.6.0
:$ cd plugins/bufbuild/es/v1.6.0
-
Update the
buf.plugin.yaml
file with the following changes:- Change the
name
field to a new upload destination (making sure not to override the version of the plugin that Buf manages). - Add the necessary plugin options for CommonJS under
registry
. - Change the import style under
npm
tocommon_js
.
- Change the
The complete patch looks like this:
--- A/plugins/bufbuild/es/v1.6.0/buf.plugin.yaml
+++ B/plugins/bufbuild/es/v1.6.0/buf.plugin.yaml
@@ -1,5 +1,5 @@
version: v1
+name: jsexample.buf.dev/custom-plugins/bufbuild-es
-name: buf.build/bufbuild/es
plugin_version: v1.6.0
source_url: https://github.com/bufbuild/protobuf-es
integration_guide_url: https://github.com/bufbuild/protobuf-es#quickstart
@@ -8,8 +8,10 @@
- javascript
- typescript
registry:
+ opts:
+ - js_import_style=legacy_commonjs
npm:
+ import_style: commonjs
- import_style: module
rewrite_import_path_suffix: pb.js
deps:
- package: '@bufbuild/protobuf'
3. Build a Docker image
Make sure to build the image for the correct platform (linux/amd64
)
$ docker build \
--platform linux/amd64 \
-t jsexample.buf.dev/custom-plugins/bufbuild-es:v1.6.0 .
4. Push the Docker image to the BSR
$ buf beta registry plugin push \
--visibility=public \
--image=jsexample.buf.dev/custom-plugins/bufbuild-es:v1.6.0 \
--override-remote=jsexample.buf.dev
Now, when you npm install
a generated SDK from this custom plugin:
$ npm install @buf/acme_petapis.custom-plugins_bufbuild-es@latest
It has commonjs
imports instead of ESM:
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
const { proto3 } = require("@bufbuild/protobuf");
const { Money } = require("@buf/googleapis_googleapis.custom-plugins_bufbuild-es/google/type/money_pb.js");
Delete a custom plugin
You can delete a custom plugin with the Buf CLI by passing a plugin reference or a plugin reference and version.
If the version is omitted, then all versions for that plugin will be deleted.
$ buf beta registry plugin delete buf.example.com/acme/go-json
$ buf beta registry plugin delete buf.example.com/acme/go-json:v1.1.0