Custom plugins

All public plugins on the BSR are maintained by Buf for security purposes—we want to make sure that the code generators everyone uses are verified by us. However, organizations often write custom Protobuf plugins to generate logic specific to their business. Custom plugins are available on our Pro and Enterprise plans—reach out if you're interested in adding this feature.

• Pro and Enterprise: Can upload unlimited custom plugins, which can be public or private within your private BSR instance:
  • Public—All user accounts with access to the instance will have access to the plugin
  • Private—Only user accounts who are members of the organization that owns a plugin will have access to the plugin

The instructions below cover how to build, package, and push a custom plugin.

Prerequisites

• A minimum Buf CLI version of v1.21.0 or higher.
• Docker must be installed and running.
• A BSR organization must exist, and the user pushing the plugin must have at least an Admin role.

An organization can be created through the BSR UI (see Manage organizations) or the Buf CLI (see the buf beta registry command reference).

buf beta registry organization create <{private_bsr_server}/{organization_name}>

Creating a custom plugin

The following are the steps we'll cover to build and push a custom plugin:

1. Build Docker image (optional)
2. Prepare buf.plugin.yaml file
3. Push the plugin

Build a Docker image

If you already have a Docker image built with a Protobuf plugin that accepts a CodeGeneratorRequest from standard input and writes a CodeGeneratorResponse to standard output, then you can skip this step.

We'll use a Go-based plugin (protoc-gen-go-json) as an example. If you have a more advanced setup and need help packaging a plugin, don't hesitate to get in touch.

# syntax=docker/dockerfile:1.6
FROM golang:1.21-bullseye AS builder
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
    go install -ldflags "-s -w" -trimpath github.com/mitchellh/protoc-gen-go-json@v1.1.0

# When building a Docker image on a host that does not match linux/amd64 (such as an M1),
# go install will put the binary in $GOPATH/bin/$GOOS_$GOARCH/. The mv command copies
# the binary to /go/bin so subsequent steps do not fail when copying from the builder.
RUN mv /go/bin/linux_amd64/* /go/bin || true

FROM scratch
COPY --from=builder --link /etc/passwd /etc/passwd
COPY --from=builder /go/bin/ /
USER nobody
ENTRYPOINT [ "/protoc-gen-go-json" ]

• Make sure the OS/arch targets linux/amd64 (hard coded above). This is important, especially if you're building Docker images on an ARM-based machine, such as an Apple M1 computer.
• If possible, use minimal images such as scratch or distroless and leverage multi-stage builds.
• Set a non-root USER, such as nobody, and make sure that user exists.

Once you've prepared the Dockerfile, build and tag the image:

docker build --platform linux/amd64 -t buf.example.com/acme/go-json:v1.1.0 .

Prepare buf.plugin.yaml

At a minimum, the buf.plugin.yaml file defines the plugin name and plugin version.

version: v1
name: buf.example.com/acme/go-json
plugin_version: v1.1.0

• The version key is the YAML configuration version, always set to v1.
• The plugin_version key has the format v{semver}—the v prefix is required and the version must be valid semantic versioning.
• The name key format is {private_bsr_server}/{organization_name}/{plugin_name}

There is additional metadata that may be captured by the buf.plugin.yaml configuration file, such as the plugin source URL, the output language, description, dependencies, etc. See the buf.plugin.yaml file documentation.

Push plugin to the BSR

Make sure you have authenticated to the BSR.

Once you have a Docker image and a buf.plugin.yaml file, run the following command:

$ buf beta registry plugin push \
    --visibility [public,private] \
    --image buf.example.com/acme/go-json:v1.1.0

The --image flag is referencing an image built in an earlier step, but this can also be an image from an external source, such as Docker Hub or Quay. Make sure to docker pull the image so it is available locally.

Setting the visibility to private makes the plugin accessible to organization members only.

A successful push outputs details about the plugin, and the plugin is immediately available in the Plugins section within the organization.

Owner  Name     Version  Revision
acme   go-json  v1.1.0   1

Upload a custom plugin for generated SDKs

Generated SDKs allow schema consumers to fetch pre-generated packages in their native ecosystem so they don't need to generate code manually. You can take any module on the BSR and a supported plugin and fetch bundled code. However, this means that we need to specify configuration and plugin defaults in advance.

Occasionally, users want different code generation behavior in generated SDKs. Because there is no way to change configuration or plugin options, the only solution is to upload (and maintain) a different flavor of the plugin to force a different behavior during code generation. The example below shows how to add CommonJS support to the bufbuild/es plugin (which is hard-coded to output ESM) by making a copy and changing the defaults, but the process is similar for any plugin you want to modify.

1. Create a BSR organization for custom plugins

Keeping the custom plugins together makes maintenance simpler, but it's not required. For this example we'll use a sample Pro BSR instance and organization at https://jsexample.buf.dev/custom-plugins.

2. Clone and modify the base plugin

The BSR's plugins are publicly accessible, so you can start from the existing plugin code and make modifications to suit your needs. You can find general best practices and a list of the buf.plugin.yaml fields in the plugin repo documentation.

1. Clone the following repository: https://github.com/bufbuild/plugins.

2. Go to the directory of the version you want to use—we'll use the current version, v1.6.0:

   $ cd plugins/bufbuild/es/v1.6.0
   

3. Update the buf.plugin.yaml file with the following changes:

   • Change the name field to a new upload destination (making sure not to override the version of the plugin that Buf manages).
   • Add the necessary plugin options for CommonJS under registry.
   • Change the import style under npm to common_js.

The complete patch looks like this:

--- A/plugins/bufbuild/es/v1.6.0/buf.plugin.yaml
+++ B/plugins/bufbuild/es/v1.6.0/buf.plugin.yaml
@@ -1,5 +1,5 @@
 version: v1
+name: jsexample.buf.dev/custom-plugins/bufbuild-es
-name: buf.build/bufbuild/es
 plugin_version: v1.6.0
 source_url: https://github.com/bufbuild/protobuf-es
 integration_guide_url: https://github.com/bufbuild/protobuf-es#quickstart
@@ -8,8 +8,10 @@
   - javascript
   - typescript
 registry:
+  opts:
+    - js_import_style=legacy_commonjs
   npm:
+    import_style: commonjs
-    import_style: module
     rewrite_import_path_suffix: pb.js
     deps:
       - package: '@bufbuild/protobuf'

3. Build a Docker image

Make sure to build the image for the correct platform (linux/amd64)

$ docker build \
    --platform linux/amd64 \
    -t jsexample.buf.dev/custom-plugins/bufbuild-es:v1.6.0 .

4. Push the Docker image to the BSR

$ buf beta registry plugin push \
    --visibility=public \
    --image=jsexample.buf.dev/custom-plugins/bufbuild-es:v1.6.0 \
    --override-remote=jsexample.buf.dev

Now, when you npm install a generated SDK from this custom plugin:

$ npm install @buf/acme_petapis.custom-plugins_bufbuild-es@latest

It has commonjs imports instead of ESM:

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });

const { proto3 } = require("@bufbuild/protobuf");
const { Money } = require("@buf/googleapis_googleapis.custom-plugins_bufbuild-es/google/type/money_pb.js");

Delete a custom plugin

You can delete a custom plugin with the Buf CLI by passing a plugin reference or a plugin reference and version.

$ buf beta registry plugin delete buf.example.com/acme/go-json

$ buf beta registry plugin delete buf.example.com/acme/go-json:v1.1.0