Authoring a plugin

The purpose of this guide is to walk you through a concrete example of how to publish an existing protoc-based plugin to the BSR.

We'll take a real-world plugin named protoc-gen-twirp and convert it to a containerized BSR plugin. This will be the building block for the Authoring a Template example in the next section.

The protoc-gen-twirp source code can be found here.

1. Docker registry authentication

To push plugins to the BSR, you will need to authenticate to the plugin Docker registry using docker login. The username doesn't matter, but has to be provided. Obtain an API token (password) from the Settings Page and run this command:

$ docker login -u myuser plugins.buf.build
Output
password:
Login Succeeded

NOTE: Even though the docker CLI says Login Succeeded, it doesn't actually test your credentials. If you're having issues doing a docker push (step 5), refer to the docker login docs.

2. Create BSR plugin

Before we can push a plugin to the BSR, it must exist first. You can create a plugin through the UI or the buf CLI.

From the UI click your avatar in the top-right corner, select Plugins and click the Create Plugin button. Follow the on-screen instructions.

For this example, however, we'll use the buf CLI.

This tutorial uses a real organization (demolab) and plugin name (twirp), make sure to substitute these with your own values.

Create the plugin with buf command:

$ buf beta registry plugin create \
    buf.build/demolab/plugins/twirp --visibility public
Output
Owner    Name
demolab  twirp

There is now a public plugin on the BSR named twirp owned by the demolab organization:

https://buf.build/demolab/plugins/twirp

3. Prepare the Dockerfile

BSR plugins are containerized protoc-based plugins that read CodeGeneratorRequest from standard input and write CodeGeneratorResponse to standard output.

Since the protoc-gen-twirp plugin is written in Go, we'll build this plugin with go install.

Here is the full Dockerfile example, the entrypoint is protoc-gen-twirp:

FROM golang as builder

ENV GOOS=linux GOARCH=amd64 CGO_ENABLED=0

RUN go install github.com/twitchtv/twirp/protoc-gen-twirp@v8.1.0+incompatible
# Note, the Docker images must be built for amd64. If the host machine architecture is not amd64
# you need to cross-compile the binary and move it into /go/bin.
RUN bash -c 'find /go/bin/${GOOS}_${GOARCH}/ -mindepth 1 -maxdepth 1 -exec mv {} /go/bin \;'

FROM scratch

# Runtime dependencies
LABEL "build.buf.plugins.runtime_library_versions.0.name"="github.com/twitchtv/twirp"
LABEL "build.buf.plugins.runtime_library_versions.0.version"="v8.1.0+incompatible"
LABEL "build.buf.plugins.runtime_library_versions.1.name"="google.golang.org/protobuf"
LABEL "build.buf.plugins.runtime_library_versions.1.version"="v1.27.1"

COPY --from=builder /go/bin /

ENTRYPOINT ["/protoc-gen-twirp"]

This Dockerfile uses multi-stage builds.

The intended GOOS/GOARCH must be linux/amd64. This is important, especially if you're building Docker images on an ARM-based machine, such as Apple M1 computers.

Normally go install would install to $GOPATH/bin/$GOOS_$GOARCH when cross-compiling, so we added this line to copy the executable into the /go/bin path.

RUN bash -c 'find /go/bin/${GOOS}_${GOARCH}/ -mindepth 1 -maxdepth 1 -exec mv {} /go/bin \;'

A plugin may generate code that depends on a runtime library, and it is important that this information is captured in the containerized BSR plugin. This is accomplished using Docker labels. In this example the protoc-gen-twirp plugin depends on github.com/twitchtv/twirp and google.golang.org/protobuf.

The twitchtv/twirp project does not support Go modules, but we have to pass a version that Go is able to recognize. For this example we'll use version v8.1.0+incompatible, however, for most Go projects with module support you would pass a normal semver version.

Since Go binaries are statically linked and have no further dependencies the binary is copied into a lightweight scratch image to reduce the final image size.

4. Build the Dockerfile

Once we prepared the Dockerfile, the next step is to build and tag an image.

We'll do so locally by running this command:

$ docker build -f Dockerfile.twirp -t plugins.buf.build/demolab/twirp:v8.1.0-1 .

We're tagging the version as v8.1.0-1 even though the upstream version of the plugin is v8.1.0. This structure enables us to make changes to the packaging of the plugin without changing the upstream version, for example, if we made a mistake in our Dockerfile. This pattern is commonly used in other systems where packaging is done externally to the upstream software, such as Debian and Arch package versioning systems.

5. Publish plugin to the BSR

Lastly, publish the containerized protoc-based plugin to the BSR. Make sure you have authenticated your docker client in step 1.

$ docker push plugins.buf.build/demolab/twirp:v8.1.0-1
Output
The push refers to repository [plugins.buf.build/demolab/twirp]
f3dfdb857337: Pushed
v8.1.0-1: digest: sha256:782f6522b2bc8cc943338b61a73e795c4309969f9974ef3431e4aa1f76150e16 size: 528

Awesome, you've successfully published your first BSR plugin!

Remember, plugins are the smallest reusable components required for code generation. In the next section we'll prepare a BSR Template and use the demolab/twirp plugin created above.

Continue to the next section to learn more about authoring and using BSR Templates