If you’re already using Protobuf, you have a schema that describes your messages and services. Protobuf is best known for generating types, clients, and server stubs for many different programming languages, but its plugin system can produce much, much more, like documentation and translations of your schema into other formats. Today I’ll cover two of those formats: JSON Schema and OpenAPI. Both let you extend your original Protobuf schema into new places.
Two plugins make this possible: Buf’s protoc-gen-jsonschema and protoc-gen-connect-openapi, a community plugin that I wrote and maintain. Let’s look at what they produce and how to add them to a project.
The example schema
We’ll use a small inventory service with a few Protovalidate rules. A product has a SKU with a particular format, a name between 2 and 100 characters long, a nonnegative quantity, and a unit price as a decimal string.
syntax = "proto3";
package acme.inventory.v1;
import "buf/validate/validate.proto";
message Product {
string sku = 1 [(buf.validate.field).string.pattern = "^[A-Z0-9-]+$"];
string name = 2 [
(buf.validate.field).string.min_len = 2,
(buf.validate.field).string.max_len = 100
];
int32 quantity = 3 [(buf.validate.field).int32.gte = 0];
string unit_price = 4 [(buf.validate.field).string.pattern = "^[0-9]+\\.[0-9]{2}$"];
}
message GetProductRequest {
string sku = 1 [(buf.validate.field).string.pattern = "^[A-Z0-9-]+$"];
}
message GetProductResponse {
Product product = 1;
}
service InventoryService {
rpc GetProduct(GetProductRequest) returns (GetProductResponse);
}JSON Schema
protoc-gen-jsonschema generates JSON Schema (draft 2020-12) definitions for your messages. Here’s what it produces for Product:
{
"$id": "acme.inventory.v1.Product.jsonschema.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"name": {
"default": "",
"maxLength": 100,
"minLength": 2,
"type": "string"
},
"quantity": {
"anyOf": [
{
"exclusiveMaximum": 2147483648,
"minimum": 0,
"type": "integer"
},
{
"pattern": "^-?[0-9]+$",
"type": "string"
}
],
"default": 0
},
"sku": {
"default": "",
"pattern": "^[A-Z0-9-]+$",
"type": "string"
},
"unitPrice": {
"default": "",
"pattern": "^[0-9]+\\.[0-9]{2}$",
"type": "string"
}
},
"title": "Product",
"type": "object"
}You can see the Protovalidate rules in the output: min_len and max_len became minLength and maxLength, and the SKU regex became a pattern. The integer branch of quantity has minimum: 0 and an upper bound for int32. The anyOf allows integers as strings too, following Protobuf’s JSON mapping. This is the .jsonschema.json file variant, so fields use their JSON names: a Protobuf field named unit_price renders as unitPrice.
By default, the plugin writes a few different files, one for each combination of three choices: Protobuf or JSON field names, inlining referenced messages or separating them into different files, and allowing or dropping alternate representations like that string-encoded integer. The plugin’s README describes each variant and all of the other options you can use.
There are quite a lot of things you can do with this JSON Schema output. You can point VS Code or a JetBrains IDE at the schema to get autocomplete on field names and an error on an out-of-range value. The most common use of this is for editing configuration files. You can use that same JSON Schema file to validate payloads at boundaries where untrusted data is coming in, like webhooks and browser clients. You can also use it to constrain structured output from LLMs like Gemini or ChatGPT, so the response parses into the shape you expect. You can also feed that same file to form generators, fake data generators, and document stores that validate on write.
OpenAPI
A Connect unary call is an HTTP POST with a JSON body, which is the kind of endpoint that OpenAPI is good at describing.
That’s what protoc-gen-connect-openapi generates. I wrote it and I maintain it, but it’s my own project rather than an official Buf one. It produces an OpenAPI 3.1 document that describes each endpoint as the Connect protocol defines it, along with all of the related types. Here’s a shortened version of the output for our inventory service:
openapi: 3.1.0
info:
title: acme.inventory.v1
paths:
/acme.inventory.v1.InventoryService/GetProduct:
post:
operationId: acme.inventory.v1.InventoryService.GetProduct
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/acme.inventory.v1.GetProductRequest'
required: true
responses:
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/connect.error'
"200":
description: Success
content:
application/json:
schema:
$ref: '#/components/schemas/acme.inventory.v1.GetProductResponse'
components:
schemas:
acme.inventory.v1.Product:
type: object
properties:
sku:
type: string
pattern: ^[A-Z0-9-]+$
name:
type: string
maxLength: 100
minLength: 2
quantity:
type: integer
minimum: 0
format: int32
unitPrice:
type: string
pattern: ^[0-9]+\.[0-9]{2}$
additionalProperties: falseI did truncate this output a bit because it also contains standard options and parameters that are useful in practice but are too noisy for this article. For example, the default response refers to a connect.error schema describing the Connect errors that any endpoint can return.
OpenAPI has some options that aren’t normally defined in Protobuf schemas, such as server URLs and authentication schemes. If you run the plugin locally, it can merge in a handwritten OpenAPI file using base=<file>. It also respects gnostic annotations from the google/gnostic project, which let you keep those details in the proto instead of a separate file: servers and security schemes at the file level, per-RPC operation settings, and field-level extras like examples and formats.
So you have an OpenAPI spec. Now what? You can load it into Scalar, Swagger UI, or Redoc to build a documentation site, or feed it to a tool like openapi-generator to generate clients in languages that Connect doesn’t directly support yet. Some API gateway products can reject traffic at the edge if it doesn’t match an OpenAPI specification, which keeps low-effort bots and scanners from ever reaching your backend. While I think Protobuf is a simpler and more precise schema format, there are many people and companies that integrate OpenAPI heavily into their API services, so being able to tap into that integration can be very powerful.
Three ways to run these plugins
There are actually three different ways to run these plugins, depending on how much you want to maintain dependencies and build pipelines yourself.
As a local plugin
Both plugins are written in Go, so you can install them with go install:
go install github.com/bufbuild/protoschema-plugins/cmd/protoc-gen-jsonschema@latest
go install github.com/sudorandom/protoc-gen-connect-openapi@latestAdd them to buf.gen.yaml and run buf generate:
version: v2
plugins:
- local: protoc-gen-jsonschema
out: gen/jsonschema
- local: protoc-gen-connect-openapi
out: gen/openapiAs a remote plugin
Both plugins are also available as remote plugins in the BSR. When you specify remote instead of local, the Buf CLI sends the generation request to the BSR, which runs the plugins and returns the generated files, so there’s nothing to install on your machine:
version: v2
plugins:
- remote: buf.build/bufbuild/protoschema-jsonschema
out: gen/jsonschema
- remote: buf.build/community/sudorandom-connect-openapi
out: gen/openapi$ buf generate
$ ls gen/openapi/acme/inventory/v1/
inventory.openapi.yamlPin the plugin versions for reproducible builds, for example buf.build/bufbuild/protoschema-jsonschema:v0.5.0.
As a generated SDK
For a module already published to the BSR, you can download the generated files directly from a URL. To get the URL, open the module’s SDKs tab, choose one of these plugins, and copy the archive URL. For example, these commands download JSON Schema and OpenAPI archives for connectrpc/eliza:
curl -fsSL -o eliza-jsonschema.zip https://buf.build/gen/archive/connectrpc/eliza/bufbuild/protoschema-jsonschema/latest.zip
curl -fsSL -o eliza-openapi.zip https://buf.build/gen/archive/connectrpc/eliza/community/sudorandom-connect-openapi/latest.zipThe URL follows this pattern for every module and plugin:
https://buf.build/gen/archive/{owner}/{module}/{plugin_owner}/{plugin}/{reference}.zipreference can be latest, a label (like main or v1.2.3), or a module commit. The archive documentation has more details on archive-based SDKs, including the alternative tar.gz output and query parameters for including imports in the archive.
Your CI pipelines and developer machines can now download the JSON Schema or OpenAPI specifications without needing the Buf CLI or maintaining their own generation configuration. The BSR takes care of it automatically.
One schema, many outputs
JSON Schema and OpenAPI are just two examples of what the plugin system can produce. Plugins can generate types, service clients, server stubs, documentation, and other schema formats: anything you can derive from a schema. Plugins are actually pretty simple to build, but that’s a topic for another post.
If you need JSON Schema or OpenAPI for schemas you already have in Protobuf, generate them rather than writing them by hand. If you run into issues or have any questions, come ask in Buf Slack.
