Register schemas

This feature is only available on the Enterprise plan.

In the Confluent ecosystem, schemas are associated with data via subjects, which are typically derived from a Kafka topic's name. For example, the default strategy, TopicNameStrategy, suffixes the topic name with -value.

You create subjects within the BSR's Confluent Schema Registry by annotating messages with a custom option that identifies the instance and subject name.

Note

All references to buf.example.com below should be replaced with the hostname for your private BSR server.

1. Add a dependency on the bufbuild/confluent managed module to your Buf workspace's buf.yaml file.

   version: v2
   modules:
     path: /path/to/local/directory
     name: buf.example.com/demo/analytics
   deps:
     - buf.example.com/bufbuild/confluent
   

2. Run buf dep update to verify and lock the dependency.

   $ buf dep update
   

3. Add one or more buf.confluent.v1.subject options to a new or existing Protobuf message, including your Confluent Schema Registry instance name and the subject's name.

   syntax = "proto3";
   
   package demo.analytics;
   
   import "buf/confluent/v1/extensions.proto";
   import "google/protobuf/timestamp.proto";
   
   message EmailUpdated {
     google.protobuf.Timestamp updated_at = 1;
     fixed64 user_id = 2;
     string previous_email = 3;
     string new_email = 4;
     bool new_email_verified = 5;
   
     option (buf.confluent.v1.subject) = {
       instance_name: "<INSTANCE_NAME>",
       name: "email-updated-value",
     }
   }
   

4. Run buf push to update the module in the BSR. If successful, the subject is created or updated with the message as its schema.

   $ buf push
   

   You can check it in the browser using your instance URL (you must be logged into the BSR): https://buf.example.com/integrations/confluent/<INSTANCE_NAME>/subjects/email-updated-value/versions/latest

Backwards compatibility checks

To preserve compatibility with previous schema versions, the BSR enforces that any changes to a Confluent Schema Registry subject must not introduce a breaking change. Any backwards-incompatible changes made to a subject or its dependencies will be strictly blocked by buf push.

sequenceDiagram
    participant User
    box CSR instance lives inside the BSR
    participant Push as BSR
    participant CSR as CSR
    end
    User->>Push: buf push
    Note over Push: typical push flow
    Push->>CSR: update subjects
    Note over CSR: detect subjects<br>& check compatibility
    alt breaking changes
        CSR->>Push: ERROR
        Push->>User: ERROR
    else otherwise
        Note over CSR: register new schemas<br>& link to subjects
        CSR->>Push: OK
        Push->>User: OK
    end

Warning

The Confluent Schema Registry backwards compatibility checks supersede the BSR breaking change policy check settings. Other breaking changes within the module may still enter the governance review flow.

The following Buf breaking change rules are enforced on .proto files including subjects and their dependencies. The entire WIRE_JSON category is enabled, with an additional subset of rules from the FILE category:

• WIRE_JSON
• ENUM_NO_DELETE
• ENUM_VALUE_NO_DELETE
• EXTENSION_MESSAGE_NO_DELETE
• FILE_NO_DELETE
• RPC_NO_DELETE
• FIELD_NO_DELETE
• FIELD_SAME_TYPE
• FILE_SAME_SYNTAX
• MESSAGE_NO_DELETE
• MESSAGE_NO_REMOVE_STANDARD_DESCRIPTOR_ACCESSOR
• ONEOF_NO_DELETE
• SERVICE_NO_DELETE

Deregistering schemas

To deregister a subject, remove the buf.confluent.v1.subject option from the message and re-push the module to the BSR with buf push. The subject's schema will no longer update, but will still be accessible from the BSR's Confluent Schema Registry API.