Installation
The BSR is designed to run on Kubernetes, and is distributed as a Helm Chart and accompanying Docker images through an OCI registry. The Helm Chart and Docker images are versioned, and are expected to be used together. The default values in the Chart use Docker images with the same version as the Chart itself.
Please review the list of BSR Dependencies before getting started.
1. Authenticate helm
To get started, authenticate helm
with the Buf OCI registry using the keyfile that was sent alongside this
documentation. The keyfile should contain a base64 encoded string.
$ cat keyfile | helm registry login -u _json_key_base64 --password-stdin \
https://us-docker.pkg.dev/buf-images-1/bsr
2. Create a namespace
Create a Kubernetes namespace in the k8s cluster for the bsr
Helm Chart to use:
$ kubectl create namespace bsr
3. Create a pull secret
Create a pull secret using the same provided keyfile from Step 1. This will be used by the cluster to pull images from the Buf OCI registry:
$ kubectl create secret --namespace bsr docker-registry bufpullsecret \
--docker-server=us-docker.pkg.dev/buf-images-1/bsr \
--docker-username=_json_key_base64 \
--docker-password="$(cat keyfile)"
4. Configure the BSR’s Helm values
The BSR is configured using Helm values through the bsr
Helm Chart.
Create a file named bsr.yaml
to store the Helm values, which is required by the helm install
step below.
This file can be in any location, but we recommend creating it in the same directory where the helm commands are run.
Set the desired host
and configure the chart to use the image pull secret (created above):
host: example.com # Hostname that the BSR will be served from
imagePullSecrets:
- name: bufpullsecret # The image pull secret that was created above
Put the values from the steps below in the bsr.yaml
file. You may skip to
Install the Helm Chart for a full example Helm chart.
Configure object storage
The BSR requires either S3-compatible object storage or Azure Blob Storage.
S3
The bufd client and oci-registry will attempt to acquire credentials from the environment. To configure the storage set the following Helm values, filling in your S3 variables:
storage:
use: s3
s3:
bucketName: "my-bucket-name"
endpoint: "s3.us-east-1.amazonaws.com"
region: "us-east-1"
# forcePathStyle: false # Optional, use path-style bucket URLs (http://s3.amazonaws.com/BUCKET/KEY)
# insecure: false # Optional, disable TLS
Alternatively, you may instead use an access key pair.
- Add the
accessKeyId
to the configuration:
storage:
use: s3
s3:
accessKeyId: "AKIAIOSFODNN7EXAMPLE"
bucketName: "my-bucket-name"
endpoint: "s3.us-east-1.amazonaws.com"
region: "us-east-1"
# forcePathStyle: false # Optional, use path-style bucket URLs (http://s3.amazonaws.com/BUCKET/KEY)
# insecure: false # Optional, disable TLS
- Create a k8s secret containing the s3 access secret key:
$ kubectl create secret --namespace bsr generic bufd-storage \
--from-literal=secret_access_key=<s3 secret access key>
Azure Blob Storage
A standard general storage account type is required in order to support block blobs.
The bufd client and oci-registry will attempt to acquire credentials from the environment. To configure the storage, set the following
Helm values by filling in your Azure variables and adding the required annotations for the bufd
and ociregistry
service accounts and deployments:
storage:
use: azure
azure:
accountName: "my-storage-account-name"
container: "my-container"
useAccountKey: false
bufd:
serviceAccount:
annotations:
azure.workload.identity/client-id: "my-client-id"
deployment:
podLabels:
azure.workload.identity/use: "true"
ociregistry:
serviceAccount:
annotations:
azure.workload.identity/client-id: "my-client-id"
deployment:
podLabels:
azure.workload.identity/use: "true"
The service accounts to be bound to the federated identity credentials are named bufd-service-account
and oci-registry-service-account
.
Alternatively, you may instead use the storage account key.
- Set the required helm values:
storage:
use: azure
azure:
accountName: "my-storage-account-name"
container: "my-container"
useAccountKey: true
- Create a k8s secret containing an Azure storage account key:
$ kubectl create secret --namespace bsr generic bufd-storage \
--from-literal=account_key=<azure storage account key>
Create a Postgres database
The BSR requires a PostgreSQL database. The BSR postgres
user requires full access to
the database, and additionally must be able to create the pgcrypto
and pg_trgm
extensions.
To configure Postgres, set the following helm values:
postgres:
host: "postgres.example.com"
port: 5432
database: postgres
user: postgres
Then create a k8s secret containing the postgres user password:
$ kubectl create secret --namespace bsr generic bufd-postgres \
--from-literal=password=<postgres password>
Note that if you are using CosmosDB, it must be configured as a single-node cluster with high availability (HA) enabled.
Configure Redis
The BSR requires a Redis instance.
- Only the Redis Standalone deployment mode is supported.
- Redis Cluster and Sentinel modes are not supported for the BSR.
To configure Redis, create a k8s secret containing the address:
$ kubectl create secret --namespace bsr generic bufd-redis \
--from-literal=address=redis.example.com:6379
Optionally, authentication and TLS for Redis are also supported. These can be set with the following Helm values:
redis:
# Set to true to enable auth for redis.
# The auth token will be read from the "auth" field in the "bufd-redis" secret
auth: true
tls:
# Whether to use TLS for connecting to Redis
# Set to "false" to disable TLS
# Set to "local" to use certs from the "ca" field in the "bufd-redis" secret
# Set to "system" to use the system trust store
use: "false"
- If authentication is enabled, the redis auth string should be added to the
bufd-redis
secret in theauth
field. - If TLS is enabled and
use
is set tolocal
, the CA certificate(s) to trust should be added to thebufd-redis
secret in theca
field.
Example of a secret containing both an authentication token and a CA certificate:
$ kubectl create secret --namespace bsr generic bufd-redis \
--from-literal=address=redis.example.com:6379 \
--from-literal=auth=<redis auth string> \
--from-file=ca=<redis ca.crt>
Example of a secret containing an authentication token, assuming a connection string like
redis.example.com:6379,password=<password>,ssl=True,abortConnect=False
:
$ kubectl create secret --namespace bsr generic bufd-redis \
--from-literal=address=redis.example.com:6379 \
--from-literal=auth=<redis password>
Configure authentication
The BSR supports authentication using an external identity provider (IdP), through Security Assertion Markup Language (SAML) or OpenID Connect (OIDC).
In the SAML IdP, create a new application to represent the BSR. It should return a single sign-on URL and IdP metadata. Either a public URL or raw XML can be specified for the SAML config. If SAML is being configured in Okta, please follow our Okta - SAML guide.
To configure SAML authentication in the BSR, set the following Helm values:
auth:
method: saml
saml:
# Endpoint where the XML metadata is available
idpMetadataURL: "https://example-provider.com/app/12345/sso/saml/metadata"
# If the authentication provider does not have a metadata url,
# the raw XML metadata can be configured using the idpRawMetadata,
# value instead.
idpRawMetadata: |
<?xml version="1.0" encoding="utf-8"?>
<EntityDescriptor etc>
# Optionally, configure the attribute containing groups membership information,
# to enable support for automated organization membership provisioning.
# Note that if configured, a user will not be permitted to log in to the BSR if the attribute is missing from the SAML assertion.
# https://buf.build/docs/bsr/private/user-lifecycle#autoprovisioning
groupsAttributeName: ""
# Optional
# A list of emails which will be granted server admin permissions on login
# Note that this list is case-sensitive
autoProvisionedAdminEmails:
- "user@example.com"
SAML requires the application to have access to a certificate used for signing/encryption as part of the authentication process. For the BSR, this is stored as a Kubernetes TLS secret named
bsr-saml-cert
, and may be self-signed. For example, you can generate a certificate and create the required secret using OpenSSL.
$ openssl req -newkey rsa:2048 -nodes -keyout ca.key -subj "/CN=example.com" -x509 -days 3650 -out ca.crt \
$ kubectl create secret --namespace bsr tls bsr-saml-cert \
--key ca.key \
--cert ca.crt
In the OIDC IdP, create a new application to represent the BSR and provide the callback URL. If OIDC is being configured in Okta, please follow our Okta - OIDC guide.
To configure OIDC authentication in the BSR, set the following Helm values:
auth:
method: oidc
oidc:
issuerURL: "https://example.okta.com"
clientID: "0oa2ho2ylo0HFI61d5d7"
# Optional
# A list of emails which will be granted server admin permissions on login
# Note that this list is case-sensitive
autoProvisionedAdminEmails:
- "user@example.com"
Additionally, a Kubernetes secret must be created in order for OIDC to function:
$ kubectl create secret --namespace bsr generic bufd-client-secret \
--from-literal=client_secret=<oidc client secret>
Configure Ingress
The BSR uses a Kubernetes Ingress resource to handle
incoming traffic and for terminating TLS. The domain used here must match the host
set in the Helm values above.
TLS is required for the BSR to function properly. HTTP2 is preferred to allow for gRPC support.
bufd:
ingress:
enabled: true
className: "" # Optional ingress class to use
annotations: {} # Optional ingress annotations
hosts:
- host: example.com
paths:
- path: /
portName: http
# Optional TLS configuration for the ingress.
# May be omitted to configure TLS termination, depending on the ingress.
# Requires a kubernetes TLS secret.
tls:
- secretName: bsr-tls-cert
hosts:
- example.com
If the load balancer does not support H2C, TLS can optionally be used for communication between the load balancer and
the BSR by enabling TLS on the listening ports of the bufd
application. This requires a
Kubernetes TLS secret named bsr-tls-cert
.
bufd:
tls:
enabled: true
# Optional. Secret name for the TLS cert
# secretName: bsr-tls-cert
# Optional. Used to add annotations to the ingress service.
# May be needed for some ingress controllers to function correctly.
service:
annotations: {}
Configure Observability
The metrics
block is used to configure the collection and exporting of metrics from your application using prometheus:
observability:
metrics:
use: prometheus
runtime: true
prometheus:
podLabels: # This is required if enabling network policies.
app: prometheus
port: 9090
path: /metrics
Trusting Additional Certificates
If you bump into issues regarding self signed certificates, such as seeing the error
tls: failed to verify certificate: x509: certificate signed by unknown authority
, you can add your root certificates
on the BSR. To trust additional certificates, mount the files on the bufd
pod and include them in the client TLS
configuration.
bufd:
deployment:
extraVolumeMounts:
- mountPath: /config/secrets/certificates/cert.pem
name: certificate
readOnly: true
subPath: cert.pem
extraVolumes:
- name: certificate
secret:
secretName: tls-cert
items:
- key: cert.pem
path: cert.pem
clientTLS:
extraCerts:
- /config/secrets/certificates/cert.pem
5. Install the Helm Chart
After following the steps above, the set of Helm values should be similar to the example below:
host: example.com
imagePullSecrets:
- name: bufpullsecret
storage:
use: s3
s3:
bucketName: "my-bucket-name"
endpoint: "s3.us-east-1.amazonaws.com"
region: "us-east-1"
postgres:
host: "postgres.example.com"
port: 5432
database: postgres
user: postgres
auth:
method: saml
saml:
idpMetadataURL: "https://example-provider.com/app/12345/sso/saml/metadata"
autoProvisionedAdminEmails:
- "user@example.com"
bufd:
ingress:
enabled: true
hosts:
- host: example.com
paths:
- path: /
portName: http
tls:
- secretName: bsr-tls-cert
hosts:
- example.com
deployment:
extraVolumeMounts:
- mountPath: /config/secrets/certificates/cert.pem
name: certificate
readOnly: true
subPath: cert.pem
extraVolumes:
- name: certificate
secret:
secretName: tls-cert
items:
- key: cert.pem
path: cert.pem
clientTLS:
extraCerts:
- /config/secrets/certificates/cert.pem
observability:
metrics:
use: prometheus
runtime: true
prometheus:
podLabels: # This is required if enabling network policies.
app: prometheus
port: 9090
path: /metrics
Using the bsr.yaml
Helm values file, install the Helm Chart for the cluster and set the correct
BSR Version:
$ helm install bsr oci://us-docker.pkg.dev/buf-images-1/bsr/charts/bsr \
--version "1.x.x" \
--namespace=bsr \
--values bsr.yaml
The BSR instance should now be up and running on https://<host>
.
To help verify that the BSR is working correctly, we expose a status page to server admins at https://<host>/-/status
.
It's also accessible on port 3001 on each bufd pod without authentication, at http://<bufd pod ip>:3001/-/status
.
More information about the status page can be found here.