A faster Protovalidate

Protovalidate’s standard rules have always been CEL expressions. That design gave every language implementation a single source of truth: validate.proto. It made new implementations easier to build and kept their behavior consistent, but it also made validation slower than we wanted.

Go, Java, and TypeScript now bypass CEL when evaluating most standard rules. Compared with the CEL path, validation is roughly 3× faster in Go, 10× faster in Java, and 13× faster in TypeScript. Python is on a different segment of its performance journey: protovalidate-py 2.0 replaces its pure-Python core with a native extension, making the complex-schema benchmark 485× faster. It also switches to the new protobuf-py library.

Native rules

Protovalidate’s standard rules are written once as CEL expressions in Protovalidate’s schema. Every implementation reads them from there. For example, the uint32.lte rule is defined by this CEL expression:

!has(rules.gte) && !has(rules.gt) && this > rules.lte ? 'must be less than or equal to %s'.format([rules.lte]) : ''

In Go, the native upper-bound check looks like this:

func (n nativeNumericCompare[T]) aboveHi(v T) bool {
    if n.upper == upperBoundLt {
        return v >= n.hi
    }
    return v > n.hi
}

The Go version is significantly faster since the host language doesn’t need to run a VM environment for the CEL program.

While building a validator, Go, Java, and TypeScript check each rule for a native implementation, much like the Go version shown above, and will fall back to the CEL expression if the native implementation is missing. So unsupported standard rules and custom rules will continue to use CEL. Python and C++ currently use CEL for all rules for now. More on that later.

Each commit runs the full conformance suite twice: once with native rules and once without. Any difference from CEL fails the build. We use the same setup in other multi-language projects like ConnectRPC.

Go

protovalidate-go v1.3.0 enables native rules by default. If you need the CEL path back for whatever reason, pass WithDisableNativeRules when you create the validator.

Go, Java, and TypeScript recognize the same set of standard rules natively: all twelve numeric types, bool, bytes, enum, string with every one of its well-known formats, the collection rules on repeated and map fields, and the wrapper types like StringValue and Int32Value.

Across these benchmarks, evaluation takes roughly one-third as long as it did before. Here are six of the nine results:

Allocations drop too, reaching zero in the simple scalar cases. The full benchstat output is in #316.

Java

Java’s implementation (#469) is a port of Go’s, with the same approach and rule coverage.

Native rules are opt-out here too:

// Only if you want the CEL path back.
Config config = Config.newBuilder().setEnableNativeRules(false).build();
Validator validator = ValidatorFactory.newBuilder().withConfig(config).build();

The relative gains are larger in Java, where the CEL baseline is considerably higher. Here are six of the benchmarks:

Allocations fall with runtime: by 93% for validateComplexSchema and 99% for validateInt32GT. Validator construction improves even more: buildBenchComplexSchema drops from 15 milliseconds to 37 microseconds. The full table is in #469.

All of this ships in protovalidate-java v1.3.0.

TypeScript

@bufbuild/protovalidate v1.3.0 is the newest of the three implementations (#162). Native rules are enabled by default; disableNativeRules forces everything through CEL:

const validator = createValidator({ disableNativeRules: true });

protovalidate-es interprets CEL in TypeScript. That makes its CEL baseline slower than Go’s or Java’s:

These results were gathered by running the protovalidate-es benchmark suite.

protovalidate-py 2.0

protovalidate-py v1 ran every rule through a pure-Python CEL interpreter. The complex-schema case took 36 milliseconds per message. That’s the latency budget for an entire API request, burned on validating a single message.

v2.0.0 replaces the interpreter with a native extension around protovalidate-cc, the C++ implementation. Creating and calling a validator works as before. The 36-millisecond case now takes 75 microseconds:

These are eight of the thirteen cases reported in the first benchmark table in #507.

Despite relying on Rust and C++, installing v2 doesn’t require a C++ or Rust toolchain. pip install protovalidate pulls a prebuilt wheel for Linux (glibc and musl), macOS, or Windows on x86-64 and arm64.

Upgrading from 1.x

The repository has been renamed from protovalidate-python to protovalidate-py, matching protobuf-py and connect-py. The PyPI package is still named protovalidate, and old repository URLs redirect automatically.

Version 2 uses protobuf-py, which we announced in July, as its primary Protobuf runtime instead of google.protobuf. You can still validate google.protobuf messages, but the Violation messages returned by the library are protobuf-py messages. The package also no longer depends on protobuf; declare that dependency directly if you previously received it transitively through protovalidate.

Most code that reads violations stays the same. There are two differences: nested messages can be None instead of automatically becoming empty messages, and JSON serialization uses protobuf-py’s API.

# Before
for violation in validator.collect_violations(message):
    print(violation.proto.rule_id)
    for element in violation.proto.field.elements:
        print(element)
    print(MessageToJson(violation.proto))
 
# After
for violation in validator.collect_violations(message):
    print(violation.proto.rule_id)
    if (field := violation.proto.field) is not None:
        for element in field.elements:
            print(element)
    print(violation.proto.to_json())

Generated buf.validate types now ship with the library. Unlike google.protobuf, protobuf-py has no process-wide registry where bundled types can conflict with a copy generated by your application. Your own generated types still need buf.validate as a dependency, so keep include_imports enabled for now. Proper import support is in progress.

What’s next

protovalidate-py 2.0 no longer interprets CEL in Python, but its standard rules still pass through CEL in the native layer. The next step is to bypass CEL on that path too.

Current releases:

• Go: buf.build/go/protovalidate v1.4.0, native rules on by default
• Python: pip install protovalidate v2.0.0, now backed by protovalidate-cc
• Java: build.buf:protovalidate v1.3.0, native rules on by default
• TypeScript: @bufbuild/protovalidate v1.3.0, native rules on by default

To try Protovalidate, see the quickstarts and playground at protovalidate.com. If you’re moving from protoc-gen-validate, start with the migration guide.