Asking Jev questions with Protobuf

System One models like Jev evaluate questions without generating pages of text. Instead, the answers come back as choices, scores, and probabilities that your code can use directly.

What does this have to do with Protobuf? We wanted a Go client to experiment with Jev (along with Jev-compatible APIs like Cloudflare’s Clef), and there wasn’t one. In Python, we also wanted the type checker to catch typos in answer names. So we wrote protoc-gen-jev, a protoc plugin that defines Jev questions in Protobuf and generates typed clients and response types for Go, TypeScript, and Python.

One disclaimer: protoc-gen-jev is an experiment, not a supported Buf product. We’re sharing it as an example of using custom options to generate clients for something other than traditional RPC services.

How to use the plugin

First, you write a .proto file describing the input you’ll send to Jev and the answers you want back, with each question attached to its answer field as an option. Then you run buf generate (see the README for setup) and call the generated code from Go, TypeScript, or Python.

Write the schema

Let’s triage support tickets. Pass Jev the ticket’s subject and body in the request message:

syntax = "proto3";
 
package support.v1;
 
message TriageTicketRequest {
  string subject = 1;
  string body = 2;
}

On the response message, (jev.v1.field) options define the questions Jev needs to answer:

import "jev/v1/options.proto";
 
enum Department {
  DEPARTMENT_UNSPECIFIED = 0;
  DEPARTMENT_BILLING = 1;
  DEPARTMENT_TECHNICAL = 2;
  DEPARTMENT_SALES = 3;
}
 
message TriageTicketResponse {
  Department department = 1 [
    (jev.v1.field).instructions = "Which team should handle this ticket?",
    (jev.v1.field).choice = {
      criteria: [
        { key: "DEPARTMENT_BILLING", value: "Payment or subscription issues" },
        { key: "DEPARTMENT_TECHNICAL", value: "Bugs or integration problems" },
        { key: "DEPARTMENT_SALES", value: "Pricing or account questions" }
      ]
    }
  ];
  int32 frustration = 2 [
    (jev.v1.field).instructions = "How frustrated does the customer appear?",
    (jev.v1.field).score = {
      levels: [
        { value: 1, description: "Calm, just stating facts" },
        { value: 2, description: "Frustrated but civil" },
        { value: 3, description: "Very angry, strong language" }
      ]
    }
  ];
  bool urgent = 3 [
    (jev.v1.field).instructions = "The message conveys urgency or time-sensitivity",
    (jev.v1.field).noul = { threshold: 0.7 }
  ];
}

Jev has three primitives: choice, score, and noul (a yes/no probability). The generated client maps their raw outputs onto your schema types: string labels become Department enums, rubric positions map to 1..3 integers, and probabilities at or above 0.7 become true. You can also omit instructions and put the question text in the field’s leading comment.

To get a client method, define a unary RPC:

service SupportTriageService {
  rpc TriageTicket(TriageTicketRequest) returns (TriageTicketResponse);
}

This RPC glues our input to our output, and it also gives us a way to name these Jev queries. In our case, we are only using this RPC definition to build a client. At runtime, that client serializes the request message to JSON and sends it to the API as the state to evaluate, alongside the questions built from the response fields.

Call the generated client

protoc-gen-jev takes SupportTriageService and generates a client that looks very similar to any other Protobuf RPC service. We tested it against Cloudflare’s Clef-omni on Workers AI. The Go client defaults to TypeSafe’s hosted API, so we point BaseURL at the clef-omni endpoint:

jevClient := jev.NewClient(os.Getenv("CLOUDFLARE_API_TOKEN"))
jevClient.BaseURL = fmt.Sprintf(
    "https://api.cloudflare.com/client/v4/accounts/%s/ai/run/@cf/cloudflare/clef-omni",
    os.Getenv("CLOUDFLARE_ACCOUNT_ID"),
)
client := supportv1.NewJevSupportTriageService(jevClient)

Here’s what calling the generated API looks like:

decision, err := client.TriageTicket(ctx, &supportv1.TriageTicketRequest{
    Subject: "Checkout API returning 500s since 09:10 UTC",
    Body: `Hi team, since about 09:10 UTC every call to POST /v1/checkout
returns a 500 and our storefront can't take orders. Happy to share
request IDs if that helps. Thanks for looking into it.`,
})
if err != nil {
    return err
}
 
if decision.GetUrgent() && decision.GetDepartment() == supportv1.Department_DEPARTMENT_TECHNICAL {
    escalate(decision)
}

Now let’s compare that outage ticket against a more nuanced example, a customer with strong opinions about a recent dark mode rollout:

Subject: Your new dark mode is AWFUL

Body: Seriously, who approved this?? The new dark mode makes the settings page a hideous shade of grey and I HATE it. Whoever designed this should be embarrassed. Change it back.

Here’s how Clef-omni evaluated both tickets:

This shows that user frustration can be high while the ticket is marked not urgent. Clef-omni performs well in this example!

Detail messages

In the previous example we only pulled back the answers. But the Jev API does return some additional information, like the probability behind each answer and metadata about the call. To get that for the urgent field, change its type from bool to jev.v1.Noul and add a jev.v1.Meta field for the call metadata:

import "jev/v1/response.proto";
 
message TriageTicketResponse {
  // department and frustration are unchanged
  jev.v1.Noul urgent = 3 [
    (jev.v1.field).instructions = "The message conveys urgency or time-sensitivity",
    (jev.v1.field).noul = { threshold: 0.7 }
  ];
  jev.v1.Meta meta = 99;
}

urgent.value is the boolean, and urgent.probability is the probability used to calculate that boolean (in Go, decision.GetUrgent().GetValue() and GetProbability()). Clef-omni returned 0.92 for the checkout outage and 0.07 for the dark-mode complaint. This extra information about the decision is great for tuning and debugging.

jev.v1.Score and jev.v1.Choice do the same for numeric and enum fields, and include an optional confidence when the API returns one. As for meta, the generated client fills in jev.v1.Meta with the model name and token usage whenever a response message includes that field.

Try it

Pick one of the example projects in Go, TypeScript, or Python. The README has the installation and generation setup, including the BSR dependency. You’ll need to run the plugin locally for now; we haven’t published a BSR remote plugin or BSR-generated SDK. The reference covers the options for writing your own questions.

If you try it out, tell us on Buf Slack whether it’s useful to you.