Salient Points for Protocol Buffers

Simply Explained (for Golang)

How to start .proto file?

syntax = "proto3";

Without the line above, the complier for protocol buffer will assume the syntax is based on proto2.

Specify Field Types

All fields are scalar type: string, float, int32, int64, uint32, uint64, sint32, sint64, fixed32 (always 4 bytes), fixed64 , bool, string, bytes etc.

message Foo { //this is a message type
   string query = 1;
   int64 large_number = 2;
   bool is_bar = 3;
   bool is_foo = 4;
}

Notice the use of underscore as convention for the field names. You can also have multiple message types in the same .proto file. Notice message name starts in big cap.

Default values

strings ----> empty string
bytes ------> empty bytes
bool -------> false
numeric types --> zero
enums ------> default value is the first defined enum value, which     
              must be 0

Composite Types — Enum

message SearchRequest {
  ...
  enum Corpus {
     UNIVERSAL = 0;
     WEB = 1;
  }
}

Every enum definition must contain a constant that maps to zero as its first element. This is so that the default value aforementioned can work.

If the same enum constants are used in other enum definition, you need the line: option allow_alias = true;

message SearchResponse {
   ... 
   enum EnumAllowinAlias {
      option allow_alias = true;
      UNIVERSAL = 0; //already used in previous enum Corpus
   }
}

Including Maps in definitions

message SearchWithMapParams {
   map<string, Project> projects = 5; // where Project is a message
}

Handling removal of fields

When you remove a field, you need to prevent the field numbers from being reused. What you need to do is to create reserved fields to “block” these field numbers. If not done, when other reuse the field numbers, severe compatibility can result.

message Foo {
   string query = 1;
   reserved 2, 3 to 4;
   reserved "is_bar", "is_foo", "large_number";
}

Nesting other message types

Other messages can be included as field types i.e. nested. For example:

message BigBar {
   repeated NestedBar inner_bar = 1;
}
message NestedBar {
   int32 weight = 1;
   string brand = 2;
   repeated string packaging = 3;
}

Here, the repeated keywork ensures you can use other messages as field types inside your message.

More nesting…

You can fall in love with nesting deeply :P

message BigBar {
   message NestedBar {
      int32 weight = 1;
      string brand = 2;
      repeated string packaging = 3;
   } 
   repeated NestedBar a_bar = 1; //repeated key again
}

Furthermore…

message Outer {
   message Middle {
     message Inner {
        int64 something = 1;
        string another = 2;
     }
  }
}

Importing .proto files

To help in nesting, you can import definitions from other .proto files

import "otherproject/some_protos.proto";

The protocol compiler searches for imported files in a set of folders specified on the protocol CLI using the flag,

-I or --proto_path

Placeholder .proto file

Imagine if you need to move your .proto file to a new file location. All your other .proto files importing the .proto file will need to be updated with a new import line. That will be a lot of work. Is there a better way?

To save the day, you can use the “import public” syntax such as —

new proto file:

//foo.proto
//All definitions of messages are moved here

old proto file to be modified:

//bar.proto
import public "foo.proto" //this line is added

other proto file using bar.proto:

//client.proto
import "old.proto"; //continue to import the old proto file

Oneof fields

If you have mutiple fields in a message and at most one field will be set at the same time, you can enforce filling one field by using the oneof feature.

message SlimMessage {
  oneof a_name {
    string this = 1;
    bool or_that = 4;
  }
}

Defining Services in .proto file

service AService {
   rpc Provide(Request) returns (Response);
}

Compiling to .pb.go

In go, the protobuf go compiler generates .pb.go file with a type for each message type in your file.

Generating your classes in go

protoc --proto_path=IMPORT_PATH --go_out=DST_DIR path/to/file.proto

GRPC support in go

If you are using grpc, you also need to generate the grpc go files with an additioanl flag of “ go-grpc_out=.”

protoc -I=. --go_out=./chunky --go-grpc_out=./chunky  ./chunky/chunk.proto

Above example generates the file into the relative folder of ./chunky