protocol buffers - store an double array, 1D, 2D and 3D

Question

How can be an array of double (1D) stored using protocol buffer? What about multi-dimensional (2D or 3D) dense arrays?

Answer 1

An array of double would be best stored via

repeated double foo = 5 [packed=true];

repeated makes it act as a list, allowing multiple items; packed avoids a header per item.

There is no direct support for rectangular (or higher) arrays in protobuf. The closest is to store something like:

repeated innerType foo = 5; // note, can't be "packed"

message innerType {
    repeated double foo = 1 [packed=true];
}

this is broadly akin to a jagged array, but with an element between each tier.

Answer 2

One could simply mimic the C/C++ memory layout:

message DoubleMatrix {
  required uint32 rows = 1;
  required uint32 cols = 2;
  repeated double data = 3 [packed=true];
}

To access the data, use data[i*cols+j] (row-major), or data[i+rows*j] (column-major). For square matrices only one of rows/cols has to be stored. Technically even in the rectangular case protobuf will know the length of the data, and the other value can be derived.

For ease of use one would probably wrap the Matrix in C++ with an Adapter class that allows access via double MatrixAdapter::get(int row, int col); it could also verify data_size()==rows()*cols().

Answer 3

If your main goal is a denser JSON representation, you could use the "well known type" [google.protobuf.ListValue][1] for the second dimension:

import "google/protobuf/struct.proto";
message DoubleMatrix {
  uint32 cols = 1;
  uint32 rows = 2;
  repeated google.protobuf.ListValue values = 3;
}

This will produce an array of arrays when marshaling the data to JSON.

{
  "cols": 2,
  "rows": 2,
  "values": [[1, 2], [3, 4]]
}

The downside is that the ListValue type internally uses an inner type that uses the one-of pattern for the values. This may be more cumbersome to handle in your source code than using your own inner type without all the magic.

[1] https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.ListValue