Drake
Drake C++ Documentation
YAML Serialization
Technical Notes

Overview

Drake provides infrastructure for reading YAML files into C++ structs, and writing C++ structs into YAML files. These functions are often used to read or write configuration data, but may also be used to serialize runtime data such as Diagram connections or OutputPort traces. Any C++ struct to be serialized must provide a Serialize() function to enumerate its fields.

Examples

Given a struct definition:

struct MyData {
...
double foo{0.0};
std::vector<double> bar;
};

Loading

Given a YAML data file:

foo: 1.0
bar: [2.0, 3.0]

We can use LoadYamlFile() to load the file:

int main() {
const MyData data = LoadYamlFile<MyData>("filename.yaml");
std::cout << fmt::format("foo = {:.1f}\n", data.foo);
std::cout << fmt::format("bar = {:.1f}\n", fmt::join(data.bar, ", "));
}

Output:

foo = 1.0
bar = 2.0, 3.0

Saving

We can use SaveYamlFile() to save to a file:

int main() {
MyData data{4.0, {5.0, 6.0}};
SaveYamlFile("filename.yaml", data);
}

Output file:

foo: 4.0
bar: [5.0, 6.0]

The following sections explain each of these steps in more detail, along with the customization options that are available for each one.

Implementing Serialize

Any C++ struct to be serialized must provide a templated Serialize() function that enumerates the fields. Typically, Serialize() will be implemented via a member function on the struct, but if necessary it can also be a free function obtained via argument-dependent lookup.

Here is an example of implementing a Serialize member function:

struct MyData {
template <typename Archive>
void Serialize(Archive* a) {
a->Visit(DRAKE_NVP(foo));
a->Visit(DRAKE_NVP(bar));
}
double foo{0.0};
std::vector<double> bar;
};

Structures can be arbitrarily nested, as long as each struct has a Serialize() function:

struct MoreData {
template <typename Archive>
void Serialize(Archive* a) {
a->Visit(DRAKE_NVP(baz));
a->Visit(DRAKE_NVP(quux));
}
std::string baz;
std::map<std::string, MyData> quux;
};

For background information about visitor-based serialization, see also the Boost.Serialization Tutorial, which served as the inspiration for Drake's design.

Style guide for Serialize

By convention, we place the Serialize function prior to the data members per the styleguide rule. Each data member has a matching Visit line in the Serialize function, in the same order as the member fields appear.

By convention, we declare all of the member fields as public, since they are effectively so anyway (because anything that calls the Serialize function receives a mutable pointer to them). The typical way to do this is to declare the data as a struct, instead of a class.

However, if the styleguide rule for struct vs class points towards using a class instead, then we follow that advice and make it a class, but we explicitly label the member fields as public. We also omit the trailing underscore from the field names, so that the Serialize API presented to the caller of the class is indifferent to whether it is phrased as a struct or a class. See drake::schema::Gaussian for an example of this situation.

If the member fields have invariants that must be immediately enforced during de-serialization, then we add invariant checks to the end of the Serialize() function to enforce that, and we mark the class fields private (adding back the usual trailing underscore). See drake::math::BsplineBasis for an example of this situation.

Built-in types

Drake's YAML I/O functions provide built-in support for many common types:

YAML correspondence

The simple types (std::string, bool, floating-point number, integers) all serialize to a Scalar node in YAML.

The array-like types (std::array, std::vector, Eigen::Matrix) all serialize to a Sequence node in YAML.

User-defined structs and the native maps (std::map, std::unordered_map) all serialize to a Mapping node in YAML.

For the treatment of std::optional, refer to Nullable types, below. For the treatment of std::variant, refer to Sum types, below.

Reading YAML files

Use LoadYamlFile() or LoadYamlString() to de-serialize YAML-formatted string data into C++ structure.

It's often useful to write a helper function to load using a specific schema, in this case the MyData schema:

MyData LoadMyData(const std::string& filename) {
return LoadYamlFile<MyData>(filename);
}
int main() {
const MyData data = LoadMyData("filename.yaml");
std::cout << fmt::format("foo = {:.1f}\n", data.foo);
std::cout << fmt::format("bar = {:.1f}\n", fmt::join(data.bar, ", "));
}

Sample data in filename.yaml:

foo: 1.0
bar: [2.0, 3.0]

Sample output:

foo = 1.0
bar = 2.0, 3.0

There is also an option to load from a top-level child in the document:

data_1:
foo: 1.0
bar: [2.0, 3.0]
data_2:
foo: 4.0
bar: [5.0, 6.0]
MyData LoadMyData2(const std::string& filename) {
return LoadYamlFile<MyData>(filename, "data_2");
}

Sample output:

foo = 4.0
bar = 5.0, 6.0

Defaults

The LoadYamlFile() function offers a defaults = ... argument. When provided, the yaml file's contents will overwrite the provided defaults, but any fields that are not mentioned in the yaml file will remain intact at their default values.

When merging file data atop any defaults, any std::map or std::unordered_map collections will merge the contents of the file alongside the existing map values, keeping anything in the default that is unchanged. Any other collections such as std::vector are entirely reset, even if they already had some values in place (in particular, they are not merely appended to).

Merge keys

YAML's "merge keys" (https://yaml.org/type/merge.html) are supported during loading. (However, the graph-aliasing relationship implied by nominal YAML semantics is not implemented; the merge keys are fully deep-copied.)

Example:

_template: &common_foo
foo: 1.0
data_1:
<< : *common_foo
bar: [2.0, 3.0]
data_2:
<< : *common_foo
bar: [5.0, 6.0]

Writing YAML files

Use SaveYamlFile() or SaveYamlString() to output a YAML-formatted serialization of a C++ structure.

The serialized output is always deterministic, even for unordered datatypes such as std::unordered_map.

struct MyData {
template <typename Archive>
void Serialize(Archive* a) {
a->Visit(DRAKE_NVP(foo));
a->Visit(DRAKE_NVP(bar));
}
double foo{0.0};
std::vector<double> bar;
};
int main() {
MyData data{1.0, {2.0, 3.0}};
std::cout << SaveYamlString(data, "root");
return 0;
}

Output:

root:
foo: 1.0
bar: [2.0, 3.0]

Document root

Usually, YAML reading or writing requires a serializable struct that matches the top-level YAML document. However, sometimes it's convenient to parse the document in the special case of a C++ std::map at the top level, without the need to define an enclosing struct.

data_1:
foo: 1.0
bar: [2.0, 3.0]
data_2:
foo: 4.0
bar: [5.0, 6.0]
std::map<std::string, MyData> LoadAllMyData(const std::string& filename) {
return LoadYamlFile<std::map<std::string, MyData>>(filename);
}

Nullable types (std::optional)

When a C++ field of type std::optional is present, then:

Sum types (std::variant)

When reading into a std::variant<>, we match its YAML tag to the shortened C++ class name of the variant selection. For example, to read into this sample struct:

struct Foo {
template <typename Archive>
void Serialize(Archive* a) {
a->Visit(DRAKE_NVP(data));
}
std::string data;
};
struct Bar {
template <typename Archive>
void Serialize(Archive* a) {
a->Visit(DRAKE_NVP(value));
}
std::variant<std::string, double, Foo> value;
};

Some valid YAML examples are:

# For the first type declared in the variant<>, the tag is optional.
bar:
value: hello
# YAML has built-in tags for string, float, int.
bar2:
value: !!str hello
# For any other type within the variant<>, the tag is required.
bar3:
value: !!float 1.0
# User-defined types use a single exclamation point.
bar4:
value: !Foo
data: hello