Configuration: Part 1

Jun 20

Introduction

Barely any application runs without a user-provided configuration. At the very least, a distributed system needs to allow users to tell how to reach remote nodes in the network or how other nodes in the network may reach this node.

CAF makes it easy to configure various aspects of applications and also allows users to fine-tune CAF components.

Config Values

The class config_value sits at the core of the configuration API. At a first glance, this type offers a variant-like interface and it natively stores one of the following types:

none
bool
int64_t
double
std::string
timespan
uri
std::vector<config_value>
dictionary<config_value>

Note that we cannot implement config_value as a simple variant, because it is a recursive type! This set of types may be stored internally but otherwise play a secondary role.

Basics

Much like a regular std::variant, a config_value accepts any input in its constructor that it can convert to one of its types. For example, we can define the three config values x, y and z as integer, floating point and string as follows:

Source Code

auto x = config_value{1};
auto y = config_value{2.0};
auto z = config_value{"three"};
println("x = ", x);
println("y = ", y);
println("z = ", z);

Output

x = 1
y = 2
z = three

Users may treat config_value as a simple sum type similar to std::variant by using the functions get, get_if, and holds_alternative. However, CAF offers a much more powerful and yet simpler way to inspect the content of a config_value: the function pair get_as and get_or.

On-the-fly Conversions with `get_as`

Configuration values never exist in a vacuum. They typically represent input by the user. Usually, a config_value was generated by the native CAF parser. However, what if we are using some other input format that cannot natively express all types of config_value? JSON, for example, can represent a timespan only by encoding it as a string. Does that mean one somehow needs to perform guesswork when importing a configuration from another data source? Luckily, no.

The function get_as is the Swiss Army knife of type conversions. The function takes one template parameter T that represents the target type and it returns expected<T>. The function may attempt type conversions, which is why it cannot just return T. An expected<T> represents an optional value, but unlike std::optional it carries an error if no value exists. Without further ado, let us see how get_as may be used:

Source Code

auto x = config_value{"5s"};
if (auto ts = get_as<timespan>(x))
  println("ts = ", *ts);
else
  println("oops: ", ts.error());

Output

ts = 5s

CAF knows how to convert the string "5s" into a timespan. It also knows how to convert numbers in case a user typed in an integer while the system expects a double. Basically, CAF may perform all sorts of type conversions as long as the target type may reasonably hold the value. If we again take a look at the native types of config_value: the only integer type is int64_t! So, what if we try converting it to smaller integer types?

Source Code

auto x = config_value{42};
if (auto narrow_x = get_as<uint16_t>(x))
  println("narrow_x = ", *narrow_x);
else
  println("oops: ", narrow_x.error());
auto y = config_value{1'000'000};
if (auto narrow_y = get_as<uint16_t>(y))
  println("narrow_y = ", *narrow_y);
else
  println("oops: ", narrow_y.error());

Output

narrow_x = 42
oops: caf::sec::conversion_failed("narrowing error")

The function get_as only performs safe conversions. In this case, by performing bound checks. Hence, only the conversion for 42 succeeds, because 1,000,000 does not fit into 16 bit!

While converting between builtin types is neat, the real power of get_as comes from the fact that it tightly integrates with the type inspection API! We will delve into more detail on this later, but here comes an examples showing the gist of it. Given the following definition of point_2d:

struct point_2d {
  int32_t x;
  int32_t y;
};

template <class Inspector>
bool inspect(Inspector& f, point_2d& x) {
  return f.object(x).fields(f.field("x", x.x), f.field("y", x.y));
}

Then we may read a point_2d directly from a config_value! There are a couple of ways the user may input a point, but for now, we leave it at this example and come back to why (and how) this works later:

Source Code

auto x = config_value{"{x = 12, y = 21}"} ;
if (auto point = get_as<point_2d>(x))
  println("got a point: (", point->x, ", ", point->y, ")");
else
  println("oops: ", point.error());

Output

got a point: (12, 21)

Conversions with Fallbacks with `get_or`

In many cases, configuration values allow users to override hard-coded default values. With get_as, this would mean writing an if-else block every time to either evaluate to user input if present or the hard-coded defaults otherwise.

In order to provide a streamlined interface for this common use case, CAF offers get_or. In the most simple case, get_or simply takes two arguments: a config value plus the fallback value.

Source Code

auto x = config_value{42};
println("(1) get x or 10: ", get_or(x, 10));
auto y = config_value{"hello world"} ;
println("(2) get y or 10: ", get_or(y, 10));

Output

(1) get x or 10: 42
(2) get y or 10: 10

When used in this way, the type of the fallback value also determines the result of get_or. By extension, this also means that CAF passes this type to get_as internally.

There are cases where it makes sense to have different types for fallback and result type. For example, types that allocate memory (such as std::string) but that may be constructible from constant expressions. To illustrate the mechanics behind this, consider the following type definition for int64_wrapper:

struct int64_wrapper {
  int64_t value;
};

template <class Inspector>
bool inspect(Inspector& f, int64_wrapper& x) {
  return f.apply(x.value);
}

The only important thing to notice about int64_wrapper is that we can construct it from an integer, e.g., int64_wrapper{42}. This allows us to illustrate how to use the optional template parameter T:

Source Code

auto x = config_value{"hello world"};
println("(1) get x or 42: ", get_or<int64_wrapper>(x, 42));

Output

(1) get x or 42: 42

When specifying the return type for get_or, CAF either returns the result of get_as<T> or it returns T{fallback}. Hence, T must provide a constructor that initializes the return value from the fallback.

Lists

Aside from storing single values, the type config_value can also store lists. Each element in the list is a config_value again. Hence, we can nest lists arbitrarily.

Creating Lists

The constructor of config_value is explicit to stop the compiler from automatically converting values to config_value everywhere. However, this makes initializing a list of config values cumbersome:

// config_value::list xs{1, 2, 3}; -- will not compile
config_value::list xs{config_value{1}, config_value{2}, config_value{3}};
println(xs);

The above snippet prints [1, 2, 3], but it requires a lot of boilerplate code to initialize the list. Constructing a config value from a list adds even more boilerplate code, because we need to wrap the entire initialization again:

config_value xs{config_value::list{config_value{1}, config_value{2},
                                   config_value{3}}};
println(xs);

The second snippet also prints [1, 2, 3]. The only difference is that xs is a config value holding a list this time. To make working with lists easier, CAF offers the factory function make_config_value_list:

Source Code

auto xs = make_config_value_list(1, 2, 3);
println(xs);

Output

[1, 2, 3]

Since config value lists are heterogeneous, we can also construct a list with mixed types:

Source Code

auto xs = make_config_value_list(1, "two", 3.0);
println(xs);

Output

[1, "two", 3]

Using `as_list`

Sometimes, we receive a config value and need to convert it to a list before continuing. If the value already contains a list then we want to make sure not to override it, because we want to keep existing entries. For this particular use case, config_value provides the member function as_list:

Source Code

auto x = config_value{};
auto y = config_value{42};
auto z = make_config_value_list(1, 2, 3);
println("(1) x as list = ", x.as_list());
println("(2) y as list = ", y.as_list());
println("(3) z as list = ", z.as_list());

Output

(1) x as list = []
(2) y as list = [42]
(3) z as list = [1, 2, 3]

In the first case, we convert nothing to a list. The only way CAF could perform this conversion is by creating an empty list. In the second case, the variable y contains the integer 42. Here, CAF simply lifts the single value into a list with one element. Lastly, z already contains a list, so CAF can simply return the stored list in this case without any conversion.

Working with as_list avoids unnecessary boilerplate code, as we can see in the following example that creates a list of three lists of three integers each:

Source Code

config_value x;
println("(1) x = ", x);
auto& ls = x.as_list();
println("(2) x = ", x);
ls.resize(3); // Fills the list with three null elements.
println("(3) x = ", x);
auto num = int64_t{0};
for (auto& element : ls) {
  auto& nested = element.as_list();
  nested.emplace_back(num);
  for (++num; num % 3 != 0; ++num)
    nested.emplace_back(num);
}
println("(4) x = ", x);

Output

(1) x = null
(2) x = []
(3) x = [null, null, null]
(4) x = [[0, 1, 2], [3, 4, 5], [6, 7, 8]]

Running the example prints x four times:

The first line prints the default-constructed x. As we can see, initially it is just null.
The second time we print x is after we have called as_list on it. This member function converts the config value to a list. Hence, the second line shows [].
After resizing the vector, we have three null objects in the list.
Finally, the fourth line displays our final result after filling x with the desired content in the for-loop.

Converting to Homogeneous Lists

We already know how to conveniently create a list of config values using make_config_value_list:

auto xs = make_config_value_list(1, 2, 3);
println(xs);

Our container xs from the snippet above consists of integers only. Just like get_or converts to single values, the function also knows how to convert the config value lists to homogeneous list types:

auto xs = make_config_value_list(1, 2, 3);
if (auto ints = get_as<std::vector<int32_t>>(xs)) {
  println("xs is a vector of int: ", *ints);
} else {
  println("xs is not a vector of int: ", ints.error());
}

Note that CAF does not limit its users to std::vector. The automatic unboxing supports all types that behave like STL containers such as std::vector, std::list, std::set and std::unordered_set

Converting to Tuples

Because config_value may hold a variety of types, config_value::list may hold elements of different types. Such lists cannot convert to std::vector or similar data structures except when converting to a type that can construct from different types. However, heterogeneous lists can convert to tuples:

Source Code

auto xs = make_config_value_list(1, "two", 3.3);
if (auto tup = get_as<std::tuple<int32_t, std::string, double>>(xs)) {
  println("tup: ", *tup);
} else {
  println("oops: ", tup.error());
}

Output

tup: [1, "two", 3.3]

This also applies to std::array, which is just a special case of tuple (because all elements have the same type). To CAF, every type that specializes std::tuple_size etc. is treated in the same way. Hence, get_as and get_or also support std::pair and std::array:

Source Code

auto xs = make_config_value_list(1, "two");
if (auto tup = get_as<std::pair<int32_t, std::string>>(xs)) {
  println("tup: ", *tup);
} else {
  println("oops: ", tup.error());
}
auto ys = make_config_value_list(1, 2, 3);
if (auto arr = get_as<std::array<int32_t, 3>>(ys)) {
  println("arr: ", *arr);
} else {
  println("oops: ", arr.error());
}

Output

tup: [1, "two"]
arr: [1, 2, 3]

Dictionaries and Settings

In CAF, settings is an alias for dictionary<config_value>. A dictionary is a map with string keys. Semantically, a dictionary<T> is equivalent to a std::map<std::string, T>.

Using `as_dictionary`

Analogous to as_list, CAF also offers an as_dictionary member function that returns the config_value as a dictionary, converting it to a dictionary if needed.

Source Code

auto x = config_value{};
println("(1) x = ", x);
auto& dict = x.as_dictionary();
println("(2) x = ", x);
dict.emplace("foo", "bar");
dict.emplace("int-value", 42);
dict.emplace("int-value", 23);
println("(3) x = ", x);

Output

(1) x = null
(2) x = {}
(3) x = {foo = "bar", "int-value" = 42}

Running the example prints x three times again:

Initially, x is null once again.
After calling as_dictionary, x now is an empty dictionary.
Just like std::map, calling emplace adds new entries to the dictionary. The first value "wins", so the dictionary associated the value 42 to the key int-value (not 23).

Just like with as_list, CAF tries to convert the content of a config value to a dictionary. However, dictionaries require two values: key and value. Hence, the only data structures that CAF can convert into dictionaries ares lists of lists, where each nested list has exactly two values, and strings representing if parsing their content results in a valid dictionary:

Source Code

auto x = make_config_value_list(make_config_value_list("one", 1),
                                make_config_value_list("two", 2),
                                make_config_value_list("three", 3));
println("(1) x = ", x);
x.as_dictionary();
println("(2) x = ", x);
auto y = config_value{"{answer = 42}"};
println("(3) y = ", y);
y.as_dictionary();
println("(4) y = ", y);

Output

(1) x = [["one", 1], ["two", 2], ["three", 3]]
(2) x = {one = 1, three = 3, two = 2}
(3) y = {answer = 42}
(4) y = {answer = 42}

The automatic parsing of strings to dictionaries also enables the conversion from strings to point_2d we observed earlier. The inspect function exposes the fields inside an object to CAF and we can naturally translate a dictionary to an object by interpreting the keys as field names. So as long as a config_value represents a dictionary, CAF can use the inspect function for trying to construct a C++ object.

Converting to Regular Map Types

At this point, you can probably guess what our next example illustrates. Yes, get_as once more!

Source Code

auto x = config_value{};
auto& dict = x.as_dictionary();
dict.emplace("1", 10);
dict.emplace("2", 20);
dict.emplace("3", 30);
if (auto m1 = get_as<std::map<double, int32_t>>(x))
  println("m1: ", *m1);
else
  println("oops: ", m1.error());
if (auto m2 = get_as<std::unordered_map<int32_t, double>>(x))
  println("m2: ", *m2);
else
  println("oops: ", m2.error());

Output

m1: {1 = 10, 2 = 20, 3 = 30}
m2: {3 = 30, 2 = 20, 1 = 10}

As you can see, get_as also performs "deep" conversions by converting the string key of the dictionary to another type. In this case, CAF converts the strings to integers or floating point numbers as needed.

Next Up

Settings (nested dictionaries) are the primary way CAF stores configurations. This is why settings follow a couple of conventions to make picking values in nested dictionaries easier. We explore more on this topic in part2.

Dominik Charousset https://charousset.de/dominik

Configuration: Part 1

Introduction

Config Values

Basics

Source Code

Output

On-the-fly Conversions with get_as

Source Code

Output

Source Code

Output

Source Code

Output

Conversions with Fallbacks with get_or

Source Code

Output

Source Code

Output

Lists

Creating Lists

Source Code

Output

Source Code

Output

Using as_list

Source Code

Output

Source Code

Output

Converting to Homogeneous Lists

Converting to Tuples

Source Code

Output

Source Code

Output

Dictionaries and Settings

Using as_dictionary

Source Code

Output

Source Code

Output

Converting to Regular Map Types

Source Code

Output

Next Up

Implementing Actors: Part 1

On-the-fly Conversions with `get_as`

Conversions with Fallbacks with `get_or`

Using `as_list`

Using `as_dictionary`