An asynchronous function starts an operation which may continue long after the function returns: The function itself returns a future<T> almost immediately, but it may take a while until this future is resolved.
When such an asynchronous operation needs to operate on existing objects, or to use temporary objects, we need to worry about the lifetime of these objects: We need to ensure that these objects do not get destroyed before the asynchronous function completes (or it will try to use the freed object and malfunction or crash), and to also ensure that the object finally get destroyed when it is no longer needed (otherwise we will have a memory leak). Seastar offers a variety of mechanisms for safely and efficiently keeping objects alive for the right duration. In this section we will explore these mechanisms, and when to use each mechanism.
The most straightforward way to ensure that an object is alive when a continuation runs and is destroyed afterwards is to pass its ownership to the continuation. When continuation owns the object, the object will be kept until the continuation runs, and will be destroyed as soon as the continuation is not needed (i.e., it may have run, or skipped in case of exception and then() continuation).
We already saw above that the way for a continuation to get ownership of an object is through capturing:
Here the continuation captures the value of i. In other words, the continuation includes a copy of i. When the continuation runs 10ms later, it will have access to this value, and as soon as the continuation finishes its object is destroyed, together with its captured copy of i. The continuation owns this copy of i.
Capturing by value as we did here - making a copy of the object we need in the continuation - is useful mainly for very small objects such as the integer in the previous example. Other objects are expensive to copy, or sometimes even cannot be copied. For example, the following is not a good idea:
seastar::future<> slow_op(std::vector<int> v) {
// this makes another copy of v:
return seastar::sleep(10ms).then([v] { /* do something with v */ });
}This would be inefficient - as the vector v, potentially very long, will be copied and the copy will be saved in the continuation. In this example, there is no reason to copy v - it was anyway passed to the function by value and will not be used again after capturing it into the continuation, as right after the capture, the function returns and destroys its copy of v.
For such cases, C++14 allows moving the object into the continuation:
seastar::future<> slow_op(std::vector<int> v) {
// v is not copied again, but instead moved:
return seastar::sleep(10ms).then([v = std::move(v)] { /* do something with v */ });
}Now, instead of copying the object v into the continuation, it is moved into the continuation. The C++11-introduced move constructor moves the vector’s data into the continuation and clears the original vector. Moving is a quick operation - for a vector it only requires copying a few small fields such as the pointer to the data. As before, once the continuation is dismissed the vector is destroyed - and its data array (which was moved in the move operation) is finally freed.
TODO: talk about temporary_buffer as an example of an object designed to be moved in this way.
In some cases, moving the object is undesirable. For example, some code keeps references to an object or one of its fields and the references become invalid if the object is moved. In some complex objects, even the move constructor is slow. For these cases, C++ provides the useful wrapper std::unique_ptr<T>. A unique_ptr<T> object owns an object of type T allocated on the heap. When a unique_ptr<T> is moved, the object of type T is not touched at all - just the pointer to it is moved. An example of using std::unique_ptr<T> in capture is:
seastar::future<> slow_op(std::unique_ptr<T> p) {
return seastar::sleep(10ms).then([p = std::move(p)] { /* do something with *p */ });
}std::unique_ptr<T> is the standard C++ mechanism for passing unique ownership of an object to a function: The object is only owned by one piece of code at a time, and ownership is transferred by moving the unique_ptr object. A unique_ptr cannot be copied: If we try to capture p by value, not by move, we will get a compilation error.
The technique we described above - giving the continuation ownership of the object it needs to work on - is powerful and safe. But often it becomes hard and verbose to use. When an asynchronous operation involves not just one continuation but a chain of continations that each needs to work on the same object, we need to pass the ownership of the object between each successive continuation, which can become inconvenient. It is especially inconvenient when we need to pass the same object into two seperate asynchronous functions (or continuations) - after we move the object into one, the object needs to be returned so it can be moved again into the second. E.g.,
seastar::future<> slow_op(T o) {
return seastar::sleep(10ms).then([o = std::move(o)] {
// first continuation, doing something with o
...
// return o so the next continuation can use it!
return std::move(o);
}).then([](T o)) {
// second continuation, doing something with o
...
});
}This complexity arises because we wanted asynchronous functions and continuations to take the ownership of the objects they operated on. A simpler approach would be to have the caller of the asynchronous function continue to be the owner of the object, and just pass references to the object to the various other asynchronous functions and continuations which need the object. For example:
seastar::future<> slow_op(T& o) { // <-- pass by reference
return seastar::sleep(10ms).then([&o] {// <-- capture by reference
// first continuation, doing something with o
...
}).then([&o]) { // <-- another capture by reference
// second continuation, doing something with o
...
});
}This approach raises a question: The caller of slow_op is now responsible for keeping the object o alive while the asynchronous code started by slow_op needs this object. But how will this caller know how long this object is actually needed by the asynchronous operation it started?
The most reasonable answer is that an asynchronous function may need access to its parameters until the future it returns is resolved - at which point the asynchronous code completes and no longer needs access to its parameters. We therefore recommend that Seastar code adopt the following convention:
Whenever an asynchronous function takes a parameter by reference, the caller must ensure that the referred object lives until the future returned by the function is resolved.
Note that this is merely a convention suggested by Seastar, and unfortunately nothing in the C++ language enforces it. C++ programmers in non-Seastar programs often pass large objects to functions as a const reference just to avoid a slow copy, and assume that the called function will not save this reference anywhere. But in Seastar code, that is a dangerous practice because even if the asynchronous function did not intend to save the reference anywhere, it may end up doing it implicitly by passing this reference to another function and eventually capturing it in a continuation.
It would be nice if future versions of C++ could help us catch incorrect uses of references. Perhaps we could have a tag for a special kind of reference, an “immediate reference” which a function can use use immediately (i.e, before returning a future), but cannot be captured into a continuation.
With this convention in place, it is easy to write complex asynchronous functions functions like slow_op which pass the object around, by reference, until the asynchronous operation is done. But how does the caller ensure that the object lives until the returned future is resolved? The following is wrong:
It is wrong because the object obj here is local to the call of f, and is destroyed as soon as f returns a future - not when this returned future is resolved! The correct thing for a caller to do would be to create the object obj on the heap (so it does not get destroyed as soon as f returns), and then run slow_op(obj) and when that future resolves (i.e., with .finally()), destroy the object.
Seastar provides a convenient idiom, do_with() for doing this correctly:
seastar::future<> f() {
return seastar::do_with(T(), [] (auto& obj) {
// obj is passed by reference to slow_op, and this is fine:
return slow_op(obj);
}
}do_with will do the given function with the given object alive.
do_with saves the given object on the heap, and calls the given lambda with a reference to the new object. Finally it ensures that the new object is destroyed after the returned future is resolved. Usually, do_with is given an rvalue, i.e., an unnamed temporary object or an std::move()ed object, and do_with moves that object into its final place on the heap. do_with returns a future which resolves after everything described above is done (the lambda’s future is resolved and the object is destroyed).
For convenience, do_with can also be given multiple objects to hold alive. For example here we create two objects and hold alive them until the future resolves:
seastar::future<> f() {
return seastar::do_with(T1(), T2(), [] (auto& obj1, auto& obj2) {
return slow_op(obj1, obj2);
}
}While do_with can the lifetime of the objects it holds, if the user accidentally makes copies of these objects, these copies might have the wrong lifetime. Unfortunately, a simple typo like forgetting an “&” can cause such accidental copies. For example, the following code is broken:
seastar::future<> f() {
return seastar::do_with(T(), [] (T obj) { // WRONG: should be T&, not T
return slow_op(obj);
}
}In this wrong snippet, obj is mistakenly not a reference to the object which do_with allocated, but rather a copy of it - a copy which is destroyed as soon as the lambda function returns, rather than when the future it returns resolved. Such code will most likely crash because the object is used after being freed. Unfortunately the compiler will not warn about such mistakes. Users should get used to always using the type “auto&” with do_with - as in the above correct examples - to reduce the chance of such mistakes.
For the same reason, the following code snippet is also wrong:
seastar::future<> slow_op(T obj); // WRONG: should be T&, not T
seastar::future<> f() {
return seastar::do_with(T(), [] (auto& obj) {
return slow_op(obj);
}
}Here, although obj was correctly passed to the lambda by reference, we later acidentally passed slow_op() a copy of it (because here slow_op takes the object by value, not by reference), and this copy will be destroyed as soon as slow_op returns, not waiting until the returned future resolves.
When using do_with, always remember it requires adhering to the convention described above: The asynchronous function which we call inside do_with must not use the objects held by do_with after the returned future is resolved. It is a serious use-after-free bug for an asynchronous function to return a future which resolves while still having background operations using the do_with()ed objects.
In general, it is rarely a good idea for an asynchronous function to resolve while leaving behind background operations - even if those operations do not use the do_with()ed objects. Background operations that we do not wait for may cause us to run out of memory (if we don’t limit their number) and make it difficult to shut down the application cleanly.
In the beginning of this chapter, we already noted that capturing a copy of an object into a continuation is the simplest way to ensure that the object is alive when the continuation runs and destoryed afterwards. However, complex objects are often expensive (in time and memory) to copy. Some objects cannot be copied at all, or are read-write and the continuation should modify the original object, not a new copy. The solution to all these issues are reference counted, a.k.a. shared objects:
A simple example of a reference-counted object in Seastar is a seastar::file, an object holding an open file object (we will introduce seastar::file in a later section). A file object can be copied, but copying does not involve copying the file descriptor (let alone the file). Instead, both copies point to the same open file, and a reference count is increased by 1. When a file object is destroyed, the file’s reference count is decreased by one, and only when the reference count reaches 0 the underlying file is actually closed.
The fact that file objects can be copied very quickly and all copies actually point to the same file, make it very convinient to pass them to asynchronous code; For example,
seastar::future<uint64_t> slow_size(file f) {
return seastar::sleep(10ms).then([f] {
return f.size();
});
// NOTE: something is wrong here! This will be explained below!
}Note how calling slow_size is as simple as calling slow_size(f), passing a copy of f, without needing to do anything special to ensure that f is only destroyed when no longer needed. That simply happens naturally when nothing refers to f any more.
However, there is one complication. The above example is actually wrong, as the comment at the end of the function suggested. The problem is that the f.size() call started an asynchronous operation on f (the file’s size may be stored on disk, so not immediately available) and yet at this point nothing is holding a copy of f… The method call does not increment the reference count of the object even if it an asynchronous method. (Perhaps this something we should rethink?)
So we need to ensure that something does hold on to another copy of f until the asynchronous method call completes. This is how we typically do it:
seastar::future<uint64_t> slow_size(file f) {
return seastar::sleep(10ms).then([f] {
return f.size().finally([f] {});
});
}What we see here is that f is copied not only to the continuation which runs f.size(), but also into a continuation (a finally) which will run after it. So as long as f.size() does not complete, that second continuation holds f alive. Note how the second continuation seems to have no code (just a {}). But the important thing is that the compiler automatically adds to it code to destroy its copy of f (and potentially the entire file if this reference count went down to 0).
The reference counting has a run-time cost, but it is usually very small; It is important to remember that Seastar objects are always used by a single CPU only, so the reference-count increment and decrement operations are not the slow atomic operations often used for reference counting, but just regular CPU-local integer operations. Moreover, judicious use of std::move() and the compiler’s optimizer can reduce the number of unnecessary back-and-forth increment and decrement of the reference count.
C++11 offers a standard way of creating reference-counted shared objects - using the template std::shared_ptr<T>. A shared_ptr can be used to wrap any type into a reference-counted shared object like seastar::file above. However, the standard std::shared_ptr was designed with multi-threaded applications in mind so it uses slow atomic increment/decrement operations for the reference count which we already noted is unnecessary in Seastar. For this reason Seastar offers its own single-threaded implementation of this template, seastar::shared_ptr<T>. It is similar to std::shared_ptr<T> except no atomic operations are used.
Additionally, Seastar also provides an even lower overhead variant of shared_ptr: seastar::lw_shared_ptr<T>. The full-featured shared_ptr is complicated by the need to support polymorphic types correctly (a shared object created of one class, and accessed through a pointer to a base class). It makes shared_ptr need to add two words to the shared object, and two words to each shared_ptr copy. The simplified lw_shared_ptr - which does not support polymorphic types - adds just one word in the object (the reference count) and each copy is just one word - just like copying a regular pointer. For this reason, the light-weight seastar::lw_shared_ptr<T> should be preferered when possible (T is not a polymorphic type), otherwise seastar::shared_ptr<T>. The slower std::shared_ptr<T> should never be used in sharded Seastar applications.
Wouldn’t it be convenient if we could save objects on a stack just like we normally do in synchronous code? I.e., something like:
Seastar allows writing such code, by using a seastar::thread object which comes with its own stack. A complete example using a seastar::thread might look like this:
seastar::future<> slow_incr(int i) {
return seastar::async([i] {
seastar::sleep(10ms).get();
// We get here after the 10ms of wait, i is still available.
return i + 1;
});
}We present seastar::thread, seastar::async() and seastar::future::get() in the seastar::thread section.