Heard of Rust? Then you may have heard of "lifetimes". They are one of the languages hallmark features, but sometimes also one of the most cursed. This isn't necessary.

Lifetimes are a interesting subject: a lot of people seem to gain a day-to-day familiarity with them, without fully understanding what they are. Maybe, they are truly Rust's Monads. Let's talk about what they are, where you encounter them and then how to get competent with them.

This blog post requires only minor Rust knowledge. The intended audience are curious programmers in general or new Rustaceans.

If you want to know more about Rust or your company wants to adopt it, remember that we offer tailored courses and development and consulting.

A Note

For the purpose of this blog post, I'll explicitely use drop to remove all values. When this is not used, Rust will insert a drop statement for all variables at the end of the scope. As we're talking about dropping and a lot and want to control it for example purposes, I found it useful to make it explicit everywhere.

A Lifetime

The lifetime of data evokes some intuitive understanding. They are often related to the stack and the heap in Rust and map neatly to those, but do arise from a far more central concept of Rust: Ownership. Let's take some plain old data struct.

struct Point {
    x: i32,
    y: i32,
}

fn main() {
    let point = Point { x: 0, y: 0 };

    drop(point);
}

Ownership dictates that every piece of data is exclusively owned by another part of the program. In this is case, it is the let-binding point. Once ownership ends, the value is dropped, that means, it is removed from memory. This happens immediately. In the case of let-bindings, this is by the end of the scope. The code would be just the same if the drop call was removed. This gives us two points: introduction of the data into the program (initialisation) and removal (dropping). The interesting thing in Rust is that those two points are always clearly present. The range between those points is the region the binding and it's associated value is alive, its lifetime.

Compare this to a garbage collected language like Java: Here, the number of held references to any piece of data decides when to remove the value from memory. This means that after initialisation, the lifetime of data is determined by how often a reference of it is taken. The Garbage Collector regularly checks if a value still has references pointing to it and will remove it once this is not the case anymore.

A Borrow

In Rust, & takes a so-called borrow. This evokes an intuitive understanding, too: what is owned, can be borrowed.

Say, we want to print any pair of x and y coordinates. We could do that by writing a function like this:

fn print_coordinates(x: &i32, y: &i32) {
    println!("x: {}, y: {}", x, y);
}

print_coordinates borrows x and y from the scope it was called from. We call it just like this:

fn main() {
    let point = Point { x: 0, y: 0 };
    
    print_coordinates(&point.x, &point.y);

    drop(point);
}

Simple enough, right?

Let's reorder that a little:

fn main() {
    let point = Point { x: 0, y: 0 };
    
    drop(point);

    print_coordinates(&point.x, &point.y);
}

If Rust were to allow that, we'd have a problem. But it doesn't, the compiler calls us out:

 Compiling playground v0.0.1 (file:///playground)
error[E0382]: use of moved value: `point.x`
  --> src/main.rs:15:24
   |
13 |     drop(point);
   |          ----- value moved here

drop takes ownership of the value (to remove it from memory). It moves it out of of main to do that. When speaking about lifetimes, the problem can also be formulated like this: we can't print parts of point, because it isn't alive anymore.

Let's try to cheat some more on the compiler:

fn main() {
    let point = Point { x: 0, y: 0 };
    
    let x = &point.x;
    let y = &point.y;

    drop(point);

    print_coordinates(x, y);
}

And run it…

   Compiling playground v0.0.1 (file:///playground)
error[E0505]: cannot move out of `point` because it is borrowed
  --> src/main.rs:16:10
   |
13 |     let x = &point.x;
   |              ------- borrow of `point.x` occurs here

Darn, the compiler is good! But the error message changed a little. Why does that happen? The compiler knows that when you borrow something, the original must be alive. So a borrow can never outlive the orginal. And so it just connects the dots: x and y were taken from point, so the moment point is not valid anymore, usage of x and y is not valid anymore. Or, put the other way: you cannot move the point if you later intend to use the borrows - moving might invalidate the borrows. They are lifetime bound.

Let's try to futher hide the situation from the compiler:

struct Point {
    x: i32,
    y: i32,
}

impl Point {
    fn x(&self) -> &i32 {
        &self.x
    }

    fn y(&self) -> &i32 {
        &self.y
    }
}

fn main() {
    let point = Point { x: 0, y: 0 };
    
    let x = point.x();
    let y = point.y();

    drop(point);

    print_coordinates(x, y);
}

No borrow in sight in main, at least x and y return just plain borrows.

   Compiling playground v0.0.1 (file:///playground)
error[E0505]: cannot move out of `point` because it is borrowed
  --> src/main.rs:26:10
   |
23 |     let x = point.x();
   |             ----- borrow of `point` occurs here

GRRRRR! Same issue! Clever beast! The compiler traces that we borrowed the point through the call to the accessor x().

So, let's spell out what the compiler followed us doing:

• We constructed the point
• We called x(), which borrows the point (&self)
• Inside x(), we took a borrow to a subfield of the point.
• We bind that borrow to a binding through let x
• We do the same with y
• We drop the point
• The compiler catches us trying to use x and y.

It assembled a chain of these actions through a function call and cought us doing naughty with one of the values.

At this point we might believe that Rust just analyses the whole program and check the validity of all borrows when in assembles the program. But whole program analysis is costly and hard to communicate well ("hey, this borrow doesn't work because something at the edge of nowhere").

And it's not what happens.

A Notation

We could write our implementation in a different fashion:

impl Point {
    fn x<'point>(&'point self) -> &'point i32 {
        &self.x
    }

    fn y<'point>(&'point self) -> &'point i32 {
        &self.y
    }
}

The reason why you don't need to do this is lifetime elision. The case is just so common that if you don't write anything at all, it is assumed.

So, what does this say? First, it introduces a lifetime parameter 'point. Because we write &'point self, we bind the input reference to lifetime. &'point i32 in the return type also binds the return reference to that lifetime.

This allows the compiler to reason about &self.x. What it checks is: "can I hand out a reference that does not live longer than 'point from &self, which does itself has the lifetime 'point?" The answer is yes. You are allowed to pass.

But the lifetime does more than that: it is part of the functions signature! That means, it communicates to the caller: "If you call this, you need to make sure that the borrow I give you back does not live longer than the thing I called x() on."

First conclusions

This allows us to finally figure out what the compiler checks when we compile our code. First, it checks if fn x(&self) -> &i32 is sound. It is, for the reasons I just described.

Now, when compiling main, it checks if we're holding to the guarantees. By calling x(), we agree to not use the returned borrow after point does not live anymore. We don't. We try to drop the point before using the borrows. The compiler catches us.

Having understood the interface of x(), you will also understand the interface of most collection APIs.

Let's come back to what we wrote. This is our program in words:

• introduce a value
• take a pointer to a field of the value
• remove the value
• dereference the value

We asked for that order. It doesn't work and the compiler does not try to make it work. And there comes the big catch with lifetimes.

You cannot program with lifetimes

I repeat: You cannot program with lifetimes. They are a tiny logic language within the language. What they do is prove validity of all references, one function at a time.

The cleverness of lifetimes is that they compose and signal. Function signatures with lifetimes communicate more than just "pointer in, pointer out", they communicate how they relate to each other.

You still - and this is where most people bang their head against the wall - have to fulfill these rules. Rust is an imperative language: what you write is how it executes. No change of lifetime annotations will make illegal situations suddenly work. This is also the beauty of them: if you declare them wrong, you can't break things.

Still: lifetimes are declarative. Indeed, if your code is validated using the borrow checker, it will just be compiled as if all your borrows were plain pointers.

Problems and Solutions

Questions like "how do I extend the lifetime of our point?" are easily answered if you approach them right. Do the appropriate thing to make the value live longer (for example, don't drop it early).

A common problem I see in trainings or the Hack & Learn is that presented with a lifetime problem, people start messing around with lifetime syntax. This is often the wrong approach. It is always the wrong approach if you don't fully understand what the compiler calls you out on. It has probably found an issue you haven't thought about, making a borrow invalid.

The solution is usually to go back to the drawing board and think about your value flow hard. I recommend a piece of paper or a whiteboard for that.

Finally, there's often the option of just copying or cloning data in memory to get owned access to again. Knowing when trying to take ownership back makes sense is an important skill.

Live longer and prosper

This is already a pretty long post, and I've only introduced one lifetime syntax. This is the a dominant one. There's another one you should know as a beginner:

struct Wrapper<'wrapped, T: 'wrapped> {
    wrapped: &'wrapped T
}

This wraps a borrow to any type. But as we have a borrow, the wrapper cannot outlive what wrapped points to. Here, 'wrapped does exactly the same: it binds together both under the same name. It's pretty rare that structs have more than one lifetime binding. Indeed, I've myself never used one that has.

Note that T: 'wrapped. Lifetimes are type bounds just like traits. This bounds T to be any type, as long as it lives longer than or as long as 'wrapped. This is necessary, because pointing with a borrow that lives 'wrapped to something which lives shorter than 'wrapped is obviously not safe.

You will see this notation often when you have something that operates over something else without destroying it: the classic case are iterators. They are lifetime bound to the thing they iterate over.

Use Lifetimes to Clarify Situations

So where do we need explict lifetimes? Everywhere where situations are unclear.

Let's have a look at the function signature of std::str.split, which splits a string slice and hands out the splitted parts:

fn split<'a, P>(&'a self, pat: P) -> Split<'a, P> where
    P: Pattern<'a>,

The standard library has the habit of using non-descriptive names like 'a for their lifetimes. Take a moment to apply your knowledge.

Let's go in correct order: first understand the problem, then the lifetime annotations. split takes 2 parameters in: self, the slice to iterate over, and pat, the pattern to use for splitting. Split is the lazy iterator that, when next is called on it, hands out subslices pointing to the original. This is crucial: Split does not copy strings, just point to the appropriate substrings of the original. For that to work, the original must be alive. But the same goes for the pattern pat! We need it every time we call next(), so it needs to be alive then! So the situation is: we have a couple of things that point to each other and they must all be kept around while they work as a group.

In this situation, no elision happens, but our description is easy enough: we just introduce a lifetime 'a, which binds all those values together. For that to work, we just use it at all appropriate slots. Now, we have bound Split to the input and the pattern. That ensures they cannot be dropped while the iterator (or any of the values it hands out subsequently!) still exists.

This checking has no runtime cost at all, but completely works at compile time!

Conclusion

I hope I made lifetimes a little more clearer for those struggling with them. I haven't introduced all lifetime notations and situations, but they are basically just forms of the same problem. This might happen in a future post.

Remember the two golden rules:

• Don't fiddle with lifetime syntax until you understood what the compiler calls you out on
• Taking ownership (e.g. through cloning or using Box) isn't cheating

Credits

Thanks Pascal for proofreading.