Closures in Rust

TL;DR

A closure is an anonymous function expression.

Closures can capture values from their environment in three ways, which directly map to the three ways a function can take a parameter: borrowing immutably, borrowing mutably, and taking ownership. The closure will decide which of these to use based on what the body of the function does with the captured values.

Closures are represented as structs that contain either the values (for move closures) or references to the values (for non-move closures) of the variables they capture.

Just as Rust automatically figures out which closures can be called only once, it can figure out which closures can implement Copy and Clone, and which cannot.

FnOnce, FnMut, and Fn are traits.

Brief

When we want some data sorted, we typically have records of some kind, and the built-in sort method typically does not work. We need to specify the sort order, like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

/// Helper function for sorting cities by population.
fn city_population_descending(city: &City) -> i64 {
    -city.population
}

fn sort_cities(cities: &mut Vec<City>) {
    cities.sort_by_key(city_population_descending);  // ok

    // Rust complains that `City` does not implement `std::cmp::Ord`.
    // cities.sort();  // error: how do you want them sorted?
}

struct City {
    name: String,
    population: i64,
    country: String,
}

The helper function, city_population_descending, takes a City record and extracts the key, the field by which we want to sort our data. (It returns a negative number because sort arranges numbers in increasing order, and we want decreasing order: the most populous city first.) The sort_by_key method takes this key-function as a parameter.

This works fine, but it’s more concise to write the helper function as a closure, an anonymous function expression:

1
2
3

fn sort_cities(cities: &mut Vec<City>) {
    cities.sort_by_key(|city| -city.population);
}

The closure here is |city| -city.population. It takes an argument city and returns -city.population. Rust infers the argument type and return type from how the closure is used.

Rust’s closures are anonymous functions you can save in a variable or pass as arguments to other functions. You can create the closure in one place and then call the closure elsewhere to evaluate it in a different context.

Other examples of standard library features that accept closures include:

• Iterator methods such as map and filter, for working with sequential data.
• Threading APIs like thread::spawn, which starts a new system thread. Concurrency is all about moving work to other threads, and closures conveniently represent units of work.
• Some methods that conditionally need to compute a default value, like the or_insert_with method of HashMap entries. This method either gets or creates an entry in a HashMap, and it’s used when the default value is expensive to compute. The default value is passed in as a closure that is called only if a new entry must be created.

Capturing Variables

Unlike functions, closures can capture values from the scope in which they’re defined. A closure can use data that belongs to an enclosing function.

In most languages with closures, garbage collection plays an important role. For example, consider this JavaScript code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

// Start an animation that rearranges the rows in a table of cities.
function startSortingAnimation(cities, stat) {
    // Helper function that we'll use to sort the table.
    // Note that this function refers to stat.
    function keyfn(city) {
        return city.get_statistic(stat);
    }

    if (pendingSort)
        pendingSort.cancel();

    // Now kick off an animation, passing keyfn to it.
    // The sorting algorithm will call keyfn later.
    pendingSort = new SortingAnimation(cities, keyfn);
}

The closure keyfn is stored in the new SortingAnimation object. It’s meant to be called after startSortingAnimation returns. Now, normally when a function returns, all its variables and arguments go out of scope and are discarded. But here, the JavaScript engine must keep stat around somehow, since the closure uses it. Most JavaScript engines do this by allocating stat in the heap and letting the garbage collector reclaim it later.

Closure Type Inference and Annotation

Closures don’t usually require you to annotate the types of the parameters or the return value like fn functions do. Type annotations are required on functions because the types are part of an explicit interface exposed to your users. Closures are stored in variables and used without naming them and exposing them to users of our library.

Closures are typically short and relevant only within a narrow context rather than in any arbitrary scenario. Within these limited contexts, the compiler can infer the types of the parameters and the return type, similar to how it’s able to infer the types of most variables (there are rare cases where the compiler needs closure type annotations too).

As with variables, we can add type annotations if we want to increase explicitness and clarity at the cost of being more verbose than is strictly necessary. Consider this example:

1
2
3
4
5

let expensive_closure = |num: u32| -> u32 { 
    println!("calculating slowly..."); 
    thread::sleep(Duration::from_secs(2)); 
    num 
};

With type annotations added, the syntax of closures looks more similar to the syntax of functions.

1
2
3
4

fn add_one_v1 (x: u32) -> u32 { x + 1 } 
let add_one_v2 = |x: u32| -> u32 { x + 1 }; 
let add_one_v3 = |x| { x + 1 }; 
let add_one_v4 = |x| x + 1 ;

The add_one_v3 and add_one_v4 lines require the closures to be evaluated to be able to compile because the types will be inferred from their usage. This is similar to let v = Vec::new(); needing either type annotations or values of some type to be inserted into the Vec for Rust to be able to infer the type.

For closure definitions, the compiler will infer one concrete type for each of their parameters and for their return value.

1
2
3

let example_closure = |x| x; 
let s = example_closure(String::from("hello")); 
let n = example_closure(5);

The first time we call example_closure with the String value, the compiler infers the type of x and the return type of the closure to be String. Those types are then locked into the closure in example_closure, and we get a type error when we next try to use a different type with the same closure.

Capturing References or Moving Ownership

Closures can capture values from their environment in three ways, which directly map to the three ways a function can take a parameter: borrowing immutably, borrowing mutably, and taking ownership. The closure will decide which of these to use based on what the body of the function does with the captured values.

The following code defines e a closure that captures an immutable reference to the vector named list:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

fn main() { 
    let list = vec![1, 2, 3]; 
    println!("Before defining closure: {list:?}"); 

    let only_borrows = || println!("From closure: {list:?}"); 

    println!("Before calling closure: {list:?}"); 
    only_borrows(); 
    println!("After calling closure: {list:?}"); 
}

The following code defines a closure that captures a mutable reference:

1
2
3
4
5
6
7
8
9

fn main() { 
    let mut list = vec![1, 2, 3]; 
    println!("Before defining closure: {list:?}"); 
    
    let mut borrows_mutably = || list.push(7); 
    
    borrows_mutably(); 
    println!("After calling closure: {list:?}"); 
}

When borrows_mutably is defined, it captures a mutable reference to list.

If you want to force the closure to take ownership of the values it uses in the environment even though the body of the closure doesn’t strictly need ownership, you can use the move keyword before the parameter list. This technique is mostly useful when passing a closure to a new thread to move the data so that it’s owned by the new thread. Consider this example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

use std::thread; 

fn main() { 
    let list = vec![1, 2, 3]; 
    println!("Before defining closure: {list:?}"); 

    thread::spawn(move || println!("From thread: {list:?}")) 
        .join() 
        .unwrap(); 
}

The new thread might finish before the rest of the main thread finishes, or the main thread might finish first. If the main thread maintained ownership of list but ended before the new thread did and dropped list, the immutable reference in the thread would be invalid. Therefore, the compiler requires that list be moved into the closure given to the new thread so the reference will be valid.

Function and Closure Types

We’ve seen functions and closures used as values. Naturally, this means that they have types. For example:

1
2
3

fn city_population_descending(city: &City) -> i64 {
    -city.population
}

This function takes one argument (a &City) and returns an i64. It has the type fn(&City) -> i64.

You can do all the same things with functions that you do with other values. You can store them in variables. Structs may have function-typed fields. Generic types like Vec can store scads of functions, as long as they all share the same fn type.

And function values are tiny: a fn value is the memory address of the function’s machine code, just like a function pointer in C++.

A function can take another function as an argument.

A closure is callable, but it’s not a fn. The closure |city| city.monster_attack_risk > limit has its own type that’s not a fn type.

Every closure you write has its own type, because a closure may contain data: values either borrowed or stolen from enclosing scopes. This could be any number of variables, in any combination of types. So every closure has an ad hoc type created by the compiler, large enough to hold that data. No two closures have exactly the same type. But every closure implements an Fn trait.

Since every closure has its own type, code that works with closures usually needs to be generic.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

fn count_selected_cities<F>(cities: &Vec<City>, test_fn: F) -> usize
    where F: Fn(&City) -> bool
{
    let mut count = 0;
    for city in cities {
        if test_fn(city) {
            count += 1;
        }
    }
    count
}

count_selected_cities takes a test_fn of any type F as long as F implements the special trait Fn(&City) -> bool. This trait is automatically implemented by all functions and most closures that take a single &City as an argument and return a Boolean value:

1
2

fn(&City) -> bool    // fn type (functions only)
Fn(&City) -> bool    // Fn trait (both functions and closures)

Function Pointers

Functions coerce to the type fn (with a lowercase f), not to be confused with the Fn closure trait. The fn type is called a function pointer.

Unlike closures, fn is a type rather than a trait, so we specify fn as the parameter type directly rather than declaring a generic type parameter with one of the Fn traits as a trait bound.

Function pointers implement all three of the closure traits (Fn, FnMut, and FnOnce), meaning you can always pass a function pointer as an argument for a function that expects a closure. It’s best to write functions using a generic type and one of the closure traits so your functions can accept either functions or closures. Consider the following example:

1
2
3
4
5
6
7

let list_of_numbers = vec![1, 2, 3];
// closure
let list_of_strings: Vec<String> = 
    list_of_numbers.iter().map(|i| i.to_string()).collect();
// function
let list_of_strings: Vec<String> = 
    list_of_numbers.iter().map(ToString::to_string).collect();

Closure Performance

Rust’s closures are designed to be fast: faster than function pointers. If you’re familiar with C++ lambdas, you’ll find that Rust closures are just as fast and compact, but safer.

In most languages, closures are allocated in the heap, dynamically dispatched, and garbage collected. So creating, calling, and collecting each of them costs a tiny bit of extra CPU time. Worse, closures tend to rule out inlining, a key technique compilers use to eliminate function call overhead and enable a raft of other optimizations. All told, closures are slow enough in these languages that it can be worth manually removing them from tight inner loops.

Inlining refers to a compiler optimization technique where the compiler replaces a function call with the actual code of the function. This can reduce the overhead associated with function calls, such as the time taken to pass arguments and return values, as well as the context switching that occurs during the call. It’s an example of trading space for time.

Rust closures have none of these performance drawbacks. They’re not garbage collected. Like everything else in Rust, they aren’t allocated on the heap unless you put them in a Box, Vec, or other container. And since each closure has a distinct type, whenever the Rust compiler knows the type of the closure you’re calling, it can inline the code for that particular closure. This makes it OK to use closures in tight loops, and Rust programs often do so, enthusiastically.

The following figure shows how Rust closures are laid out in memory. At the top of the figure, we show a couple of local variables that our closures will refer to: a string food and a simple enum weather, whose numeric value happens to be 27.

Closure (a) uses both variables. In memory, this closure looks like a small struct containing references to the variables it uses. Note that it doesn’t contain a pointer to its code! That’s not necessary: as long as Rust knows the closure’s type, it knows which code to run when you call it.

Closure (b) is exactly the same, except it’s a move closure, so it contains values instead of references.

Closure (c) doesn’t use any variables from its environment. The struct is empty, so this closure does not take up any memory at all.

These closures don’t take up much space. But even those few bytes are not always needed in practice. Often, the compiler can inline all calls to a closure, and then even the small structs shown in this figure are optimized away.

Closures and Safety

The way a closure captures and handles values from the environment affects which traits the closure implements, and traits are how functions and structs can specify what kinds of closures they can use. Closures will automatically implement one, two, or all three of these Fn traits, in an additive fashion, depending on how the closure’s body handles the values:

1. FnOnce applies to closures that can be called once. All closures implement at least this trait, because all closures can be called.
2. FnMut applies to closures that don’t move captured values out of their body, but that might mutate the captured values. These closures can be called more than once.
3. Fn applies to closures that don’t move captured values out of their body and that don’t mutate captured values, as well as closures that capture nothing from their environment.

Closures That Kill

The most straightforward way to drop values is to call drop():

1
2
3
4
5

let my_str = "hello".to_string();
let f = || drop(my_str);

f();  // ok
f();  // error: value used here after move

When f is called, my_str is dropped. The first time we call f, it drops my_str, which means the memory where the string is stored is freed, returned to the system. The second time we call f, the same thing happens. It’s a double free, a classic mistake in C++ programming that triggers undefined behavior. Dropping a String twice would be an equally bad idea in Rust. Rust knows this closure can’t be called twice.

The idea of values being used up (that is, moved) is one of the core concepts in Rust. It works the same with closures as with everything else.

FnOnce

The following generic function tries to trick Rust into dropping a String twice:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

fn call_twice<F>(closure: F) where F: Fn() {
    closure();
    closure();
}

let my_str = "hello".to_string();
let f = || drop(my_str);
call_twice(f);

// error[E0525]: expected a closure that implements the `Fn` trait, but this closure only implements `FnOnce`
//   --> src/main.rs:63:13
//    |
// 63 |     let f = || drop(my_str);
//    |             ^^      ------ closure is `FnOnce` because it moves the variable `my_str` out of its environment
//    |             |
//    |             this closure implements `FnOnce`, not `Fn`
// 64 |     call_twice(f);
//    |     ---------- - the requirement to implement `Fn` derives from here
//    |     |
//    |     required by a bound introduced by this call

This generic function may be passed any closure that implements the trait Fn(): that is, closures that take no arguments and return (). (As with functions, the return type can be omitted if it’s (); Fn() is shorthand for Fn() -> ().)

Closures that drop values, like f, are not allowed to have Fn. They are, quite literally, no Fn at all. They implement a less powerful trait, FnOnce, the trait of closures that can be called once.

The first time you call a FnOnce closure, the closure itself is used up. It’s as though the two traits, Fn and FnOnce, were defined like this:

1
2
3
4
5
6
7
8

// Pseudocode for `Fn` and `FnOnce` traits with no arguments.
trait Fn() -> R {
    fn call(&self) -> R;
}

trait FnOnce() -> R {
    fn call_once(self) -> R;
}

Just as an arithmetic expression like a + b is shorthand for a method call, Add::add(a, b), Rust treats closure() as shorthand for one of the two trait methods shown in the preceding example. For an Fn closure, closure() expands to closure.call(). This method takes self by reference, so the closure is not moved. But if the closure is only safe to call once, then closure() expands to closure.call_once(). That method takes self by value, so the closure is used up.

FnMut

Rust has one more category of closure, FnMut, the category of closures that write. FnMut closures are called by mut reference, as if they were defined like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

// Pseudocode for `Fn`, `FnMut`, and `FnOnce` traits.
trait Fn() -> R {
    fn call(&self) -> R;
}

trait FnMut() -> R {
    fn call_mut(&mut self) -> R;
}

trait FnOnce() -> R {
    fn call_once(self) -> R;
}

Any closure that requires mut access to a value, but doesn’t drop any values, is an FnMut closure. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

let mut i = 0;
let incr = || {
    i += 1;  // incr borrows a mut reference to i
    println!("Ding! i is now: {}", i);
};
call_twice(incr);

// error[E0525]: expected a closure that implements the `Fn` trait, but this closure only implements `FnMut`
//   --> src/main.rs:63:16
//    |
// 63 |     let incr = || {
//    |                ^^ this closure implements `FnMut`, not `Fn`
// 64 |         i += 1; // incr borrows a mut reference to i
//    |         - closure is `FnMut` because it mutates the variable `i` here
// ...
// 67 |     call_twice(incr);
//    |     ---------- ---- the requirement to implement `Fn` derives from here
//    |     |
//    |     required by a bound introduced by this call

Summary:

• FnOnce is the family of closures that can be called once, if the caller owns the closure.
• FnMut is the family of closures that can be called multiple times if the closure itself is declared mut.
• Fn is the family of closures and functions that you can call multiple times without restriction. This highest category also includes all fn functions.
  • Fn trait 的限制最少，反过来说它对具体实现的约束最严格。

Every Fn meets the requirements for FnMut, and every FnMut meets the requirements for FnOnce.

Fn() is a subtrait of FnMut(), which is a subtrait of FnOnce(). This makes Fn the most exclusive and most powerful category. FnMut and FnOnce are broader categories that include closures with usage restrictions.

Now it’s clear that to accept the widest possible swath of closures, our call_twice function really ought to accept all FnMut closures, like this:

1
2
3
4
5
6
7
8

fn call_twice<F>(mut closure: F) where F: FnMut() {
    closure();
    closure();
}

let mut i = 0;
call_twice(|| i += 1);  // ok!
assert_eq!(i, 2);

The bound on the first line was F: Fn(), and now it’s F: FnMut(). With this change, we still accept all Fn closures, and we additionally can use call_twice on closures that mutate data.

Copy and Clone for Closures

Just as Rust automatically figures out which closures can be called only once, it can figure out which closures can implement Copy and Clone, and which cannot.

Closures are represented as structs that contain either the values (for move closures) or references to the values (for non-move closures) of the variables they capture. The rules for Copy and Clone on closures are just like the Copy and Clone rules for regular structs. A non-move closure that doesn’t mutate variables holds only shared references, which are both Clone and Copy, so that closure is both Clone and Copy as well:

1
2
3
4

let y = 10;
let add_y = |x| x + y;
let copy_of_add_y = add_y;                // This closure is `Copy`, so...
assert_eq!(add_y(copy_of_add_y(22)), 42); // ... we can call both.

On the other hand, a non-move closure that does mutate values has mutable references within its internal representation. Mutable references (has exclusive access to the value) are neither Clone nor Copy, so neither is a closure that uses them:

1
2
3
4
5

let mut x = 0;
let mut add_to_x = |n| { x += n; x };

let copy_of_add_to_x = add_to_x;         // this moves, rather than copies
assert_eq!(add_to_x(copy_of_add_to_x(1)), 2); // error: use of moved value

For a move closure, the rules are even simpler. If everything a move closure captures is Copy, it’s Copy. If everything it captures is Clone, it’s Clone. For instance:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

let mut greeting = String::from("Hello, ");
let greet = move |name| {
    greeting.push_str(name);
    println!("{}", greeting);
};
greet.clone()("Alfred");
greet.clone()("Bruce");

// Hello, Alfred
// Hello, Bruce

This .clone()(...) syntax means that we clone the closure and then call the clone.

When greeting is used in greet, it’s moved into the struct that represents greet internally, because it’s a move closure. So, when we clone greet, everything inside it is cloned, too. There are two copies of greeting, which are each modified separately when the clones of greet are called. This isn’t so useful on its own, but when you need to pass the same closure into more than one function, it can be very helpful.

Callbacks

A lot of libraries use callbacks as part of their API: functions provided by the user, for the library to call later.

1
2
3

struct BasicRouter<C> where C: Fn(&Request) -> Response {
    routes: HashMap<String, C>
}

router.add_route("/", |_| get_form_response()) 后，routes: HashMap<String, C> 中的 C 的类型就被固定为第一次调用 add_route 时的闭包类型。

Closures that don’t capture anything from their environment are identical to function pointers, since they don’t need to hold any extra information about captured variables. If you specify the appropriate fn type, either in a binding or in a function signature, the compiler is happy to let you use them (closures) that way:

1
2

let closure_ptr: fn(u32) -> u32 = |x| x + 1;
let two = closure_ptr(1); // 2

Unlike capturing closures, these function pointers take up only a single usize.

By using fn pointers in functions that take callbacks, you can restrict a caller to use only these noncapturing closures, gaining some perfomance and flexibility within the code using callbacks at the cost of flexibility for the users of your API.

Using Closures Effectively

In languages with GC, you can use local variables in a closure without having to think about lifetimes or ownership.

For example, take the Model-View-Controller design pattern, illustrated below. For every element of a user interface, an MVC framework creates three objects: a model representing that UI element’s state, a view that’s responsible for its appearance, and a controller that handles user interaction. The general idea is that three objects divvy up the UI responsibilities somehow.

Here’s the problem. Typically, each object has a reference to one or both of the others, directly or through a callback. Whenever anything happens to one of the objects, it notifies the others, so everything updates promptly. The question of which object “owns” the others never comes up.

You can’t implement this pattern in Rust without making some changes. Ownership must be made explicit, and reference cycles must be eliminated. The model and the controller can’t have direct references to each other.

Rust’s radical wager is that good alternative designs exist. Sometimes you can fix a problem with closure ownership and lifetimes by having each closure receive the references it needs as arguments. Sometimes you can assign each thing in the system a number and pass around the numbers instead of references. Or you can implement one of the many variations on MVC where the objects don’t all have references to each other. Or model your toolkit after a non-MVC system with unidirectional data flow, like Facebook’s Flux architecture, shown below.

In short, if you try to use Rust closures to make a “sea of objects,” you’re going to have a hard time. But there are alternatives.

References

• Programming Rust, 2nd Edition (Covers the Rust 2021 Edition)
• https://doc.rust-lang.org/book/ch13-01-closures.html
• https://doc.rust-lang.org/book/ch19-05-advanced-functions-and-closures.html#function-pointers