Converting from Generic Unsized Parameter to Trait Object in Rust

Introduction

In Rust, the ?Sized trait and trait objects (dyn Trait) are key tools for handling types with unknown sizes at compile time. However, converting a generic function that accepts a ?Sized parameter into a trait object can be tricky, especially when dealing with coercions. In this article, we'll explore how to achieve this conversion and highlight some of the challenges and workarounds involved.

Problem Context

Consider a generic function that takes a parameter of any type implementing the Write trait. This function should accept both sized types (like Vec<u8>) and unsized types (like a dyn Write). The challenge is to create a trait object (&mut dyn Write) from a generic parameter that may or may not be Sized.

Here's a simplified example:

use std::io::Write;

fn only_unsized(_: &mut dyn Write) {}

fn allow_unsized<W: Write + ?Sized>(w: &mut W) {
    // Attempting to pass `w` directly to `only_unsized`
    // only_unsized(w); // This doesn't compile
    // only_unsized(w.by_ref()); // This also doesn't compile

    // The workaround:
    only_unsized(&mut *w); // This compiles
}

In the above code, Rust complains about the inability to convert w directly into a trait object. The workaround (only_unsized(&mut *w)) might look confusing at first, but it works by creating an indirection level that facilitates coercion.

Understanding `?Sized` and Coercion

The ?Sized bound allows a generic function to accept types that may or may not be sized at compile time. However, a problem arises when we want to pass these types as trait objects for dynamic dispatch. The unsizing coercion doesn't work directly because of the incompatibility between the pointer types involved.

In the function allow_unsized, the parameter w is of type &mut W, where W: Write + ?Sized. The problem is that we want to pass w to another function that expects a &mut dyn Write, but unsizing directly from &mut W to &mut dyn Write is not always possible due to Rust's type system.

The Workaround Explained

To overcome this limitation, a level of indirection is added:

fn allow_unsized<W: Write + ?Sized>(mut w: &mut W) {
    only_unsized(&mut w);
}

By introducing mut w, we create a mutable reference to a mutable reference (&mut &mut W). This enables the coercion from &mut W to &mut dyn Write to work smoothly because the extra level of reference allows Rust to handle the coercion more effectively. In other words, by creating a reference to w, we let Rust handle the unsizing coercion without a direct type mismatch.

Example: Handling Unsized Structs

Let's explore a more comprehensive example involving an unsized struct:

struct MaybeUnsized<T: ?Sized> {
    value: T,
}

impl Write for MaybeUnsized<[u8]> {
    fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
        // Example implementation
        Ok(buf.len())
    }
    fn flush(&mut self) -> std::io::Result<()> {
        Ok(())
    }
}

fn example<W: Write + ?Sized>(w: &mut W) {
    let mut buffer = MaybeUnsized { value: [0u8; 5] };
    only_unsized(&mut buffer);
}

In this example, MaybeUnsized<[u8]> is an unsized type, and we can use it in conjunction with only_unsized through the same kind of workaround shown earlier.

Why Does This Work?

The workaround works because of Rust's type coercion rules and how pointers to unsized types are handled. A reference to an unsized type (like a slice or a trait object) is a fat pointer, which carries both the reference to the data and some additional metadata (like length for slices or vtable pointer for trait objects).

When you try to pass w directly to only_unsized, Rust tries to perform an unsizing coercion that drops critical metadata, making it impossible to create a &mut dyn Write from a type that may already be unsized in some cases. Introducing mut w allows the compiler to do this conversion because you’re working with a mutable reference to a reference, giving Rust the ability to perform the necessary type conversions and maintain metadata consistency.

A More Efficient Approach with Traits

Another approach to handle this would be by using a specialized trait. Instead of trying to coerce directly into a trait object, we can define an intermediate trait that enables this kind of conversion explicitly:

trait UnsizedWrite: Write {
    fn as_dyn_write(&mut self) -> &mut dyn Write;
}

impl<T: Write + ?Sized> UnsizedWrite for T {
    fn as_dyn_write(&mut self) -> &mut dyn Write {
        self
    }
}

fn allow_unsized_v2<W: UnsizedWrite + ?Sized>(w: &mut W) {
    only_unsized(w.as_dyn_write());
}

Here, the UnsizedWrite trait is implemented for any type that implements Write. It provides a method as_dyn_write that returns a reference to the trait object, effectively handling the conversion explicitly without relying on Rust's automatic coercion.

Last Words

Handling generic unsized parameters in Rust, especially when converting to trait objects, requires understanding how coercion and fat pointers work. While the solution might look like adding unnecessary levels of references, it's a necessary step to help Rust maintain the correct type information and avoid losing critical metadata.

By using helper traits or introducing extra levels of indirection, we can make these conversions possible while keeping the type safety guarantees that Rust is known for.