ENOSUCHBLOG

Programming, philosophy, pedaling.

totally_safe_transmute, line-by-line

Mar 16, 2021     Tags: curiosity, programming, rust    

This post is at least a year old.

A few weeks ago, Twitter deigned to share this with me:

The file linked (written by Benjamin Herr) in the tweet purports to implement a version of Rust’s std::mem::transmute without any use of unsafe. If you run it, you’ll find that it does indeed work!

1
2
3
4
5
6

#[test]
fn main() {
    let v: Vec<u8> = b"foo".to_vec();
    let v: String = totally_safe_transmute(v);
    assert_eq!(&v, "foo");
}

Yields:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

$ git clone https://github.com/ben0x539/totally-safe-transmute
$ cargo build
$ cargo test
   Compiling totally-safe-transmute v0.0.3 (/tmp/totally-safe-transmute)
    Finished test [unoptimized + debuginfo] target(s) in 0.49s
     Running target/debug/deps/totally_safe_transmute-be2ea6d9a3f8d258

running 1 test
test main ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests totally-safe-transmute

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

This blog post will go through that implementation, line-by-line, and explain how it works. Nothing about it is especially complicated; I just got a huge kick out of it and figured I’d provide a detailed explanation. Rust newcomers are the intended audience.

Quick background: transmutation

Most unsafe languages have mechanisms for transmuting (or reinterpreting) the data at some memory address as an entirely new type. C includes reinterpretation under its casting syntax; C++ provides the more explicit reinterpret_cast<T> (with plenty of warnings about when reinterpret_cast is well-defined).

Reinterpretation has plenty of use cases:

Each of the above is useful, but incredibly unsafe: transmutation is not an operation at runtime that turns one type into another, but rather a directive at compile time to treat some position in memory as if its type is different. The result: most possible transmutations between types result in undefined behavior.

Transmutation in Rust

Rust needs to interface with C, so4 Rust supports transmutation. It does so via std::mem::transmute. But transmutation is a fundamentally unsafe operation, so Rust forbids the use of mem::transmute except for in explicitly unsafe contexts:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

use std::mem;

#[repr(C)]
pub struct Foo {
    pub a: u8,
    pub b: u8,
    pub c: u8,
    pub d: u8
}

#[repr(C)]
pub struct Bar {
    pub a: u32
}

fn main() {
    let foo = Foo { a: 0xaa, b: 0xbb, c: 0xcc, d: 0xdd };
    let bar: Bar = unsafe { mem::transmute(foo) };

    // output (on x86-64): bar.a = 0xddccbbaa
    println!("bar.a = {:x}", bar.a);
}

(View it on Godbolt.)

transmute can, of course, be wrapped into safe contexts. But the underlying operation will always be fundamentally unsafe, and should not be possible in otherwise safe Rust code.

So, how does totally_safe_transmute do it?

Breakdown

First, here’s the entirety of totally_safe_transmute:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

#![forbid(unsafe_code)]

use std::{io::{self, Write, Seek}, fs};

pub fn totally_safe_transmute<T, U>(v: T) -> U {
    #[repr(C)]
    enum E<T, U> {
        T(T),
        #[allow(dead_code)] U(U),
    }
    let v = E::T(v);

    let mut f = fs::OpenOptions::new()
        .write(true)
        .open("/proc/self/mem").expect("welp");

    f.seek(io::SeekFrom::Start(&v as *const _ as u64)).expect("oof");
    f.write(&[1]).expect("darn");

    if let E::U(v) = v {
        return v;
    }

    panic!("rip");
}

Let’s go through it, (mostly) line-by-line.

#![forbid(unsafe_code)]

forbid is an attribute that controls the rustc linter (along with allow, warn, and deny). In this case, we’re telling rustc to forbid anything that trips the unsafe_code lint, which does exactly what it says on the tin: catches use of unsafe.

In this case, forbidding use of unsafe doesn’t do anything: a quick read of the code shows that unsafe never shows up. But it’s a top-level proof to the reader that, if rustc accepts the code (and it does), then there is no use of unsafe.

totally_safe_transmute

Here’s our signature:

1

pub fn totally_safe_transmute<T, U>(v: T) -> U { ... }

In sum: totally_safe_transmute takes two type parameters: T and U.

It then takes one concrete parameter, v, which is of type T. Finally, it returns a U.

We know that the job of a transmutation function is to reinterpret a type of some value as some other type, so we can rewrite this signature as:

1

pub fn totally_safe_transmute<SrcTy, DstTy>(v: SrcTy) -> DstTy { ... }

enum E

Our next bit is a terse enum with some funky attributes. Rewritten with our friendly type parameters:

1
2
3
4
5
6

#[repr(C)]
enum E<SrcTy, DstTy> {
    T(SrcTy),
    #[allow(dead_code)] U(DstTy),
}
let v = E::T(v);

First, we’re marking E as repr(C). This is an ABI-modifying attribute: it tells rustc to lay E out using the platform’s C ABI rather than the (intentionally) unstable Rust ABI.

What does this actually mean? For enums with fields (like this one), Rust uses a “tagged union” representation. In effect, E becomes something like this (in C syntax):

1
2
3
4
5
6
7

struct E {
  int discriminant;
  union {
    SrcTy T;
    DstTy U;
  } data;
};

We’ll see why that’s important in a bit.

Next: E has two variants: the first holds a value of type SrcTy, and the other holds a value of DstTy.

But wait! Another rustc linter annotation: this time, we’re telling rustc that it’s okay for the U variant to fail the dead_code lint. Normally, rustc would warn us upon statically inferring that U is never used; with dead_code enabled, it silences that warning. Like the ABI layout, we’ll see why that’s important shortly.

Finally, we shadow our v parameter with a new binding. v was already of type T, so creating an E::T from it is no problem at all.

I/O

This is where the (main) magic happens:

1
2
3
4
5
6

let mut f = fs::OpenOptions::new()
    .write(true)
    .open("/proc/self/mem").expect("welp");

f.seek(io::SeekFrom::Start(&v as *const _ as u64)).expect("oof");
f.write(&[1]).expect("darn");

First, we’re opening a file. Specifically, we’re opening /proc/self/mem in write mode.

/proc/self/mem is a very special5 file: it presents a view of the current process’s memory, sparsely mapped by virtual address ranges.

As a quick hack, we can prove this to ourselves in Python by checking out the in-memory representation of a str object6:

1
2
3
4
5
6
7
8
9

>>> x = "this string is long enough to prevent any string interning"
>>> # in cpython, an object's id is (usually) its pointer
>>> x_addr = id(x)
>>> hex(x_addr)
'0x7ff1bc7cfce0'
>>> mem = open("/proc/self/mem", mode="rb")
>>> mem.seek(x_addr)
>>> mem.read(len(x) * 4)
b'[SNIP] "thi\x00\x00\x00\x00\x00\x00\x00\x00this string is long enough to prevent any string interning\x00e\'[SNIP]'

(I trimmed the output a bit. You get the point.)

We can even poke memory by writing into /proc/self/mem:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

>>> # using ctypes to avoid the layout muckery we saw above
>>> import ctypes
>>> cstr = ctypes.c_char_p(b"look ma, no hands")
>>> cstr_addr = ctypes.cast(cstr, ctypes.c_void_p).value
>>> hex(cstr_addr)
'0x7f47f3e9c790'
>>> mem = open("/proc/self/mem", mode="r+b")
>>> mem.seek(cstr_addr)
>>> mem.read(len(cstr.value))
b'look ma, no hands'
>>> mem.seek(cstr_addr + 5)
>>> mem.write('p')
>>> mem.seek(cstr_addr)
>>> mem.read(len(cstr.value))
b'look pa, no hands'

The next two pieces of totally_safe_transmute should now make sense: we seek to the address of our v variable (which is now a variant of E) within our own running process, and we write a single u8 to it ([1]).

But why 1? Recall our C ABI representation of E above! The first piece of E is our union discriminator. When data is SrcTy, discriminant is 0. When we forcefully overwrite it to 1, data is now interpreted as DstTy!

The last bit

Okay, so we’ve poked memory and turned our E::T into an E::U. Let’s see how we get it out:

1
2
3
4
5

if let E::U(v) = v {
    return v;
}

panic!("rip");

At first glance, there’s nothing special about this: we’re simply discarding the enum wrapper that we added earlier so that we can return our newly-minted value of DstTy.

But this is actually deceptively clever, and involves fooling the compiler:

This is why we needed allow(dead_code) earlier: no E::U is ever constructed in a manner that could possibly reach the return statement, so there’s simply no need for it as a variant. And indeed, we can confirm this by removing the allow attribute:

1
2
3
4
5
6
7
8
9
10
11
12
13

$ cargo build
   Compiling totally-safe-transmute v0.0.3 (/tmp/totally-safe-transmute)
warning: variant is never constructed: `U`
 --> src/lib.rs:9:9
  |
9 |         U(U),
  |         ^^^^
  |
  = note: `#[warn(dead_code)]` on by default

warning: 1 warning emitted

    Finished dev [unoptimized + debuginfo] target(s) in 0.15s

But alas: it’s not really dead code: the compiler is “wrong,” and we pop an E::U into existence at runtime by modifying the program’s own memory. We then hit our impossible condition, and return our transmuted value.

Wrapup

totally_safe_transmute is a delightful hack that demonstrates a key limitation when reasoning about a program’s behavior: every behavior model is contingent on an environmental model and how the program (or the program’s runtime, or the compiler, or whatever else) chooses (or doesn’t choose) to handle seemingly impossible conditions in said environment.

The ability to do this doesn’t reflect fundamental unsafety in Rust, any more than it does any safe language: from Rust’s perspective, what totally_unsafe_transmute does is impossible and therefore undefined; there’s no point in in handling something that cannot happen.

Some other interesting bits:

  1. Usually for reasons of mis-design. But it happens. ↩

  2. And a few others, like unsigned char* and std::byte. ↩

  3. One of the great sins of application and network programming, and a common source of vulnerabilities. ↩

  4. Among other reasons. As mentioned, transmutation is useful when all of need is a “bag of bytes” view of some object, or when you can guarantee consistent type layouts. It’s also useful for advanced lifetime hackery. ↩

  5. And platform-specific: it’s part of Linux’s procfs. As a result, totally_safe_transmute won’t work (as-is) on other OSes. ↩

  6. Which, as you’ll notice, is not trivial (it’s not just a length-data pair). unicodeobject.h in the CPython source has the full structure details, which are completely irrelevant to this post. ↩

Discussions: Reddit