Mar 16, 2021 Tags: curiosity, programming, rust
A few weeks ago, Twitter deigned to share this with me:
The file linked
(written by Benjamin Herr) in the tweet purports to implement a
version of Rust’s std::mem::transmute
without any use of unsafe. If you run it, you’ll find that it does indeed work!
1
2
3
4
5
6
#[test]
fn main() {
let v: Vec<u8> = b"foo".to_vec();
let v: String = totally_safe_transmute(v);
assert_eq!(&v, "foo");
}
Yields:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ git clone https://github.com/ben0x539/totally-safe-transmute
$ cargo build
$ cargo test
Compiling totally-safe-transmute v0.0.3 (/tmp/totally-safe-transmute)
Finished test [unoptimized + debuginfo] target(s) in 0.49s
Running target/debug/deps/totally_safe_transmute-be2ea6d9a3f8d258
running 1 test
test main ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
Doc-tests totally-safe-transmute
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
This blog post will go through that implementation, line-by-line, and explain how it works. Nothing about it is especially complicated; I just got a huge kick out of it and figured I’d provide a detailed explanation. Rust newcomers are the intended audience.
Most unsafe languages have mechanisms for transmuting (or reinterpreting)
the data at some memory address as an entirely new type. C includes
reinterpretation under its casting syntax; C++
provides the more explicit reinterpret_cast<T>
(with plenty of warnings
about when reinterpret_cast is well-defined).
Reinterpretation has plenty of use cases:
C-style “generic” APIs typically produce results in the form of a void *,
with the caller being expected to cast the void * to a suitable type.
Callers are responsible for ensuring that the destination type
is identical to or compatible with the type that was initially cast to void *.
C and C++ callback patterns frequently provide a void * parameter, allowing
users to supply additional data or context between callbacks. Each callback
is then responsible for casting to the appropriate type.
Pointer values occasionally need1 to be round-tripped through an integral type. C++ specifically allows this, so long as the destination integral type has at least sufficient width to represent all possible pointer values.
Polymorphism: the Berkeley sockets API
specifies connect(2)
as accepting a struct sockaddr *, which is actually reinterpreted internally
as one of the family-specific sockaddr structures (like sockaddr_in for IPv4
sockets). C++ also explicitly allows this under its “similarity” rules.
Cheap object serialization or conversion: related to the above, but slightly
different: both C and C++ are okay with you converting pretty much any object
to char *2. This allows objects to be treated as bags of bytes,
which is handy when writing a hash table (you don’t care what the contents
are, you just want to uniquely identify them) or when serializing structures
in a host-specific format3.
Each of the above is useful, but incredibly unsafe: transmutation is not an operation at runtime that turns one type into another, but rather a directive at compile time to treat some position in memory as if its type is different. The result: most possible transmutations between types result in undefined behavior.
Rust needs to interface with C, so4 Rust supports transmutation. It does so via
std::mem::transmute. But transmutation
is a fundamentally unsafe operation, so Rust forbids the use of mem::transmute except
for in explicitly unsafe contexts:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
use std::mem;
#[repr(C)]
pub struct Foo {
pub a: u8,
pub b: u8,
pub c: u8,
pub d: u8
}
#[repr(C)]
pub struct Bar {
pub a: u32
}
fn main() {
let foo = Foo { a: 0xaa, b: 0xbb, c: 0xcc, d: 0xdd };
let bar: Bar = unsafe { mem::transmute(foo) };
// output (on x86-64): bar.a = 0xddccbbaa
println!("bar.a = {:x}", bar.a);
}
(View it on Godbolt.)
transmute can, of course, be wrapped into safe contexts. But the underlying operation
will always be fundamentally unsafe, and should not be possible in otherwise safe Rust code.
So, how does totally_safe_transmute do it?
First, here’s the entirety of totally_safe_transmute:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#![forbid(unsafe_code)]
use std::{io::{self, Write, Seek}, fs};
pub fn totally_safe_transmute<T, U>(v: T) -> U {
#[repr(C)]
enum E<T, U> {
T(T),
#[allow(dead_code)] U(U),
}
let v = E::T(v);
let mut f = fs::OpenOptions::new()
.write(true)
.open("/proc/self/mem").expect("welp");
f.seek(io::SeekFrom::Start(&v as *const _ as u64)).expect("oof");
f.write(&[1]).expect("darn");
if let E::U(v) = v {
return v;
}
panic!("rip");
}
Let’s go through it, (mostly) line-by-line.
#![forbid(unsafe_code)]forbid is an attribute that controls the rustc linter (along with allow, warn, and deny).
In this case, we’re telling rustc to forbid anything that trips the
unsafe_code lint,
which does exactly what it says on the tin: catches use of unsafe.
In this case, forbidding use of unsafe doesn’t do anything: a quick read of the code shows
that unsafe never shows up. But it’s a top-level proof to the reader that, if rustc accepts
the code (and it does), then there is no use of unsafe.
totally_safe_transmuteHere’s our signature:
1
pub fn totally_safe_transmute<T, U>(v: T) -> U { ... }
In sum: totally_safe_transmute takes two type parameters: T and U.
It then takes one concrete parameter, v, which is of type T. Finally, it
returns a U.
We know that the job of a transmutation function is to reinterpret a type of some value as some other type, so we can rewrite this signature as:
1
pub fn totally_safe_transmute<SrcTy, DstTy>(v: SrcTy) -> DstTy { ... }
enum EOur next bit is a terse enum with some funky attributes. Rewritten with our friendly type
parameters:
1
2
3
4
5
6
#[repr(C)]
enum E<SrcTy, DstTy> {
T(SrcTy),
#[allow(dead_code)] U(DstTy),
}
let v = E::T(v);
First, we’re marking E as repr(C). This is an
ABI-modifying attribute: it tells
rustc to lay E out using the platform’s C ABI rather than the (intentionally) unstable Rust ABI.
What does this actually mean? For enums with fields (like this one), Rust
uses a “tagged union” representation.
In effect, E becomes something like this (in C syntax):
1
2
3
4
5
6
7
struct E {
int discriminant;
union {
SrcTy T;
DstTy U;
} data;
};
We’ll see why that’s important in a bit.
Next: E has two variants: the first holds a value of type SrcTy, and the other holds a value
of DstTy.
But wait! Another rustc linter annotation: this time, we’re telling rustc that it’s okay
for the U variant to fail the
dead_code lint.
Normally, rustc would warn us upon statically inferring that U is never used; with
dead_code enabled, it silences that warning. Like the ABI layout, we’ll see why that’s important
shortly.
Finally, we shadow our v parameter with a new binding. v was already of type T, so
creating an E::T from it is no problem at all.
This is where the (main) magic happens:
1
2
3
4
5
6
let mut f = fs::OpenOptions::new()
.write(true)
.open("/proc/self/mem").expect("welp");
f.seek(io::SeekFrom::Start(&v as *const _ as u64)).expect("oof");
f.write(&[1]).expect("darn");
First, we’re opening a file. Specifically, we’re opening /proc/self/mem in write mode.
/proc/self/mem is a very special5 file: it presents a view of the current process’s
memory, sparsely mapped by virtual address ranges.
As a quick hack, we can prove this to ourselves in Python by checking out the
in-memory representation of a str object6:
1
2
3
4
5
6
7
8
9
>>> x = "this string is long enough to prevent any string interning"
>>> # in cpython, an object's id is (usually) its pointer
>>> x_addr = id(x)
>>> hex(x_addr)
'0x7ff1bc7cfce0'
>>> mem = open("/proc/self/mem", mode="rb")
>>> mem.seek(x_addr)
>>> mem.read(len(x) * 4)
b'[SNIP] "thi\x00\x00\x00\x00\x00\x00\x00\x00this string is long enough to prevent any string interning\x00e\'[SNIP]'
(I trimmed the output a bit. You get the point.)
We can even poke memory by writing into /proc/self/mem:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
>>> # using ctypes to avoid the layout muckery we saw above
>>> import ctypes
>>> cstr = ctypes.c_char_p(b"look ma, no hands")
>>> cstr_addr = ctypes.cast(cstr, ctypes.c_void_p).value
>>> hex(cstr_addr)
'0x7f47f3e9c790'
>>> mem = open("/proc/self/mem", mode="r+b")
>>> mem.seek(cstr_addr)
>>> mem.read(len(cstr.value))
b'look ma, no hands'
>>> mem.seek(cstr_addr + 5)
>>> mem.write('p')
>>> mem.seek(cstr_addr)
>>> mem.read(len(cstr.value))
b'look pa, no hands'
The next two pieces of totally_safe_transmute should now make sense: we seek
to the address of our v variable (which is now a variant of E) within our own running process,
and we write a single u8 to it ([1]).
But why 1? Recall our C ABI representation of E above! The first piece of E is our
union discriminator. When data is SrcTy, discriminant is 0. When we forcefully
overwrite it to 1, data is now interpreted as DstTy!
Okay, so we’ve poked memory and turned our E::T into an E::U. Let’s see how we get it out:
1
2
3
4
5
if let E::U(v) = v {
return v;
}
panic!("rip");
At first glance, there’s nothing special about this: we’re simply discarding the enum wrapper
that we added earlier so that we can return our newly-minted value of DstTy.
But this is actually deceptively clever, and involves fooling the compiler:
totally_safe_transmute must return DstTy.DstTy is for v to be an E::U.v was unconditionally initialized as an E::T, so that return is never reached.panic!s.This is why we needed allow(dead_code) earlier: no E::U is ever constructed in a manner that
could possibly reach the return statement, so there’s simply no need for it as a variant.
And indeed, we can confirm this by removing the allow attribute:
1
2
3
4
5
6
7
8
9
10
11
12
13
$ cargo build
Compiling totally-safe-transmute v0.0.3 (/tmp/totally-safe-transmute)
warning: variant is never constructed: `U`
--> src/lib.rs:9:9
|
9 | U(U),
| ^^^^
|
= note: `#[warn(dead_code)]` on by default
warning: 1 warning emitted
Finished dev [unoptimized + debuginfo] target(s) in 0.15s
But alas: it’s not really dead code: the compiler is “wrong,” and we pop
an E::U into existence at runtime by modifying the program’s own memory. We then hit
our impossible condition, and return our transmuted value.
totally_safe_transmute is a delightful hack that demonstrates a key limitation when reasoning
about a program’s behavior: every behavior model is contingent on an environmental model and
how the program (or the program’s runtime, or the compiler, or whatever else) chooses (or doesn’t
choose) to handle seemingly impossible conditions in said environment.
The ability to do this doesn’t reflect fundamental unsafety in Rust, any more than it does
any safe language: from Rust’s perspective, what totally_unsafe_transmute does is impossible
and therefore undefined; there’s no point in in handling something that cannot happen.
Some other interesting bits:
/proc/self/mem.
Other OSes may have similar mechanisms.write would probably need to be adjusted.totally_safe_transmute
rewrites the in-memory representation of the program to accomplish equivalent behavior at runtime.
I don’t think this is a distinction that makes a difference.totally_safe_transmute relies on undefined behavior (an impossible program state),
Rust would be correct in erasing the E::U branch altogether and reducing the function to an
unconditional panic!. It doesn’t do that in my testing (even in release mode), but there’s
absolutely nothing in the program semantics that prevents it from doing so. But maybe
one day it will, and totally_safe_transmute will stop working!Usually for reasons of mis-design. But it happens. ↩
And a few others, like unsigned char* and std::byte. ↩
One of the great sins of application and network programming, and a common source of vulnerabilities. ↩
Among other reasons. As mentioned, transmutation is useful when all of need is a “bag of bytes” view of some object, or when you can guarantee consistent type layouts. It’s also useful for advanced lifetime hackery. ↩
And platform-specific: it’s part of Linux’s procfs. As a result, totally_safe_transmute won’t work (as-is) on other OSes. ↩
Which, as you’ll notice, is not trivial (it’s not just a length-data pair). unicodeobject.h in the CPython source has the full structure details, which are completely irrelevant to this post. ↩