29C3 - Version 1.9

F/a{hr-p).l//a,n
2.9/C-3

Speakers
Julian Bangert
Sergey Bratus
Schedule
Day Day 4 - 2012-12-30
Room Saal 4
Start time 17:15
Duration 01:00
Info
ID 5265
Event type Lecture
Language used for presentation English
Feedback

Page Fault Liberation Army or Gained in Translation

a history of creative x86 virtual memory uses

x86 processors contain a surprising amount of built-in memory translation logic, which is driven by various data tables with intricate entry formats, and can produce various kinds of traps and other interesting computational effects. These features are mostly relics of earlier, more civilized times, when Jedi Knights tried to protect the Old Republic OSes with segmentation, supervisor bits, and hardware task support, but were defeated by processor de-optimizations and performance concerns and left unused by both Windows and UNIX systems – and explored only by hackers. For the rest of the world, an x86 PC was a "von Neumann architecture" with most of its strangeness unused.

In reality, the x86 memory system is a weird love child of von Neumann and Harvard, due to the split paths that code and data bytes are fetched through; the separate Translation Lookaside Buffers (TLBs) give a degree of control over address translation logic, and can be used to hide code from scanning the way ShadowWalker did. In multiprocessor systems, seemingly innocent optimizations like Paging Structure Caches can lead to two processors seeing the same address space differently, which creates unexpected bugs for kernel developers and opportunities for rootkit authors, which we will discuss.

In this talk we will give a (nearly) complete historic overview of creative uses of memory-related traps and faults by hardening patches such as OpenWall, PaX, and other less known but interesting projects, as well as by rootkit designs such as ShadowWalker, and by unorthodox reverse engineering and debugging systems such as OllyBone. We will then show some novel tricks with the x86 systems to both conceal and protect memory contents.

Every address a program issues, calls, or jumps to is an illusion or even a composition of several illusions created by different pieces of the MMU. In this universe of illusions, memory translation is what holds its together, and on x86 it's underappreciated and underused. Not only is the MMU the habitual liar, it could also be a schisophrenic one. Ain't that nifty?