VEX is the JIT at the core of Valgrind. It unpicks blocks of machine code, hands them off to the tool for instrumentation, recompiles the result, and links the instrumented version into the running image. By using a target independent intermediate representation, it insulates tools from the complexity of the underlying instruction sets.
Back in 2003, when the framework was designed, I never dreamt that it would end up supporting X86, ARM, POWER, MIPS, S390 and TILEGX in both 32- and 64-bit variants. From that perspective VEX has been amazingly successful. But the framework is now showing its age. Recent instruction set features (transactional memory, LoadLinked/StoreConditional, wide vectors) have proven difficult to implement. It supports precise memory exceptions only poorly. And perhaps worst, its simplistic compilation pipeline causes it to generate code that is uncompetitive compared to other frameworks, particularly DynamoRIO and PIN.
In this talk I'll outline VEX's structure, then talk about these problems and what can be done about them. And I'd be particularly interested to hear opinions on how much effort, and for which problem areas, should be invested in upgrading it.
For the audience, some background in compiler internals and assembly code programming would be helpful, but is not essential.
Speakers: Julian Seward