Many language runtimes like the JVM rely on JIT compilation to improve
application performance, but JIT compilers actively compete with
applications for CPU and memory resources. Because of this, a JIT
compiler's activities may hinder application throughput and create
performance hiccups that can affect quality of service while also
complicating resource provisioning. On top of that, it takes a while even
for the best JIT compilers to fully compile the performance critical
methods of an application to deliver the steady state performance we all
expect from the JVM. An approach we are exploring is to decouple the JIT
compiler from the rest of the JVM so that JIT compilation can operate as a
separate, remote service. This approach opens up the possibility of an
elastic, cloud based service offering JIT compilation for many
applications, languages, and even different architectures simultaneously.
In this talk we will present our experience building an open source
JIT-as-a-service prototype based on the Eclipse OpenJ9 JVM. We will discuss
some of the engineering challenges we encountered, demonstrate the
advantages we've seen thus far, and present performance data for enterprise
grade Java EE benchmarks.