Matias Vara

Unikernels aims at improving the way single purpose systems are built with minimalist kernels that the user’s application compiles within. This results in deployments that require less memory, less disk, less cpu and less time to be up and running. Also, the whole system spends most of the time in the user application or doing IO for that single application thus cpu time is used more efficiently. In this presentation, we talk about the use of unikernels for High Performance Computing. We present a work-in-progress that aims at implementing the MPI standard on top of Toro, an open-source non-POSIX unikernel. In this work, we implement a library that conforms to the Open MPI implementation. This library relies on Toro API to implement the MPI functions. In particular, the library leverages Toro’s features like per-CPU memory allocation, cooperative scheduler, thread migration and inter-core communication based on Virtio. During the initialization, Toro creates one instance of the MPI application per core. Each instance is a thread that is migrated to the corresponding core and then executes without any interference. When applications are required to allocate memory, each core has its own memory pool from where memory is allocated. This allows us to keep memory allocation local, thus improving the way cache is used. Also, primitives like MPIGather() or MPIScatter() that require communication between instances are implemented by relying on a new Virtio device named virtio-bus that allows core-to-core communication without locking. At the moment, we have implemented the following APIs: - MPIGather() - MPIScatter() - MPIReduce() - MPIBarrier() The goal of this PoC is to port benchmarks from the osu-microbenchmarking (http://mvapich.cse.ohio-state.edu/benchmarks/) to compare with existing implementations. During the presentation, we present how this is implemented and we demonstrate the use of the current implementation by executing different MPI applications on top of Toro.

This talk presents ToroV, a novel open-source technology that combines virtualization and containerization to enable the execution of users’ applications in a safer and improved manner. In ToroV, applications run as Virtual Machines without the need of an OS, unikernel nor device-model. ToroV combines a minimalist Virtual Machine Monitor and a virtualized guest program communicating through POSIX APIs. When the guest application requires to open or write a file, it just invokes the VMM using hypercalls. The VMM intercepts those hypercalls, processes the request, and returns to the guest. The sysadmin defines the ACL (Access Control List) of authorized hypercall per virtualized guest application. This allows the user to control the host’s surface that is exposed to the guest. In this talk, we propose to present the ToroV architecture together with several ongoing experiments. For example, the minimalist VMM allows us to boot up a VM in KVM in less than 6 ms. Also, we show how we debug guest applications by simply using GDB and the KVM API for debugging. During the talk, we discuss the main differences with unikernels and containers and how ToroV gets the best of both worlds. Also, we present the main differences with gVisor, which is a similar project from Google.

Microservices propose a new paradigm to split monolithic cloud infrastructure into unitary services which allow better scaling of the global infrastructure by integrating heterogeneous technologies. Current cloud infrastructure allows to implement cheap microservices by hosting them in VMs. To leverage host resources, VMs are launched on demand, i.e., when the microservice is idle, the VM is shutdown and it is only powered on when a request arrives. In this context, the time that takes the VM to be ready and serves incoming requests becomes very important. We have identified two problems that prevent VMs to start up on time: first, the initialization of the hypervisor, and second, the size of the image. In this talk, we present three approaches that work around on these problems, i.e., Qboot, NEMU and Firecracker. We evaluate these approaches on a Toro appliance that is a unikernel dedicated to microservices. In Toro, the kernel and the microservice are compiled together and the result is a single image that can be used to launch a Virtual Machine (VM) on different hypervisors like Xen, KVM or HyperV.

Toro is a x64 kernel that dedicates all system resources to the embedded user application. The result of the compilation of Toro is an image that contains both the kernel and the application. The kernel image can be booted by all majors hypervisors Hyper-V, KVM, VMWare ESXi, and emulators Qemu, VMWare WorkStation, VirtualBox. In these environments, the application runs directly inside the kernel without the need of a general purpose operating system. In this talk, I will present recent experimentation in which the scheduler of Toro is optimized to reduce the CPU usage of a VM. It has been identified three cases in which the scheduler can be optimized to reduce the impact of idle loops. To validate these techniques, I will present the impact of the improvements on a live running Toro VM on top of QEMU and Hyper-V. Open questions are welcome throughout the session to discuss the benefits and drawbacks of this approach but also take-away lessons.

In systems with high-grade multiprocessing, we have identified two bottlenecks: in the accessing of the memory bus and in the shared resources. To deal with these issues, we propose a dedicated kernel named TORO that is optimized to run a single multi-threading application. TORO is demonstrating an innovative operating system by integrating at the same ring level both kernel and the user application server. In addition, the accessing to resources (e.g., Filesystem, Networking, Memory) is dedicated to specific processor. Then, only the CPU where is dedicated the resource is allowed to access to the instance of the resource. As a result of this design, the kernel provides direct access to all resources without any overhead, and therefore maximizes performance for the overall system. To sum-up the approach, in TORO, the threads of the user application server are distributed evenly on all CPUs and running independently in parallel. The memory model chosen is NUMA without pagination. During the initialization, the memory is divided proportionally for each processor installed on the system. When a thread needs memory, the memory allocator returns a free block of memory depending on which CPU the thread is running. In the same way, TORO can dedicate resources to specific processor, i.e., a FileSystem. This only CPU then can access to this instance of FileSystem. The scheduler is based on the cooperative threading model, therefore due to this design, TORO can migrate threads between CPUs and send messages between threads without using any lock instruction.

I am Matias Vara (http://www-sop.inria.fr/members/Matias.Vara_Larsen/), founder and main developer of TORO (http://sourceforge.net/projects/toro/, http://torokerneleng.blogspot.com/) since 2006. Currently, I am doing a PhD in Computer Science at INRIA, Nice, France. FOSDEM is a very good place to discuss general purpose operating systems vs. dedicated operative systems. This discussion can be motivated by showing the characteristics of TORO by highlighting the main differences with a general purpose OS. The talk will take around 25 minutes.