Tarantool is an open source in-memory database and execution platform, based on Lua. The database supports transactions, secondary keys, replication, triggers, and the application server provides an inside-the-database semantics for cooperative multitasking and non-blocking I/O. This presentation focuses on a practical use case: task queue application, using Tarantool as an application server and a database.
The idea of the task queue is that producers put tasks (objects) into a queue, and consumers take tasks, perform them, mark as completed.
The queue must guarantee certain properties: if a consumer failed, a task should return to the queue automatically, a task can't be taken by more than one consumer, priorities on tasks should be satisfied.
With Tarantool, a task queue is a distributed networked application: there are multiple consumer/producer endpoints (hosts) through which a user can interact with the queue.
The queue itself is a fault-tolerant distributed database: every task is stored in Tarantool database and replicated in multiple copies.
If a machine goes down, the state of a task is tracked on a replica, and the user can continue working with the queue through a replica. Total power failure is also not an issue, since tasks are stored persistently on disk with transactional semantics.
Performance of such an application is in hundreds of thousands of transactions per second.At the same time, the queue is highly customizable, since it's written entirely in Lua, is a Lua rock, but the code is running inside the database. This is the strength of Lua: one size doesn't have to fit all, and you don't have to sacrifice performance if you need customization.
Speakers: Konstantin Osipov