In this presentation we present some optimizations we have developed for the Minio object server. To maximize hashing performances for bit-rot protection we have exploited the Go Assembly capabilities to natively optimizing BLAKE2 and SHA2 techniques for both Intel and ARM platforms. In addition we will present a distributed locking package to aid with scaling a single server solution into a multi server solution.
In this presentation we would like to focus on a couple of developments that we have done for the Minio object server in terms of performance improvements and scaling techniques.
Regarding hashing we have developed Go packages for two hashing algorithms (BLAKE2 and SHA256) that take advantage of the Golang (or Plan9) assembly capabilities. These packages exploit SIMD instructions of Intel and the Cryptography Extensions for the ARM platform. The speed up range from 3 to 4 times (Intel) to 100X (on ARM). Due to the use of Golang assembly there is no need for a C-compiler in order to use these packages (when compiling from scratch). For the Intel platform SSE, AVX, and AVX2 specific version are available and the most performant architecture is automatically chosen.
In addition we have developed a distributed locking and syncing package for Go. Its main features are: - Simple design for avoiding many tricky edge cases - No need for a master node - Resilient so other nodes are not affected when a node goes down - Drop-in replacement for sync.RWMutex -Automatically reconnect to (restarted) nodes - Limited scalability of up to 16 nodes
We are successfully using this package for the Distributed version of the Minio Object Server
Speakers: Frank Wessels