This talks will present how to perform benchmarking of Big Data systems, from low-powered devices, HPC clusters, to cloud IaaS and PaaS. It will guide participants on how to define clusters, select benchmark suites and configuration with the ALOJA open source tools. ALOJA (http://aloja.bsc.es) is a research initiative from the Barcelona Supercomputing Center to explore new cost-effective hardware architectures for Big Data. During its first year, ALOJA's benchmarking efforts have produced the largest public repository with over 50,000 Hadoop benchmark runs. The searchable repository features different applications for Hadoop, software configurations, data sizes, and more than 100 different hardware deployment options.
The talk will cover the following topics:
-Cluster definition and automated deployment on local (vagrant clusters), on-prem and cloud environments
-Automating and orchestrating OS, Hadoop, JVM configuration across clusters
-Benchmark selection and iteration of configurations
-Metrics collections, results gathering, and importing
-Advanced data views for aggregate results with filters
-Analyzing performance results and selecting configurations
Speakers: Nicolas Poggi