Topics: partitioning of large calculations, computation networks, python, celery, rabbitmq, redis
The open source OpenQuake seismic calculation engine (http://openquake.org/) is developed and maintained by the GEM foundation (http://www.globalquakemodel.org/) which is a public-private partnership that drives a collaborative effort aimed at developing and deploying tools and resources for earthquake risk assessment worldwide. Seismic hazard calculations can be massive making it necessary to partition and distribute them across a compute network. The OpenQuake engine (http://openquake.org/) partitions seismic calculation and uses - celery - rabbitmq - redis to push the tasks into the compute network and to collect the results. There were a number of lessons learned while scaling up OpenQuake to perform seismic calculation for the whole of Europe and I will touch upon the most important ones.
Speakers: Muharem Hrnjadovic