Numpy is considered the go-to library for fast performant python scientifc computing, as it is a heavily optimized scientific computing library written in C. But is it really that fast (spoiler: yes, but not when used with python)? In this talk, I will be covering a real-world example in the domain of genetics, and demonstrating how I was able to out-perform numpy code running on my intel-7700k machine with a raspberry pi running code from the Nim programming language!
I will also be providing optimization tips for speeding up your code, with a focus on comparing python with the Nim programming language.
The talk can be roughly divided into the following sections:
Part 1 (32 min) - Introduction to myself. Describing a computationally expensive problem that I have experienced when working with Python.
- My background is in data science and biology
- Biologists tend to work with large datasets
- One common problem we face is preprocessing of large amounts of data. A common technique used here using distance calculations
- Why I decided to learn a different language
Part 2 (1 min) - High level introduction to Nim
- Nim is a statically compiled programming language
- Etc...
Part 3 (10 min) - Going through step-by-step on how to write performance optimal code with Nim. Describing common performance pitfalls that Python users may not be aware of.
- Inlining
- Using the type system to your advantage
- Taking advantage of domain knowledge
- Multi-threading (openmp)
- Bounds checking
- Compiler flags
- Mathematical and algorithmic optimizations
Part 4 (3 min) - Benchmarks and closing statement
- Show benchmark comparing the naive python numpy implementation with the highly optimized Nim one
- Top tips to keep in mind when writing high performance code.
Please note: the purpose of this talk isn't to be an exploration of some new state-of-the-art way of doing performance optimization or some relatively unknown optimization techniques. This is a general introduction for people hoping to use Nim for HPC that are coming from Python. Lots of emphasis will be put comparing the two languages.