Power up your work with compiling and profiling

By Cheuk Ting Ho

Elevator Pitch

Have you been troubled by Python code that took too long to run? Do you want to know why and how to improve? In this workshop, we will introduce Numba - a JIT compiler that is designed to speed up numerical calculations. We will provide all the knowledge that you need to make Numba works for you.

Description

Have you ever heard of Numba? It is (mainly) a JIT (Just-In-Time) compiler to make your math-heavy Python code runs faster under certain conditions. Most people found all of it is like a mystery - It sounds like magic, but how does it work? Under what conditions does it work? And because of it, new users found it hard to start using it and it requires a steep learning curve to get the hang of it.

This workshop requires no prior experience. However, it will be most beneficial to those who are working with numerical data, like data scientists and researchers. We also expect participants have no knowledge of how compilers work and not much understanding of how CPython works. Through exercises, we will explore in what situation Numba works, when it does not and the reason why. We will also look at some cases where we can make Numba works by changing a few things in your code. Hopefully, by finishing the workshop, you will have a better understanding of how Numba works before you even start using it. This knowledge can save you some time on try and error, making your experience in using it better.

For whom is your Workshop

Data scientists or developers who have math-heavy code that would like to speed up with the benefit of Numpy and Numba.

Workshop Agenda

  • brief introduction (15 mins): To introduce what is Numba and how it works in general. Overview of what exercises we will cover at the hands-on session and what they should know about when they start.

  • exercise that we do together (60 mins): Through various exercises, we will get more understanding of how Numba actually works. This can explain some questions that new users may have about Numba. In this part, we will go through 2 Jupyter notebooks: 1) We will first go through a Jupyter notebook that gives an overview of how Numba compile the function to make code go fast. Where it truly shines and compares its performance with the original pure Python code. 2) The second notebook will go through some common mistake people make and we try to troubleshoot some of the problems users may encounter.

  • break to refresh (30 mins): break between exercises. Can be adjusted to be in line with the coffee break at the conference.

  • hands-on exercise to be finished at own pace (90 mins): In this part, there will be more deep dive into the use of Numba and profiling of the work in order to understand more about how Numba is working under the hood and polish the skill to find where the code can be improved. Participants are required to finish most of the exercises but not all, so they can finish them in pairs or at their own pace. Or even continue after the workshop. All the exercises in this part will be working towards a small project in which the participants can learn the process of how to profile and speed up their work: 1) First learn how to implement a neural network using Numpy 2) Profile the process and see if there are areas there can be improvements 3) Use Numba to speed up and compare the result to the first exercises.

  • conclusion on what is taught in the exercises (15 mins): At the end, we will go through some of the findings participants would have by the end of the exercises. This is to make sure everyone has got the right ideas in the end.

What is required from attendees

  • A computer with a stable internet connection (useful to look up information);
  • Python 3.8 or above;
  • Jupyter and Numba installed (detail about Python libraries required will be announced later);
  • An opened mind and ready to learn something new

What Attendees will Learn

By the end of the workshop, you will have some understanding of what Numba is and how it speeds up your Python code. You will also have a better idea about the limitation of Numba and when it does not help. You may also know how to change your code to make it benefit from the speeding up of Numba. You will also learn some troubleshooting skills and where to look for help if got stuck in the future.