41
loading...
What does it mean to scale your python powered ML pipeline?
Scaling, in general, is starting out with a small solution and growing it into a bigger solution.
Different people mean different things when they talk about scaling. Do you want to scale a mountain, a web app, or an ML pipeline?
Even when talking about machine learning, scaling can mean different things:
Scaling requires resources and cost to execute. The cost can be in the form of:
There are two important things to consider before you invest in scaling:
What are your goals? You must decide beforehand, what is good enough? Over-engineering is a trap, all engineers are familiar with. Don't over-engineer!
Measure your code? To treat a patient, a doctor has to determine what the illness is? Likewise, you need to measure your code for bottlenecks before deciding on a scaling strategy.
There are many tools that can use to profile your python code. Scalene is one of the most complete measuring tools available at the moment.
Increase code efficiency: Python is designed for ease of use and easy extension, but not performance. As a developer, the onus is on you to do more work so that the application executes less code. Whenever possible use vectorized library functions instead of loops.
Python is successful in data science because of the pre-compiled code offered by data-appropriate libraries in the pydata stack such as pandas and numpy.
Adding parallelism: You can achieve parallelism on a single machine or by using multiple machines. In principle, you want to stay on a single machine as much as possible as distributed computing increased complexity because of the following challenges:
- network transfer
- task management
- cluster management
2a. Hardware accelerators: You can leverage GPUs and
TPUs for your ML workflows. Libraries such as CuPy and
RAPIDS leverage GPUs. There are non-GPU accelerators also
like TPUs offered by Google and Apple Neural Engine.
TPUs for your ML workflows. Libraries such as CuPy and
RAPIDS leverage GPUs. There are non-GPU accelerators also
like TPUs offered by Google and Apple Neural Engine.
2b. Distributed Computing: Going distributed is much
harder for the aforementioned reasons but if you have to
then you can look at the following projects:
harder for the aforementioned reasons but if you have to
then you can look at the following projects:
Whichever direction you take on your scaling journey, be sure to pause before you commit, and solve one problem at a time.
P.S. I'll cover I/O bottlenecks in a separate post.
41