11-30, 10:55–11:25 (Europe/Amsterdam), Auditorium
In this talk, we will delve into the various techniques and tools for ML model optimization during both training and inference stages. We will trace the journey of an ML model from its beginnings in a Jupyter notebook to its final deployment with a high-performance inference runtime. Along the way, valuable insights will be shared that you can seamlessly incorporate into your own workflow.
Overview
This introductory talk is designed to address the prevalent industry challenge of Machine Learning (ML) model deployment. Given the plethora of frameworks, compilers, and runtimes, ML engineers and Data Scientists often find this a daunting task. Our discussion will traverse all phases of this process, aiming to simplify the complexity.
We'll initiate the discussion with the ecosystem, tools, and methods, focusing initially on the training stage before transitioning to the inference stage. Beginning with a model in Jupyter Notebook, we'll illustrate how to expedite training time utilizing auto mixed precision, multiple GPUs, batch-size strategies, and JIT compilation. Subsequently, we'll concentrate on how to fine-tune your model inference for specific target hardware using necessary tools (NVIDIA Tensor RT (LLM) and Triton, ONYX) or adopting a more comprehensive approach with the new Mojo programming language.
Target Audience
Our primary audience is Machine Learning Engineers, Data Scientists, Software Engineers, and students aspiring to these roles. A basic understanding of Python and a healthy dose of enthusiasm are the sole prerequisites for this seminar.
Agenda
0-5 mins: Introduction to the Ecosystem and Tooling
5-15 mins: Strategies to Accelerate Model Training
15-25 mins: Techniques to Boost Model Inference
25-30 mins: Concluding Remarks and Future Directions
No previous knowledge expected
As an Engineering Lead with 7 years of experience, I specialize in projects that involve machine learning, software engineering, and cloud infrastructure. I am known for my practical and result-oriented approach, guiding customers through the entire lifecycle of data-driven projects, from requirement definition to production deployment and operations. My expertise lies in bridging the gap between innovative technologies and real-world applications, ensuring successful project outcomes. With a strong focus on delivering tangible results, I am skilled in leading end-to-end project execution, leveraging my deep understanding of machine learning, software engineering, and cloud technologies. My dedication to driving innovation and my ability to effectively collaborate with cross-functional teams make me an asset in delivering data-driven solutions that meet business goals.