HACKER Q&A
📣 p1esk

Advanced Pytorch Course/Workshop?


I have a solid intermediate Pytorch knowledge that I accumulated over the last ~3 years implementing various models and reading docs. However I feel like I still don't quite understand what's going on under the hood, and would enjoy listening to a Pytorch expert explaining advanced topics. My employer offered to pay for a training course or a workshop, but unfortunately I can't find anything suitable.

Topics I'm interested in are autograd mechanics, debugging, profiling, speed/memory optimizations, distributed training, implementing custom ops and layers (I have a little bit of cuda experience), torchscript, model deployment, a tour of aten library, and other topics that can be considered "advanced".

I am not at all interested in deep learning basics or details on how to implement any particular model types, unless they serve as a good illustration of the advanced Pytorch features.

After some googling I found a few relevant blog posts: https://pytorch.org/blog/a-tour-of-pytorch-internals-1/ http://blog.christianperone.com/2018/03/pytorch-internal-architecture-tour/ http://blog.ezyang.com/2019/05/pytorch-internals/

These seem to be at the right level of detail, and I'm wondering if there's a way to learn all that in an interactive manner somehow. I believe I could get another 5-8 people from work who would also be interested in attending a workshop like this.


  👤 t-vi Accepted Answer ✓
Sounds like my advanced PyTorch course (except that I didn't put as much emphasis on distributed). :) I haven't become enchanted with moving to online yet. You ask about interactivity, and that seems to be the difficult part to me. For the exercises, I haven't found anything just like sitting in front of the same screen with my students.