Intro

I wanted to showcase some of the courses I’ll be taking in probably my hardest term yet at WPI. Needless to say, I’ll probably be very busy and won’t be able to upload as many posts as I could over break.

Courses

WPI runs on 7-week terms, so each of these courses will be short but packed with information.

I’ll be taking my first 4000 level Math course related to Stochastic Processes. Our first lecture went over a review of Markov Chains, so it was a good review for RL!

I’ll also be taking my first 4000 level ECE course, Computer Organization and Design. I’ve read Patt & Patel as well as Hennessy & Patterson, so classes like these really peak my curiosity in the field of computer architecture. My professor said our project will be creating our own assembly language based on the MIPS ISA, so I’m excited to add some cool assembly instructions that can get lots of stuff done. I want to try implementing at least one vector instruction, but it’s easier said than done. I’m also planning on taking Computer Architecture as a graduate level course next year.

Finally, I’ll be taking a 4000 level CS course, Numerical Methods for Linear and Nonlinear Systems. I expected this to be more of a Linear Algebra course, but I’ve been pleasantly surprised with the amount of calculus and algorithmic thinking involved. I’ve also learned a whole lot about quantization in the first 3 lectures so far, learning a lot about error analysis and floating point representation.

To be honest, I was quite average at calculus in the past, in large part because I’ve been ‘forced’ to take those courses in order to graduate. Looking back on the beauty of how calculus works and seeing it from an approximation perspective, it’s truly jaw-dropping at how some of these concepts work. I’ve found that I’m able to do pretty well in classes that I naturally find interesting, as my curiosity in the subjects force me to explore outside of class for fun, not as a means to an end for a good grade.

Next Steps

I plan on doing my best to contribute to NVIDIA/Apex, a framework for mixed precision & distributed training in Pytorch. Based off what I’ll learn about quantization, maybe I’ll see if I can contribute to torch/ao as well.

I also plan on sharing what I know about torch.compile, breakpoints, and a new project I’m working on inspired by some papers I’ve been reading. I’ve been wanting to create a project for a while that combines a lot of moving pieces together to create a large codebase that somehow ends up working out.

I hope to also pick up some research in the field of Computer Architecture or Operating Systems, Deep Learning, or something along those lines.

Lastly, I plan on doing the TinyML course by Song Han, which should be fun since it’ll get into GANs and Diffusion Models, as well as efficient inference!

After all this, I think I’d like to re-implement NanoChat from Andrej Karpathy from scratch. It’s easy to fool yourself into thinking you understand transformers, but as the saying goes, “If I can’t build it, I can’t understand it.”

Cheers, Syed