Some projects I'm using to learn AI since my MS

First follow specific instructions for your operating system listed in subheaders below.

Consider modifying pyproject.toml to handle your necessary cuda version as an optional dependency.

Run uv sync to install all dependencies.

Jupyter is great to work in, but it has several shortcomings:

1. A lack of a proper git diff when making a PR
2. You can't import jupyter notebooks from other jupyter notebooks
3. They can't be linted or formatted by tools like ruff or pyright

So I am using jupytext to make syncronized copies of the jupyter notebooks in plain text .py format.

If you then wrap cells (the code you run) in if __name__ == "__main__" you also now gain the ability to use these notebooks as importable libraries in future work.

I use logseq to manage my ./notes and flashcards. This uses the zettelkasten method to create a knowledge graph, which is perfect for studying.

See .github/pull_request_template.md for the styleguide.

brew install sdl sdl_ttf sdl_image sdl_mixer portmidi

sudo apt-get install libsdl2-dev libsdl2-image-dev libsdl2-mixer-dev libsdl2-ttf-dev libfreetype6-dev libportmidi-dev libjpeg-dev python3-setuptools python3-dev python3-numpy

• Emoji Meanings

  • ❗ Indicates Priority
  • 📖 Paper Read
  • 📓 Notes Taken
  • 💻 Implementation Completed

• Reinforcement Learning

  • Value Based Methods - I'm pretty much up to date with these methods, but might as well implement them. I may go into less explanation though.
    • 📖📓💻 $TD(\lambda)$
    • 📖📓💻❗ Deep Q Learning
      • https://lightning.ai/docs/pytorch/LTS/notebooks/lightning_examples/reinforce-learning-DQN.html
    • 📖❗Prioritized Experience Replay
    • 📖❗Double Q Learning
    • Dueling Q Learning
    • Multi Step Learning
    • Distributional DQN
    • Noisy Nets
    • 📖 RAINBOW
  • Policy Based Methods
    • 📖📓💻 REINFORCE *
    • 📖❗ Actor-Critic (A2C, A3C) *
    • Trust Region Policy Optimization (TRPO)
    • [ ]❗Proximal Policy Optimization (PPO) *
    • Deep Deterministic Policy Gradient (DDPG)
  • Model Based Reinforcement Learning
    • 📖❗AlphaZero
    • MuZero
      • https://deepmind.google/discover/blog/muzero-mastering-go-chess-shogi-and-atari-without-rules/
    • Dreamer
      • https://research.google/blog/introducing-dreamer-scalable-reinforcement-learning-using-world-models/
    • Efficient Zero
    • Efficient Zero V2
    • SIMA
      • https://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/
    • Genie 1
      • https://deepmind.google/research/publications/60474/
    • Genie 2
      • https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/
  • Exploration in RL
    • Go-Explore
    • NoisyNet
    • DQN-PixelCNN
    • #Exploration
    • EX2
    • ICM
    • RND
    • NGU
    • Agent57
    • VIME
    • EMI
    • DIYAN
    • SAC
    • BootstrappedDQN
    • PSRL
    • HER
    • DQfD
    • R2D3
  • Multi Agent RL
    • Emergent Communication through Negotiation
    • Warp Drive
      • https://lightning.ai/docs/pytorch/LTS/notebooks/lightning_examples/warp-drive.html
  • Human-Timescale Adaptation in an Open-Ended Task Space
    • Muesli
    • Transformer-XL
    • Robust PLR
  • Distributed RL
    • Survey
    • RLLib

• Transformers

  • Tokenization
  • Word Embeddings
  • 📖❗Transformers
    • https://pytorch.org/tutorials/beginner/transformer_tutorial.html
    • https://lightning.ai/docs/pytorch/stable/notebooks/course_UvA-DL/05-transformers-and-MH-attention.html
  • 📖❗BERT
  • [ ]❗Sentence-BERT
  • Fine Tuning
  • RLHF
  • Direct Preference Optimization
  • Multimodality
  • Mamba and SSM's
  • Sentence Transformers
  • Multi token prediction
  • Time Series
    • https://www.datadoghq.com/blog/datadog-time-series-foundation-model/

• RAG

  • 📖 Survey on RAG
    • [ ]❗REALM
    • [ ]❗Hyde
    • [ ]❗DPR
    • [ ]❗Raft
    • PRCA
    • EAE
    • MIPS
    • Self reinforce
  • Survey on Graph RAG

• Diffusion Models

  • https://lightning.ai/lightning-community-labs/studios/build-diffusion-models-with-pytorch-lightning-hf-diffusers

• [ ]❗Graph Neural Networks (GNN)

  • https://lightning.ai/docs/pytorch/stable/notebooks/course_UvA-DL/06-graph-neural-networks.html

• Cognitive Science

  • Hopfield Network
  • Boltzman Machine
  • Conformal Prediction
  • Predictive Coding Models
  • Liquid Neural Networks

• Techniques

  • Profiling
  • Debugging Metrics

• Key Papers in Deep RL
• Spinning Up
• Lightning Tutorials
• Hugging Face RL
• Pytorch