This repository contains information about the course on Advanced Data Analysis and Machine Learning, spanning from weekly plans to lecture material and various reading assignments. The emphasis is on deep learning algorithms, starting with the mathematics of neural networks (NNs), moving on to convolutional NNs (CNNs) and recurrent NNs (RNNs), autoencoders and other dimensionality reduction methods to finally discuss generative methods. These will include Boltzmann machines, variational autoencoders, generalized adversarial networks and more.

FYS5429 zoom link https://msu.zoom.us/j/6424997467?pwd=TEhTL0lmTmpGbHlnejZQa1pCdzRKdz09

Meeting ID: 642 499 7467 Passcode: FYS4411

Furthermore, all teaching material is available from this GitHub link.

• Presentation of course and overview
• Discussion of possible projects
• Deep learning, neural networks, basic equations
• Recommended reading Goodfellow et al chapters 6 and 7
• Video of first lecture at https://youtu.be/dP8g_tjQ_9c Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week1/ipynb/week1.ipynb

• Mathematics of deep learning, basics of neural networks
• Introducing TensorFlow and Pytorch
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week2/ipynb/week2.ipynb
• Recommended reading Goodfellow et al chapters 6 and 7 and Raschka et al chapter 11. For Pytorch see Raschka et al chapter 12.

• Implementing neural networks with applications to differential equations
• Discussion of first project
• More on TensorFlow and Pytorch
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week3/ipynb/week3.ipynb
• Recommended reading Goodfellow et al chapters 6 and 7 and Raschka et al chapter 11. For Pytorch see Raschka et al chapter 12.

• Convolutional neural networks (CNNs), basic mathematics
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week4/ipynb/week4.ipynb
• Recommended reading Goodfellow et al chapter 9. Raschka et al chapter 13

• Convolutional neural networks (CNNs), building our own code
• Excursion into Graph CNNs
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week5/ipynb/week5.ipynb
• Recommended reading Goodfellow et al chapter 9. Raschka et al chapter 13

• Examples of applications of CNNs
• Recurrent neural networks
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week6/ipynb/week6.ipynb
• Recommended reading Goodfellow et al chapters 9 and 10

• Summary of RNNs and discussion of Long-Short-Term memory and short excursion into transformers
• Start discussions of Autoencoders
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week7/ipynb/week7.ipynb
• Recommended reading Goodfellow et al chapters 10 and 14

• Autoencoders and discussions of codes and links with PCA
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week8/ipynb/week8.ipynb
• Recommended reading Goodfellow et al chapter 14

• Autoencoders and discussions of codes and links with PCA
• Monte Carlo methods and structured probabilistic models for deep learning
• Partition function and Boltzmann machines
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week9/ipynb/week9.ipynb
• Reading recommendation: Goodfellow et al chapters 16-18

• Monte Carlo methods and structured probabilistic models for deep learning
• Boltzmann machines
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week10/ipynb/week10.ipynb
• Reading recommendation: Goodfellow et al chapters, 17, 18

• Boltzmann machines
• Variational autoencoders
• Reading recommendation: Goodfellow et al chapters 17, 18 and 20.1-20.7
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week11/ipynb/week11.ipynb

• Variational autoencoders
• Reading recommendation: Goodfellow et al chapter 20.10-20.14
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week12/ipynb/week12.ipynb

• Variational autoencoders
• Generative Adversarial Networks (GANs)
• Reading recommendation: Goodfellow et al chapter 20.10-20.14
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week13/ipynb/week13.ipynb

• Generative Adversarial Networks (GANs)
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week14/ipynb/week14.ipynb

• Generative Adversarial Networks (GANs)
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week15/ipynb/week15.ipynb

• Generative Adversarial Networks (GANs)
• Slides at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week16/ipynb/week16.ipynb

o Goodfellow, Bengio and Courville, Deep Learning at https://www.deeplearningbook.org/

o Brunton and Kutz, Data driven Science and Engineering at https://www.cambridge.org/highereducation/books/data-driven-science-and-engineering/6F9A730B7A9A9F43F68CF21A24BEC339#overview

o Sebastian Raschka, Yuxi Lie, and Vahid Mirjalili, Machine Learning with PyTorch and Scikit-Learn at https://www.packtpub.com/product/machine-learning-with-pytorch-and-scikit-learn/9781801819312, see also https://sebastianraschka.com/blog/2022/ml-pytorch-book.html

o David Foster, Generative Deep Learning, https://www.oreilly.com/library/view/generative-deep-learning/9781098134174/