Neurocomputing

Overview

This website contains the materials for the module Neurocomputing taught by Dr. Julien Vitay at the Technische Universität Chemnitz, Faculty of Computer Science, Professorship for Artificial Intelligence.

Each section/lecture is accompanied by a set of videos, the slides and a written version of the content. The (slightly outdated) videos are integrated in the lecture notes, but you can also access the complete playlist on Youtube.

Exercises are provided in the form of Jupyter notebooks, allowing to implement in Python at your own pace the algorithms seen in the lectures and to learn to use machine learning libraries such as scikit-learn, keras and tensorflow. A notebook to work on (locally or on Colab) and the solution are available in the Exercises section.

Syllabus

Lectures

1. Introduction
   1. Introduction
   2. Math basics (optional)
   3. Neurons
2. Linear algorithms
   1. Optimization
   2. Linear regression
   3. Linear classification
   4. Learning theory
3. Neural networks
   1. Multi-layer perceptron
   2. Modern neural networks
4. Computer Vision
   1. Convolutional neural networks
   2. Object detection
   3. Semantic segmentation

5. Generative modeling
   1. Autoencoders
   2. Restricted Boltzmann machines (optional)
   3. Generative adversarial networks
6. Recurrent neural networks
   1. Recurrent neural networks, LSTM
   2. Natural Language Processing and attention
7. Self-supervised learning
   1. Transformers
   2. Contrastive learning
   3. Vision transformers
8. Outlook
   1. Limits of deep learning
   2. Beyond deep learning

Exercises

Notebooks and videos are in the List of Exercises. Below are links to the rendered solutions.

1. Introduction to Python
2. Numpy and Matplotlib
3. Linear Regression
4. Multiple linear regression
5. Cross-validation
6. Linear classification
7. Softmax classification

8. Multi-layer perceptron
9. MNIST classification using keras
10. Convolutional neural networks
11. Transfer learning
12. Variational autoencoder
13. Recurrent neural networks

Recommended readings

• (Murphy, 2022) Kevin Murphy. Probabilistic Machine Learning: An introduction. MIT Press, 2022. https://probml.github.io/pml-book/book1.html

• (Goodfellow et al., 2016) Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org.

• (Chollet, 2017) François Chollet. Deep Learning with Python. Manning publications, 2017. https://www.manning.com/books/deep-learning-with-python.

• (Haykin, 2009) Simon S. Haykin. Neural Networks and Learning Machines, 3rd Edition. Pearson, 2009. http://dai.fmph.uniba.sk/courses/NN/haykin.neural-networks.3ed.2009.pdf.

Chollet, F. (2017). Deep Learning with Python. Manning publications https://www.manning.com/books/deep-learning-with-python.

Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning. MIT Press http://www.deeplearningbook.org.

Haykin, S. S. (2009). Neural Networks and Learning Machines, 3rd Edition. Pearson http://dai.fmph.uniba.sk/courses/NN/haykin.neural-networks.3ed.2009.pdf.

Murphy, K. P. (2022). Probabilistic Machine Learning: An introduction. MIT Press https://probml.github.io/pml-book/book1.html.