Sequence Learning - CS4575

This course will be taught during the summer term only.

Content

This course will cover machine learning models and techniques for processing sequential data and time series. A major focus will be on recurrent structures and their learning rules, for instance, recurrent neural networks (RNNs) and backpropagation through time (BPTT)bidirectional RNNsgated RNNs, and reservoir computing. We will investigate online learning and modern, efficient alternatives to BPTT. Moreover, we will delve into the rising generation of energy-efficient spiking neural networks and discuss recent models of practical relevance. Furthermore, we will cover temporal convolution networks (TCNs)attention, the query-key-value principle, the transformer architecture, leading to the introduction of large language models (LLMs), and, finally, recent state space models (SSMs).