HMM - Machine Learning - Fall 2021

Two weeks about hidden Markov models as part of the Machine Learning class at the Department of Computer Science, Aarhus University.

Lectures

Slides are preliminaries until the day of the lecture.

The pre-recorded lectures are from last year (Fall 2020), and is available as supplementary material. Wathcing these video lecture do not substitute attending the physical lectures.

Wednesday, Nov 3: Hidden Markov Models. Terminology and basic algorithms.
Reading material: Bishop section 13.1-13.2 (not 13.2.1, 13.2.4 and 13.2.6).
Supplementary material: See Rabiner for an alternative presetation of HMM methods and applications. See note about graphical models for background about probability theory and terminology.
Slides from lecture:
- HMMs - Terminology, representation and basic problems [video]
- HMMs - Algorithms for decoding [video]
Friday, Nov 5: Hidden Markov Models. Implementing the basic algorithms.
Reading material: Bishop section 13.2.4.
Supplementary reading: You can also take a look at Appendix B - Floating Point Numbers from A. Tanenbaum, Structured Computer Organization, in order to se why we do not want numbers to become too small.
Slides from lecture:
- HMMs - Implementing the forward-, backward-, and Viterbi-algorithms [video]
Wednesday, Nov 10: Hidden Markov Models. Training and selecting model parameters.
Reading material: Bishop section 13.2.1.
Slides from lecture:
- HMMs - Training - Selecting model parameters [video, part 1] [video, part 2]
Friday, Nov 12: Hidden Markov Models. Selecting the initial model parameter, using HMMs for (simple) gene finding.
Slides from lecture:
- HMMs - Selecting the initial model parameters and using HMMs for (simple) gene finding [video]
Wednesday, Nov 17: Hidden Markov Models. Some useful extensions.
Slides from lecture:
- HMMs - Some useful extensions [video]
- HMMS - Hand In 3 [video]

Exercises

The exercise texts are preliminary until the Friday the week before the exercises are planned.

Solutions to theoretical exercises will be posted after the exercise sessions. Solutions to the practical exercises will not be posted as these are a part of hand in 3.

Week 45 (8/11 - 11/11):
- Theoretical exercises: html, ipynb (exercise 1 refers to the image graphical-models.png, download it and put it in the same directory as the notebook in order to display it in the notebook) [solutions (pdf)]
- Practical exercises: html, ipynb
Week 46 (15/11 - 19/11):
- Theoretical exercises: html, ipynb
- Practical exercises: html, ipynb

Project

There is a mandatory project (hand in 3) about HMMs, which relate to the practical exercises above. The deadline for hand in 3 is Wednesday, December 8, 23:59.

Project description (hand in 3): html and ipynb.

Material

[Bishop]: Christopher M. Bishop, Pattern Recognition and Machine Learning, Cambridge University Press.
[Rabiner]: Lawrence R. Rabiner. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceeding of IEEE, 77:257-286, 1989.
[Mailund]: Thomas Mailund. Conditional probabilities and graphical models.