{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ML - Week 14 - Theoretical Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hidden Markov Models\n",
    "\n",
    "***Exercise 1***: Questions to slides *Hidden Markov Models - Training*:\n",
    "\n",
    "1. Explain why training by counting as explained on slides 3-9 yields a maximum likelihood estimate, i.e. why the parameters $\\Theta = (A_{jk}, \\pi_k, \\phi_{ik})$ as computed from a given ${\\bf X}$ and ${\\bf Z }$ cf. slide 9 maximizes the joint probability $P({\\bf X}, {\\bf Z} | \\Theta)$. \n",
    "\n",
    "2. Consider Viterbi training as explained on slides 18-19. If a parameter in the initial model ${\\bf \\Theta^0}$ is set to zero, i.e. if a particular transition or emission probability is set to zero, then it will remain zero during all the iterations of Viterbi training (if we do not perform pseudo counts). Why? \n",
    "\n",
    "3. Explain why you can stop Viterbi training if the Viterbi decoding does not change between two iterations? \n",
    "\n",
    "4. Consider EM for HMMs (Baum-Welch training as outlined on slides 30 and 46. It also has the property that if a parameter in the initial model ${\\bf \\Theta^0}$ is set to zero, i.e. if a particular transition or emission probability is set to zero, then it will remain zero during all the iterations of the EM training. Why? \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}