7.2 The hidden Markov model

The derivative with respect to $\pi_{k}$ is exactly as above, just confined to the very first sample ($t=1$), and so the optimal prior probability is

Likewise for the class-conditional means and covariances, although here we have to remember to sum across all samples:

Finally, we derive the optimal state-transition probabilities:

The optimal parameters again have intuitive interpretations. The class-conditional means and covariance are computed exactly as with the GMM, except that the averages are now across time and independent sequences rather than independent samples. In the case of EM, there is one other crucial distinction from the GMM: The posterior means of the HMM are conditioned on samples from all time; that is, they are the smoother means. This distinction has no relevance in the GMM, which has no temporal dimension.

Similarly, the optimal mixing proportions for the initial state look like the optimal mixing proportions for the latent classes in the GMM, although the sample average for the HMM is over sequences, rather than individual samples. One upshot is that it would be impossible to estimate the initial mixing proportions properly without access to multiple, independent sequences (this makes sense). And again, we must be careful in the case of EM: the expectation should be taken under the smoother distribution over the initial state. That is, to estimate the initial mixing proportions, the inference algorithm should first run all the way to the end (filter) and back (smoother)! This may at first be surprising, but note that future observations should indeed have some (albeit diminishing) influence on our belief about initial state. (We can imagine, colorfully, an unexpected future observation in light of which we revise our belief about the initial state: “Oh, I guess it must have started in state five, then….”)

We turn to the remaining parameters, the elements of the state-transition matrix, $\mathbf{{A}}$. When the states have been fully observed, the optimal $a_{ij}$ is merely the proportion of times state $j$ transitioned to state $i$ (rather than some other state). Under EM, the numerator is the expected frequency of state $j$ followed by $i$, where the expectation is taken under the smoother. To transform this into a transition probability, the base frequency of state $j$ must be taken into account; under EM, we estimate it with its expected frequency under the smoother (averaged over sequences).

Let us again make explicit the expected sufficient statistics for EM:

There are a few points to note. First, there are really only two kinds of expections: over a single latent variable, ${\check{X}}^{k}_{t}$, and over adjacent random variables, ${\check{X}}^{i}_{t}{\check{X}}^{j}_{t-1}$. We worked out how to compute these in Section 2.2. In particular, both expectations can be computed with the smoother—that is, after a complete forward and backward pass, since the expectations depend on the observations at all times. As we have lately noted, this is even true for the statistics at the very first time step. Finally, notice that we need not keep the statistics separately at each time step; we can sum across time. However, there is one subtlety: one of these sums omits the very last time step (since it is used to estimate the probability of the states out of which the system transitions).