4.1 Undirected graphical models

Some readers may still find potential functions to be less intuitive than condition distributions as descriptions of the relationships among random variables. In fact, a direct translation is possible. To see this, consider the undirected graphical model in Fig. 4.1a, for which the joint distribution is

for some normalizer $Z$. The independence statements asserted by this graph—e.g., ${\bm{\hat{X}}}_{1}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern 2.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptstyle\perp$\hss}\mkern 2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern 2.0mu{\scriptscriptstyle\perp}}}{\bm{\hat{X}}}_{4}|{\bm{\hat{X}}}_{2},{\bm{\hat{X}}}_{3}$—follow from the usual graph-separation criterion.

Now consider the graphical model in Fig. 4.1b, which we assert to be normalized:

In this graph, ${\bm{\hat{X}}}_{1}$ and ${\bm{\hat{X}}}_{4}$ are no longer independent conditioned on ${\bm{\hat{X}}}_{2},{\bm{\hat{X}}}_{3}$, since there is a connecting path through ${\bm{\hat{Y}}}$. But conditioning on ${\bm{\hat{Y}}}$ clearly restores all of the independence statements of Fig. 4.1a. Therefore if, for a particular value $\bm{\hat{y}}$ of ${\bm{\hat{Y}}}$, we define

then

The last line follows from summing both sides over all configurations of ${\bm{\hat{X}}}$. In fine, the missing normalizer is $\check{p}(\bm{\hat{y}})$.

More generally, the product of potentials for any undirected graphical model with nodes ${\bm{\hat{X}}}_{1},\ldots,{\bm{\hat{X}}}_{N}$ can be interpreted as a slice through some normalized distribution, $\check{p}({\color[rgb]{.75,0,.25}\definecolor[named]{pgfstrokecolor}{rgb}{.75,0,.25}\bm{x}_{1}},\ldots,{\color[rgb]{.75,0,.25}\definecolor[named]{pgfstrokecolor}{rgb}{.75,0,.25}\bm{x}_{N}},\bm{\hat{y}})$, where the auxiliary random variable ${\bm{\hat{Y}}}$ did not occur in the original graph. Furthermore, the individual potentials $\psi$ can all be interpreted as unnormalized conditional distributions, or equivalently as slices through the corresponding marginal distributions. Under this interpretation, computing the partition function is equivalent to computing the marginal probability of $\bm{\hat{y}}{}$—another instance of inference with Bayes’s theorem.