Deep Learning in Medical Image Analysis

Deep Boltzmann machine. A DBM (55) is also constructed by stacking multiple RBMs in a hierarchical manner. However, in contrast to DBNs, all the layers in DBMs form an undi- rected generative model following the stacking of RBMs (Figure 2c). Thus, for hidden layer l , except in the case of l = 1 and l = L, the layer’s probability distribution is conditioned by its two neighboring layers, l + 1 and l − 1 [i.e., P (h^{(l )}|h^{(l +1)}, h^{(l −1)})]. The incorporation of information from both the upper and lower layers improves a DBM’s representational power so that it is more robust to noisy observations.

Annu. Rev. Biomed. Eng. 2017.19:221-248. Downloaded from www.annualreviews.org Access provided by 82.215.98.77 on 06/08/22. For personal use only.
Let us consider a three-layer DBM, namely the L = 2 DBM shown in Figure 2c. Given the values of the units in the neighboring layer(s), the probability of either the binary visible or binary hidden units being set to one is computed as follows:





i

k

j

ij

jk

k
P h⁽¹⁾ = 1|v, h⁽²⁾ = σ W ⁽¹⁾v_i + W ⁽²⁾h⁽²⁾ , (5)
P h⁽²⁾ = 1|h⁽¹⁾ = σ ^⎝⎛ W ⁽²⁾h^(1)⎠⎞ , (6)

_j

j

k

jk

j

ij

j
P v_i = 1|h⁽¹⁾ = σ ^⎝⎛ W ⁽¹⁾h^(1)⎠⎞ , (7)
where σ (·) denotes a logistic sigmoid function. In order to learn the parameters ① = {W⁽¹⁾, W⁽²⁾},³ we maximize the log likelihood of the observations. The derivative of the log likelihood of the observations with respect to the model parameters takes the following simple form:

∂

_∂ l
_{W( )} ln P (v; ①) = E_data h^{(l −1)}(h^{(l )})^T − E_model h^{(l −1)}(h^{(l )})^T , (8)
where E_data[·] denotes the data-dependent statistics obtained by sampling the model conditioned on the visible units v(= h⁽⁰⁾) and E_model[·] denotes the data-independent statistics obtained by sampling from the model. When the model approximates the data distribution well, data-dependent and data-independent statistics reach equilibrium.

4.