Deep Learning in Medical Image Analysis

Deep Models

ji

kj

j

k
Under a mild assumption on the activation function, a two-layer neural network with a finite number of hidden units can approximate any continuous function (67); therefore, it is regarded as a universal approximator. However, it is also possible to approximate complex functions to the same accuracy by using a deep architecture (i.e., one with more than two layers), with a far fewer number of units (8). Thus, it is possible to reduce the number of trainable parameters, enabling training with a relatively small data set (68).


² W⁽¹⁾ = [W ⁽¹⁾ ] ∈ R^{M ×D}; W⁽²⁾ = [W ⁽²⁾ ] ∈ R^{K ×M} ; b⁽¹⁾ = [b ⁽¹⁾ ] ∈ R^M ; b⁽²⁾ = [b ⁽²⁾ ] ∈ R^K .

a Stacked auto-encoder b Deep belief network c Deep Boltzmann machine

_h(L)
_h(L – 1)





_h(1)
v
_h(L)
_h(L – 1)





_h(1)
v
_h(L)
_h(L – 1)





_h(1)
v

Annu. Rev. Biomed. Eng. 2017.19:221-248. Downloaded from www.annualreviews.org Access provided by 82.215.98.77 on 06/08/22. For personal use only.


Figure 2
Three representative deep models with vectorized inputs for unsupervised feature learning. The red links, whether directed or undirected, denote the full connections of units in two consecutive layers but no connections among units in the same layer. Note the differences among models in directed/undirected connections and the directions of the connections that depict conditional relationships.

3.