Deep Learning in Medical Image Analysis

A _j = f ^⎝


i =1
A_i ∗ k_ij + b _j ^⎠ , (9)

Annu. Rev. Biomed. Eng. 2017.19:221-248. Downloaded from www.annualreviews.org Access provided by 82.215.98.77 on 06/08/22. For personal use only.



j
where M ^{(l −1)} denotes the number of feature maps in layer l −1, the asterisk denotes a convolutional operator, b ^(l) is a bias parameter, and f (·) is a nonlinear activation function. Due to the mechanisms of weight sharing and local receptive field, when the input feature map is slightly shifted, the activation of the units in the feature maps is shifted by the same amount.
A pooling layer follows a convolutional layer to downsample the feature maps of the preceding convolutional layer. Specifically, each feature map in a pooling layer is linked to a feature map in the convolutional layer; each unit in a feature map of the pooling layer is computed on the basis of a subset of units within a local receptive field from the corresponding convolutional feature map. Similar to the convolutional layer, the receptive field finds a representative value (e.g., maximum or average) among the units in its field. Usually, a change in the size of the receptive field during

Annu. Rev. Biomed. Eng. 2017.19:221-248. Downloaded from www.annualreviews.org Access provided by 82.215.98.77 on 06/08/22. For personal use only.
convolution is set equal to the size of the receptive field for subsampling, helping the CNN to be translation invariant.
Theoretically, the gradient descent method combined with a back-propagation algorithm can also be applied to the learning parameters of a CNN. However, due to the special mechanisms of weight sharing, the local receptive field, and pooling, slight changes need to be made; that is, one needs to sum the gradients for a given weight over all the connections using the kernel weights in order to determine which patch in the layer’s feature map corresponds to a unit in the next layer’s feature map, and to upsample the feature maps of the pooling layer to recover the reduced size of the maps.

6.