Deep Learning in Medical Image Analysis

Reducing Overfitting

A critical challenge in training deep models arises from the limited number of training samples compared with the number of learnable parameters. Thus, reducing overfitting has long presented a challenge. Recent studies have devised algorithmic techniques to better train deep models. Some of the techniques are as follows.

1. 
   Initialization/momentum (77, 78) involves the use of well-designed random initialization and a particular schedule of slowly increasing the momentum parameter as iteration passes.
   
2. 
   Rectified linear unit (ReLU) (12, 79, 80) applies to nonlinear activation functions.
   
3. 
   Denoising (11) involves stacking layers of denoising auto-encoders, which are trained locally to reconstruct the original “clean” inputs from the corrupted versions of them.
   
4. 
   Dropout (13) and DropConnect (81) randomly deactivate a fraction (e.g., 50%) of the units
   

or connections in a network on each training iteration.

5. 
   Batch normalization (14) performs normalization for each minibatch and back-propagates the gradients through the normalization parameters.
   

See the references cited for further details.

3. APPLICATIONS IN MEDICAL IMAGING

Compared with other machine learning techniques in the literature, deep learning has witnessed significant advances. These successes have prompted researchers in the field of computational medical imaging to investigate the potential of deep learning in medical images acquired with, for example, CT, MRI, PET, and X-ray. In this section, we discuss the practical applications of deep learning in image registration and localization, detection of anatomical and cellular structures, tissue segmentation, and computer-aided disease prognosis and diagnosis.

1. Deep Feature Representation Learning in Medical Images

Many existing medical image processing methods rely on morphological feature representations to identify local anatomical characteristics. However, such feature representations were designed mostly by human experts, and the image features are often problem specific and not guaranteed to work for other image types. For instance, image segmentation and registration methods designed for 1.5-T T1-weighted brain MR images are not applicable to 7.0-T T1-weighted MR images (28, 52), not to mention to other modalities or different organs. Furthermore, 7.0-T MR images can reveal the brain’s anatomy with a resolution equivalent to that obtained from thin slices in vitro (82). Thus, researchers can clearly observe fine brain structures at the micrometer scale, which pre- viously was possible only with in vitro imaging. However, the lack of efficient computational tools substantially hinders the translation of new imaging techniques into the medical imaging arena.

Annu. Rev. Biomed. Eng. 2017.19:221-248. Downloaded from www.annualreviews.org Access provided by 82.215.98.77 on 06/08/22. For personal use only.



Examples of the learned weights

Training image patches