Examples, neuroscience analogy Perceptrons, mlps: How they work

Outline

• Examples, neuroscience analogy

• Perceptrons, MLPs: How they work

• How the networks learn from examples

• Backpropagation algorithm

• Learning parameters

• Overfitting

Examples in Medical Pattern Recognition

• Diagnosis

• Protein Structure Prediction

• Diagnosis of Giant Cell Arteritis

• Diagnosis of Myocardial Infarction

• Interpretation of ECGs

• Interpretation of PET scans, Chest X-rays

• Prognosis

• Prognosis of Breast Cancer

• Outcomes After Spinal Cord Injury

Biological Analogy

Perceptrons

Perceptrons

AND

XOR

XOR

XOR

Linear Separation

Abdominal Pain

Multilayered Perceptrons

Activation Functions...

• Linear

• Threshold or step function

• Logistic, sigmoid, “squash”

• Hyperbolic tangent

Neural Network Model

“Combined logistic models”

Hidden Units and Backpropagation

Minimizing the Error

Gradient descent

Overfitting

Overfitting

Overfitting in Neural Nets

Parameter Estimation

• Logistic regression

• It models “just” one function

• Maximum likelihood
• Fast
• Optimizations
• Fisher
• Newton-Raphson

What do you want? Insight versus prediction

• Insight into the model

• Explain importance of each variable

• Assess model fit to existing data

Model Selection Finding influential variables

• Logistic

• Forward

• Backward

• Stepwise

• Arbitrary

• All combinations

• Relative risk

Small sets: Cross-validation

• Several training and test set pairs are created so that the union of all test sets corresponds exactly to the original set

• Results from the different models are pooled and overall performance is estimated

• “Leave-n-out”

• Jackknife

ECG Interpretation

Thyroid Diseases

Time Series

Time Series

Evaluation

Evaluation: Area Under ROCs

ROC Analysis: Variations

Dijagnoza

• Dijagnoza

• Struktura proteina

• Diajagnoza arteritisa gigantskih ćelija

• Infarkta miokarda

• Interpretacije EKG-a

• Interpretacija PET scena, rentgenskog snimka

• Prognoza

• Karcinoma dojke

• Ishoda posle povrede kičmene moždine

Primeri, analogno neuronaukama

• Primeri, analogno neuronaukama

• Perceptrons, MLPs: kako oni rade?

• Kako mreža uči iz primera

• Backpropagation algoritam

• Parametri učenja

• Overfitting