The arboretum procedure

The NEURAL Procedure

Overview

The NEURAL procedure trains a wide variety of feedforward neural networks using proven statistical

methods and numerical algorithms. Please read the chapters "Introduction to Predictive Modeling" and

"Neural Network Node: Reference" before reading this chapter.

Terminology and Architectural Description

Each INPUT, HIDDEN and TARGET statement defines a "layer". For the INPUT statement, a layer is a

convenient grouping of variables, serving as inputs to the network, having common values for LEVEL

and STD. Similarly, for the TARGET statement, a layer is a convenient grouping of variables, serving as

outputs of the network, having common values for LEVEL, STD, ACTIVATION, COMBINATION,

and other characteristics. Each layer is made up of "units". A unit is synonomous with the term "neuron"

in the literature. It is the smallest computational entity in the network.

The INPUT and TARGET statements require a list of variables. If the variables are interval type, there is

one unit corresponding to each variable. If the variables are nominal or ordinal, there is a unit for each

level of each variable. The HIDDEN statement requires a number which determines the number of units

in the associated layer. This layer is a grouping of units having common values for ACTIVATION,

COMBINATION, and other characteristics.

Each INPUT statement produces an input layer. Because multiple INPUT statements are allowed, a

network can have multiple input layers. However, connections from multiple input layers must be

parallel; there cannot be serial connections between input layers. Similarly, multiple TARGET

statements generate multiple output layers. The connections to multiple output layers must be in parallel;

there cannot be serial connections between output layers. Hidden layers can be connected serially or in

parallel.

Running the Neural Procedure

Before running the NEURAL procedure, you must run the DMDB procedure to create a DMDB-encoded

training set and a DMDB catalog entry.

A typical application of the NEURAL procedure uses the following statements:

A PROC NEURAL statement to specify the training set, DMDB catalog, and random number

seed. If you specify a 0 or negative random number seed, running the same NEURAL step

repeatedly may produce different results because of different random initial weights.

q   

One or more INPUT statements to specify input variables.

q   

One or more HIDDEN statements to define hidden layers.

q   

One or more TARGET statements to specify target variables.

q   

One or more CONNECT statements to connect layers in the network.

q   

---

A PRELIM statement to do preliminary training to avoid bad local optima.

q   

A TRAIN statement to train the network.

q   

One or more SCORE statements to create output data sets.

q   

Two kinds of statements are used with the NEURAL procedure. All of the statements in the list above

are action statements, which directly affect the network or directly produce output. There are also option

statements (NETOPTIONS, NLOPTIONS, and RANOPTIONS) that set options for future use. Options

specified in an action statement apply only to that statement and do not affect subsequent statements. For

example, the default technique for least-squares training is Levenberg-Marquardt (TECH=LEVMAR). If

you execute a TRAIN statement with the option TECH=CONGRA, conjugate gradient training will be

used for that particular training run. If you then execute another TRAIN statement without a TECH=

option, the technique will revert to the default value of TECH=LEVMAR. But if you submit an

NLOPTIONS statement with TECH=CONGRA, conjugate gradient training will be used for all

subsequent TRAIN statements until you explicitly specify a different technique.

Each layer in the network has an identifier specified by the ID= option in the INPUT, HIDDEN, or

TARGET statements. An identifier can be any SAS name, but to avoid confusion, you should not use the

name of a variable in the training set. Layer identifiers are used in various statements, such as

CONNECT, to specify previously defined layers.

Each unit in the network has a name. For units corresponding to interval variables in input or output

layers, the name of the unit is the same as the name of the variable. For units corresponding to dummy

variables for categorical (nominal or ordinal) inputs or targets, the name of each unit is constructed by

concatenating the name of the input or target variable with the value of the category, truncating as

necessary to make the length of the name eight characters or less. For hidden units, the names are

constructed by concatenating the layer ID with an integer.

Copyright 2000 by SAS Institute Inc., Cary, NC, USA. All rights reserved.

---

The NEURAL Procedure

Procedure Syntax

PROC NEURAL <option-list>;

ARCHITECTURE architecture-name

<HIDDEN=n>

<DIRECT>;

CODE FILE=file-name

<FORMAT=format>

<RESIDUAL|NORESIDUAL>

<ERROR|NOERROR>

<GROUP=name>;

CONNECT id-list/

<RANDIST=name>

<RANLOC=number>

<RANSCALE=number>;

CUT id-list| ALL;

DECISION DECDATA=<libref.>SAS-data-set

<DECVARS=decision-variable(s)> <option(s)>;

DELETE id-list| ALL;

FREEZE weight-list /

<VALUE=number>

<EST=SAS-data-set>;

FREQ variable(s);

HIDDEN integer /

ID=name

<ACT=keyword>

<BIAS|NOBIAS>

< COMBINE=keyword>;

---