*************************************************************************

PROGRAM: NETLE

PURPOSE: Feedforward Neural Networks and Lyapunov Exponents Estimation

COPYRIGHT: Chung-Ming Kuan, Tung Liu, and Ramazan Gencay

VERSION: 4.1. July 2007

Revision History:

7/20/2007: Distributed as Version 4.1.

2/12/2006: V4.01 Compute the standard errors for LE estimate.

1/2/1997: V3.01. Compute LE for inputs up to 20.

2/1995: V2.1

*************************************************************************CONTENTS:

Section 1. INTRODUCTION

Section 2. HOW TO RUN NETLE PROGRAM

Section 3. DATA FILE

Section 4. CONTROL FILE

Section 5. PROGRAM OUTPUT

Section 6. CORRESPONDENCE1. INTRODUCTION

NETLE 4.1 is a Windows program to estimate feedforward neural network models and compute Lyapunov exponents (LE). The selected neural network models are estimated by the method of nonlinear least

squares (NLS) (Kuan and Liu (1995)). Lyapunov exponents are calculated from the derivative matrices of estimated network models (Gencay and Dechert (1992)). A positive Lyapunov exponent estimate indicates

that the underlying series is chaotic. The standard error of the LE estimate is derived by Shintani and Linton (2004).The running of the current program is under DOS prompt. The future version will run under Windows. Since the NLS estimation requires a lot of CPU time, a high speed PC is recommended. The maximum number of data values is 7000. The maximum inputs and hiddens are 20.

This disk is distributed for academic purposes only; proper citation of this program is gratefully appreciated. Please contact Tung Liu at tliu@bsu.edu for any questions on this program.

REFERENCES:

Kuan, Chung-Ming and Tung Liu (1995).

"Forecasting exchange rates using feedforward and recurrent networks,"

Journal of Applied Econometrics, 10:347-364.Gencay, Ramazan and W. D. Dechert (1992).

"An algorithm for the n Lyapunov exponents of an n-dimensional unknown

dynamical system," Physica D, 59:142-157.Shintani, M. and Linton, O. (2004).

Nonparametric Neural Network Estimation of Lyapunov Exponents and a

Direct Test for Chaos. Journal of Econometrics. 120(1):133.2. HOW TO RUN NETLE PROGRAM

The current version can only run under DOS prompt in Windows

Operation System. This zip file contains the main program,

a sample data, and a control file:README.TXT This file.

NETLE.EXE The main program.

HENON.IN A sample control file for NETLE.EXE.

HENON.DAT An example data set with 600 observations and 4 inputs.To run NETLE, unzip the file NETLE41.ZIP to a directory. Before you

run the program, you will need to have a data file and a control file.

The structure of your data file must follow the format described in

Section 4 below. The detail of the control file is explained in

Section 5. Put the main program, data file, and the control file

in one directory. And run the program from the DOS prompt by typing:C:> NETLE control_file_name

You may test NETLE program using the sample control file and data file

in this zip file by typingC:> NETLE HENON.IN

3. DATA FILE

The first column of the data file must be a series of ID numbers (e.g.

1,2,3,....,n); the second column should contain the target (dependent)

variable; the other columns should be input (explanatory) variables.4. CONTROL FILE

NETLE is not an interactive program. The execution of this program

is controlled by a control file which contains information of the

data file name, network structure, and other options for computation.

A sample of this control file HENON.IN is included in the disk. The

simplest way to create your own control file is to revise the content

of HENON.IN and rename it with a new file name. After you create your

own control file, you can run NETLE from the DOS prompt by typingC:> NETLE control_file_name

For example, to run the program with HENON.IN, type

C:> NETLE HENON.IN

4.1 Contents of a Control File:

This file contains 16 instruction lines. Each line is followed by

a single information line. The user must supply all the information

in these information lines. If no information is needed, a blank line

must exist between two instruction lines. The program will not function

properly if wrong information or more than one information line is supplied.

If two numbers are required in an information line, these numbers must be

separated by space(s). DO NOT erase the instruction lines in your control

file as the program will read them as NULL lines.The following are the instruction lines and explanation.

(1) [* NETLE Version 4.1 (neural NETwork for LE) by Kuan, Liu, & Gencay. *]

The TITLE you would like to have in your output file.

(2) [* Data File Name: *]

The name of the data file to be read by the program.

(3) [* Output File Name: *]

The name of the output file.

(4) [* Number of Observations: *]

Total number of observations in the data file specified in (2).

(5) [* Number of Inputs: *]

The number of inputs (explanatory variables).

(6) [* Minimum and Maximum Number of Hidden Units: *]

Two integer values. The first value, q, is a number specifying the

minimum number of hidden units; the second value, Q, is a number

specifying the maximum number of hidden units. The program will

estimate total (Q-q+1) networks.(7) [* Calculating the Lyapunov Exponent: 0/No, 1/Yes *]

Enter 0 or 1. The Lyapunov exponents will be computed if 1 is entered.

(8) [* Method of Scaling Inputs: *]

Enter one of the following 4 choices:

0: no scaling is made;

1: each input is standardized by subtracting its mean and divided

by its standard deviation;

2: each input is standardized between 0 and 1 using a logistic

transformation: x*=1/(1+exp(-x));

3: each input is standardized between -1 and 1 using a logistic

transformation: x*=2/(1+exp(-x)) - 1;

Notice that this value will be set to 0 by the program if the

calculation of Lyapunov exponents is requested.(9) [* Seeds (Positive Number); NTRY: *]

The first number specifies the seed of the (standard normal)

random number generator; the second gives the number of

sets of initial parameters to be generated. The program will

select the set of initial parameters which results in the lowest

MSE as the starting values for NLS estimation.(10) [* Number of Iterations; Tolerance in NLS: *]

The first number specifies the maximum number of iterations in

NLS estimation; the second specifies the tolerance level

for NLS convergence.(11) [* Creating the Parameters File: 0/No, 1/Yes *]

Enter 0 or 1. A file for parameter estimates will be created under

the name specified in (12) if 1 is entered.(12) [* Parameters File Name: (The Option Above Must Be "1") *]

The file name for parameter estimates. If 0 is specified in

(11), you may leave this line blank, or the program will ignore the

content of this line.(13) [* Creating the Fitted Values File: 0/No, 1/Yes *]

Enter 0 or 1. A file for fitted values will be created under the

name specified in (14) if 1 is entered.(14) [* Fitted Values File Name: *]

The file name for fitted values. If 0 is specified in (13),

you may leave this line blank, or the program will ignore the content

of this line.(15) [* Creating the Derivatives: 0/No, 1/Yes *]

Enter 0 or 1. A file for derivatives will be created under the

name specified in (16) if 1 is entered.(16) [* Derivatives File Name: *]

The file name for derivatives which is used to compute Lyapunov

exponents. If 0 is specified in (15), you may leave this line blank,

or the program will ignore the content of this line.

4.2. A Sample Control File

Below is a sample control file HENON.IN. The user may create a new

control file by suitably modifying this sample file.[* NETLE Version 4.1 (neural NETwork for LE) by Kuan, Liu, & Gencay. *]

Henon Map

[* Data File Name: *]

henon.dat

[* Output File Name: *]

henon.out

[* Number of Observations: *]

600

[* Number of Inputs: *]

4

[* Minimum and Maximum Number of Hidden Units: *]

2 6

[* Calculating the Lyapunov Exponent: 0/No, 1/Yes *]

1

[* Method of Scaling Inputs: *]

0

[* Seeds (Positive Number); NTRY: *]

56666459 500

[* Number of Iterations; Tolerance in NLS: *]

900 1.0d-9

[* Creating the Parameters File: 0/No, 1/Yes *]

1

[* Parameters File Name: (The Option Above Must Be "1") *]

henon.par

[* Creating the Fitted Values File: 0/No, 1/Yes *]

1

[* Fitted Values File Name: *]

henon.fit

[* Creating the Derivatives: 0/No, 1/Yes *]

1

[* Derivatives File Name: *]

henon.der

5. PROGRAM OUTPUT

The results of the program will be displayed on the screen and

written to an output file as specified in the control file.Part of the result from the control file HENON.IN is as follows.

The Henon1.out contains the main results. For the model with 2 hiddens,Hiddens 2: MSE = 0.125331E-07 SIC = -18.0563

Lyapunov Exponents:

lambda( 1) = 0.4249725

HAC Std Error (Newey/West) 0.0136785

HAC Std Error (Parzen) 0.0136320

HAC Std Error (Quad. Spect.) 0.0135662

lambda( 2) = -1.6241288

HAC Std Error (Newey/West) 0.0202770

HAC Std Error (Parzen) 0.0227045

HAC Std Error (Quad. Spect.) 0.0249118

lambda( 3) = -4.4671052

HAC Std Error (Newey/West) 0.3413844

HAC Std Error (Parzen) 0.1545655

HAC Std Error (Quad. Spect.) 0.1425157

lambda( 4) = -4.5935032

HAC Std Error (Newey/West) 0.3800361

HAC Std Error (Parzen) 0.1592727

HAC Std Error (Quad. Spect.) 0.1465124The values of lambda's are the estimated Lyapunov exponents. The three

different standard errors of the estimate can be used for confidence

interval or the hypothesis testing. If the calculation of Lyapunov

exponents is set to "0" in HENON.IN, the values of lambda will not

be given.

5.1 The file for Parameter EstimatesIf the parameter file is requested, both the initial values from the

random number generator and the final estimates from NLS will be written

to the parameter file. There are two lines above each set of parameters:Initial Est.: ntobs nv nq nw

xxx x x xxNLS Est.: ntobs nv nq nw

xxx x x xxHere, "ntobs" is the total number of observations in the data file,

"nv" is the number of inputs, "nq" is the number of hidden units, and

"nw" is the total number of parameters of the estimated network. The

parameters of each network in the parameter file are arranged according

to the following order:Output bias (constant term),

Parameters connecting hidden and output units,

The first hidden bias,

The parameter connecting the first input and first hidden units,

The parameter connecting the second input and first hidden units,

...

The parameter connecting the v-th input and first hidden units,

...

The q-th hidden bias,

The parameter connecting the first input and q-th hidden units,

The parameter connecting the second input and q-th hidden units,

...

The parameter connecting the v-th input and q-th hidden units,5.2. The file for Fitted Values

The file for fitted values has 3 columns: the first contains

original target values, the second contains network fitted values,

and the third contains network errors (target - fitted values).6. CORRESPONDENCES

Questions of this program should be addressed to:

Tung Liu, Ph. D.

Department of Economics

Ball State University

Muncie, Indiana 47306

U.S.A.

(765) 285-5360

E-Mail: tliu@bsu.edu

http://tliu.iweb.bsu.eduOther coauthors of this program can be reached at:

Chung-Ming Kuan, Ph.D.

Institute of Economics

Academia Sinica

128 Academia Road, Sec. 2

Taipei 115, TAIWAN

886 2 2782–2791 ext. 646

886 2 2651–0647 (Direct)

886 2 2782–2019 (Fax)

E-Mail: ckuan@econ.sinica.edu.tw

URL: www.sinica.edu.tw/~ckuanRamazan Gencay, Ph.D.

Department of Economics

Simon Fraser University

8888 University Drive

Burnaby, British Columbia, V5A 1S6, Canada

Phone: (604) 291-3934

Email: gencay@sfu.ca

http://www.sfu.ca/~rgencay

Author: Tung Liu | Last Modified: July 4, 2007 | Comments to Webmaster