How to predict time dependent variables with the help of neural networks ?

Objective: To predict the probability of rainfall based on monthly mean data of rainfall, runoff and evapotranspiration collected from a single gauge station for the last fifty years

Data Preprocessing : The fifty years mean monthly data of Rainfall,Runoff and Evapotranspiration is available.The data values of the variables is ranked in a descending way where best or minimum rank is assigned to the largest value and worst or maximum rank was given to smallest value of the variable.The probability of each of the data value of the variables can be calculated with the help of the following formula :

P = m/(n+1)

The data thus converted become dimensionless and thus normalization is not followed. If the data values have dimensions it is better to remove the scale from the same. This will make the job of the neural network program faster and easier.

Input Variables of the Present Study : Input to the model will be Probability of Runoff and Evapotranspiration.

Output Variable of the Present Study: Probability of Rainfall

Model Development :

In development of neural network model the following three steps are required to be adopted :

A) Selection of Network Topology

The topology i.e.,the number of hidden layers and nodes are important for development of an error-free model and will also enable the model to repeat its learning process per iteration. The hidden layers are responsible for learning the problem. It is also employed to store whatever it has learned from the previous trainings. The selection of topology depends on the problem type,number of data rows available,number of inputs and outputs etc. Generally trial and error is applied to find an optimal topology but nowadays metaheuristic search algorithms like Genetic Algorithm are widely utilized due to the quality and time taken to identify the optimal topology.

B)Selection of Activation Function

All the input signals are given a random weightage. These weightage is multiplied with the data values of the input and combined for feeding into the activation functions which actually excite/modifies the signal for prediction of the output. There are many kind of activation functions which are utilized according to the purpose and data availability of the model. Some of the examples may include : Step,Pulse,Sigmoidal,Tangential,Hypertan etc.

The selection of activation function is also conducted with the help of either trial and error or by metaheuristic search algorithms like Genetic Algorithm,Particle Swarm Optimization etc.

C) Selection of Training Algorithm

The training algorithms as discussed earlier is the way the weights are updated to learn the inherent problems from the given dataset. Actually in training process weights are randomly updated,combined and fed to the activation function to predict the output.If their are hidden layers then the process is repeated twice or thrice based on the number of hidden layers selected for the model.

The predicted output is compared with the actual output(in this problem data for output variable is available) to find the difference or error between them. Until and unless this difference reduces below the 'desired difference of the user' or maximum iterations allowed by the user is reached the weights are updated and the training procedure is continued. In this present investigations the data for the output is available and thus the training procedure is called as Supervised Learning.

Actually the entire dataset is divided into three parts : for training,testing and validation(generally 70,15 and 15%).

The test data is utilized when training is completed and the trained model is tested for accuracy as when the model is used for real prediction the output data will be absent.

It is found that if test error is less than training error then the model IS said to learn the inherent problem depicted by the datasets of input and output variable.

The weights are updated by various manner and procedures for updating is known as Training Algorithms.

The amount of data available for training,complexity of the problem,availability of computational infrastructures are influential in selecting the ideal training algorithm for a given problem.

Some of the examples : Conjugate Gradient Descent,Quick Propagation,Batch Back Propagation.

After the above three steps are followed the model is tested again with the entire available dataset so that the accuracy of the model can be verified with the help of performance metrics like Mean Square Error,Correlation Coefficient,Coefficient of Efficiency etc.

Parameters

In the development of neural network model a developer has to specify the following to prevent infinite iterations(A and B),generalization losses(D),consumption of time and computational infrastructures(C):

A) Maximum Number of Iterations Allowed

B)Desired MSE Wanted after training and testing

C)Improvement in Error in the las few iterations

D)Maximum generalization loss allowed

In case of the present problem as amount of dataset is adequate and the variables are dimensionless(probability values) thus the chance of generalization loss or undertraining will be minimum.

My ResearchGate Id : Mrinmoy_Majumder

Home Page: http://www.mrinmoymajumder.com 

Author of: Lecture Notes on MCDM

Indian Link ; Global Link: