-->
A FREE SEMINAR ON DEFAULT PREDICTORS IN AGRICULTURAL CREDIT SCORING: EVIDENCE FROM BANK OF AGRICULTURE DATA SOKOTO

A FREE SEMINAR ON DEFAULT PREDICTORS IN AGRICULTURAL CREDIT SCORING: EVIDENCE FROM BANK OF AGRICULTURE DATA SOKOTO

A FREE SEMINAR ON DEFAULT PREDICTORS IN AGRICULTURAL CREDIT SCORING: EVIDENCE FROM BANK OF AGRICULTURE DATA SOKOTO



Abstract

Decisions of Whether to grant Credit or not to the applicants are extremely important for any financial institution since it stimulates huge financial losses generated from defaulters. So many banks use Judgmental decisions, i.e. credit analysts go through every application separately and other banks develop credit Scoring Systems or combination of both. Credit Scoring System uses many types Predictive statistical models. But in recent times, researchers have begun looking for alternative algorithms with better predictive accuracy regarding classification.  Neural network can be suitable alternative. It is very clear from the classification outcomes of this research that neural network compares well with other statistical techniques. It gives slight better results than k-Nearest Neighbor Classifier, Logistic regression and Discriminant Analysis and performs slight poorer than Classification and Regression Tree (CART). However, it is noteworthy that a “Bad accepted” generates much high costs than a “Good rejected” and neural network acquires less amount of “Bad accepted” than the other predictive statistical models. So, neural network achieves less cost of misclassification for the dataset use in the research. Furthermore, in the final section of this research, an optimization algorithm (Genetic Algorithm) is proposed in order to obtain better classification accuracy through the configurations of the neural network architecture. On the contrary, it is important to note that the success of the predictive model largely depends on the predictor variables selection to be used as inputs of the model.


Keywords:  predictive analysis, credit scoring, pattern recognition, Discriminant analysis, Logistic regression  

 

1.1 Introduction

Predictive analysis encompasses a variety of statistical techniques from modeling, machine learning, data mining and game theory that analyzes current and historical facts to make predictions about future events.

 Predictive analytics is used in actuarial science, financial services, insurance, telecommunications, retail, travel, health care, pharmaceuticals and other fields. One of the most well-known applications is credit scoring, which is used throughout financial services. Scoring models process a customers’ credit history, loan application , customers’ data, etc, in order to rank-order individuals by their likelihood of making future credit payments on time.

A credit score is a numerical expression based on a statistical analysis of a person’s credit files, to represent the creditworthiness of that person. A credit score is primarily based on credit report information typically sourced from credit bureaus. The credit scoring models are developed to categorize applicants as either accepted (good) or rejected (bad) credits with respect to their characteristics such as age, income and marital condition. Creditors accept the application provided that it is expected to repay the financial obligation, and vice versa. Creditors can construct the classification rules based on the data of the previous applicants.

The way credit scoring works is simple theoretically. According to the author Jentzsch (2007), the basic working procedure can be explained in the following way: the dependent variable (Y) represents credit risk (the probability of repayment). The independent variables (predictor variables or Xi) are used to explain the dependent variable. The list and the value of the independent variables are extracted from the “Application Form” generally, or sometimes from the credit report.  Then the performance of a specific customer is decided based on the performance of the similar types of customers by using credit scoring system, credit scoring system awards points on every possible factor to calculate the probability of repayment. After these points are awarded, credit score comes up. Normally, the higher the achieved points, the lower the risk is.


1.2 Statement of the problem        

Credit decisions are very crucial for any type of financial institution because it can determine huge financial losses provoked from defaulters. A number of banks use judgmental decisions, means credit analysts go through every application separately and other banks use credit scoring system or combination of both. This therefore, primarily depends on the type of the product. In the case of small amount of credits like agricultural credits given to farmers (especially in credit cards), banks try to pursue automated system (credit scoring). The system provides decision based on the pattern recognition that is known from the previous customer’s database. Many types of algorithms are used in credit scoring. But recently, professionals started looking for more accurate decisions alternatives, for example neural network. But there are few guidelines on this topic and its application in credit decisions.

Justification of the study 

Classification plays a crucial role in business planning, especially in Credit Scoring. But, conventional methods suffer from several limitations, for example, many conventional methods assume linear relationship among the variables although there may be non-linear relationship in reality and non-linear models like the multiple regression models require model selection which is based on trial and error process (Leondes 2005). So, using neural network to forecast can be a very good alternative. The higher predictive power of neural network application can be attributed to their ability of reproducing human intelligence (Chaffey et al. 2009) and to their powerful pattern recognition capability (Zhang 2003).



Aim and Objectives

The aim is to compare the predictive ability of Artificial Neural network with the conventional statistical models for credit scoring. This aim can be achieved through the following objectives:

 To build  a  linear discriminant model capable of identifying an applicant credit status

To build a logistic regression model capable of identifying an applicant credit status

To build a k-nearest Neighbor Classifier (model) capable of identifying an applicant credit status 

To build a classification and regression tree (CART) model capable of identifying an applicant credit status

To build an artificial neural network model capable of identifying an applicant credit status

Provide the concepts and theories that should be reviewed and considered during the selection of the predictor variables for the development of any type of credit scoring system.


2.0Literature review

Shi et al., (2002) utilized Multiple Criteria Linear Programming (SAS Software) to classify credit card database in to two groups (good and bad) and three groups (good, normal and bad). Onget al., (2005) applied Genetic Programming to classify good and bad customers. Wang et al., (2005) used Fuzzy Support Vector Machine (Fuzzy SVM) on credit database; based on the assumption that one customer can’t be absolutely good or bad. And the study also included other algorithms like linear regression, Logistic Regression and BP- network. Lee et al., (2006) employed Classification and Regression Tree (CART) and Multivariate Adaptive Regression Splines (MARS) and identified better performance in comparison with the Discriminant Analysis, Logistic Regression, Neural Network and Support Vector Machine. Sexton et al., (2006) used GA-based algorithm, called Neural Network Simultaneous Optimization Algorithm on a credit dataset. The performance was satisfactory and the model was able to identify significant variables among the other existing variables.

Min et al. (2006) proposed methods for improving SVM performance in two aspects: feature subset selection and parameter optimization. Gestelet al., (2006) used LS-SVM for credit rating of banks, and compared the results with ordinary least squares, LR and multilayer perceptron (MLP). The results show the LS-SVM classifier’s accuracy is better than the three methods. Huang et al. (2007) designed hybrid SVM-based credit scoring models for assessing the credit scores of applicants. Zhang et al., (2008) developed a hybrid credit scoring model applying genetic programming and SVM. The accuracy of their hybrid model was better when compared with SVM, genetic programming, decision tree, LR, and back propagation neural networks.   

Thomas, Edelman et al. (2004) states that there should be a clear, rational, explanatory relationship between each variable and credit performance

Siddiqi (2005) pointed to use variable selection algorithms (Chi Square, R Square) before grouping the characteristics

Abrahams and Zhang (2009) drew attention to the importance of the “Sequence” of the independent variables selection

Fabozziet al., (2006) attempted to summarize the variables that have been found very helpful for the prediction of the future credit performance based on a large sample of customers

Anderloniet al. (2006) focused on statistical analysis to select the best set of variables in predicting the credit performance

However, this research work is different from the reviewed literatures in the following ways:

That the predictor variables used are different from those used by the reviewed literatures.

That none of the reviewed literature built credit scoring models for Nigerian Banks. Hence the need to build the credit scoring models based on the predictor variables that are available in the application form of the Nigerian Banking system. 

That the Statistical models used are linear discriminant analysis, Logistic regression, k- nearest neighbor analysis and classification and regression tree analysis.

That the artificial neural network is also used.



3.0 MATERIALS AND METHODS

3.1 DATA USED

A real world credit dataset is used in this research. The dataset is extracted from the application forms of Bank of Agriculture, Sokoto(BOA). Thedataset is referred to as “Credit Dataset”.  After preparing or cleaning the dataset, it is used in the subsequent sections for conducting the analysis with Discriminant Analysis, k- Nearest Neighbor Classifier, Classification and Regression tree (CART), Logistic Regression and Neural Network.


3.2 METHOD

Let us assume that the population of loans consists of two groups or classes G and B that denote loans that (after being granted) will turn good and bad in the future, respectively.

Let probability that a randomly chosen customer belongs to group G and B be denoted as and. Let x be a vector of independent variables used in the process of deciding whether an applicant belongs to group G or B. Let the probability that an applicant with measurement vector x belongs to group G is p(G|x), and that of B is p(B|x). Let the probability p(x|G) indicate that a good applicant has measurement vector x. Similarly, for a bad applicant the probability is p(x|B).  The task is to estimate probabilities from the set of given data about applicants which turn out to be good or bad and to find a rule for how to partition the space X of all measurement vectors in to the two groups andbased on these probabilities so that inwould be the measurement vectors of applicants who turn out to be good and vice versa.However, it is usually not possible to find perfect classification as it may happen that the same vector is given by two applicants where one is good and the other is bad. Therefore it is necessary to find a rule that will minimize the costs of the agency providing credit connected with the misclassification of applicants. Let us denote as the costs connected with misclassifying a good applicant as bad and as the costs connected with misclassifying a bad applicant as good (>). If applicants with x are assigned to class G the expected costs are p(B|x) and the expected loss for the whole sample is


wherep(x) is a probability that the measurement vector is equal to x. This is minimized when into group G such applicants are assigned who have their group of measurement vectors


Without loss of generality we can normalize misclassification costs to  += 1. In that case the rule for classification is to assign an applicant with x to class G if p(G|x)and otherwise to class B.


3.2.1 Linear Discriminant analysis

The aim of Linear Discriminant Analysis (LDA) is to classify a heterogeneous population in to homogenous subsets and further the decision process on these subsets. We can assume that for each applicant there are a specific number of explanatory variables available. The idea is to look for such a linear combination of explanatory variables which separates most subsets from each other. In a simple case of two subsets, the goal is to find the linear combination of explanatory variables, which leaves the maximum distance between means of the two subsets.

In a general case we consider the distributions p(x|G) and p(x|B) which are multivariate normal distributions with the common variance. Then equation (2) reduces to 


Where .


3.2.2 Logit Analysis

The distribution of the credit information data is usually non-normal, and this fact may theoretically pose a problem when conducting an LDA. One way to overcome the problems with non-normality of data is to use an extension of the LDA model that allows for some parametric distribution. In this case a suitable extension is a generalized linear model known as logit model. Given a vector of application characteristics x, the probability of default p is related to vector x by the relationship



3.2.3 k-nearest Neighbor Classifier

The k-nearest Neighbor Classifier serves as an example of the non-parametric statistical approach. This techniques assesses the similarities between the pattern identified in the training set and input pattern. One chooses a metric on the space of applicants and takes k-nearest neighbor (k-NN) of the input pattern that is nearest in some metric sense. A new applicant will be classified in the class to which majority of the neighbors belong (in the case when the costs of misclassification are equal) or according to rule explained by equation (1). This means that this method estimates the p(G|x) (or p(B|x)) probability by the proportion of G (or B) class points among the k-nearest neighbors to the point x to be classified. 

A commonly used metric is the standard Euclidean norm given by


wherexand y are measurement vectors 

However, when the variables are in different units or categorized, it is necessary to use some appropriate standardization of variables as well as to select some data-dependent version of the Euclidean metric such as:


WhereA is an × n matrix with n number of variables. As matrix A can depend on x we can define two types of metrics according to how A is selected: local metrics are those where A depends on x; global metrics are those where A is independent of x. 

The choice of the number of nearest neighbors chosen (k) determines the bias/variance trade-off in the estimator. The k has to be much smaller than the smallest class. A simulation study by Enas and Choi (1986) suggested that k ≈ n2/8 or n3/8 is reasonable


3.2.4 Classification and Regression Trees (CART)

Classification and Regression Trees (CART) is a nonparametric method that is due to Breiman, Friedman, Olshen and Stone (1984). It is a flexible and potent technique; however, it is used in banking practice chiefly only as a supporting tool to accompany the parametric estimation methods described earlier. It serves, for example, in the process to select repressors’ or characteristics with the highest explanatory power. The CART method employs binary trees and classifies a dataset into a finite number of classes. It was originally developed as an instrument for dealing with binary responses and as such it is suitable for use in credit scoring where the default and non-default responses are contained in data. Similarly as with the methodologies reviewed earlier, we make the assumption of having a training set of measurement vectors 

.

Along with information whether an individual j defaulted, or not (hence,is coded 1 or  

The CART tree consists of several layers of nodes: the first layer consists of a root node; the last layer consists of leaf nodes. Because it is a binary tree, each node (except the leaves) is connected to two nodes in the next layer. The root node contains the entire training set; the other nodes contain subsets of this set. At each node, this subset is divided into 2 disjoint groups based on one specific characteristic  from the measurement vector. The classification process is a recursive procedure that starts at the root node and at each further node (with exception of the leaves) one single characteristic and a splitting rule (or constant c) is selected. First, the best split is found for each characteristic. Then, among these characteristics the one with the best split is chosen.


3.2.5 Neural Networks

A neural network (NNW) is a mathematical representation inspired by the human brain and its ability to adapt on the basis of the inflow of new information. Mathematically, NNW is a non-linear optimization tool. Many various types of NNW have been specified in the literature. The NNW design called multilayer perceptron (MLP) is especially suitable for classification and is widely used in practice. The network consists of one input layer, one or more hidden layers and one output layer, each consisting of several neurons. Each neuron processes its inputs and generates one output value that is transmitted to the neurons in the subsequent layer. Each neuron in the input layer (indexed i = 1,…,n) delivers the value of one predictor (or the characteristics) from vector x. When considering default/non-default discrimination, one output neuron is satisfactory. 

In each layer, the signal propagation is accomplished as follows. First, a weighted sum of inputs is calculated at each neuron: the output value of each neuron in the proceeding network layer times the respective weight of the connection with that neuron. A transfer function g(x) is then applied to this weighted sum to determine the neuron's output value. So, each neuron in the hidden layer (indexed j = 1,…, q) produces the so-called activation


The neurons in the output layer (indexed k = 1,…, m) behave in a manner similar to the neurons of the hidden layer to produce the output of the network: 


Where andare weights

The most popular algorithm for training multilayer perceptrons is the back-propagation algorithm. As the name suggests, the error computed from the output layer is back-propagated through the network, and the weights are modified according to their contribution to the error function. Essentially, back-propagation performs a local gradient search, and hence its implementation; although not computationally demanding, it does not guarantee reaching a global minimum. For each individual, weights are modified in such a way that the error computed from the output layer is minimized.

The BP learning involves three stages: the feed-forward of the input training pattern, the calculation of the associated error, and the adjustment of the weights. After the network reaches a satisfactory level of performance, it learns the relationships between independent variables (applicant’s attributes) and dependent variable (credit class). The trained BP network can then be adopted as a scoring model to classify the credit as either good or bad by inserting the values of applicant’s attributes.  




Figure 1: The BP-based credit scoring architecture



3.2.6 Wilks’ Lambda Test for significance of canonical correlation

Hypothesis canonical correlation:

: There is no relationship between the two sets of variables

    : There is linear relationship between two set of variables

 Test statistic:

  H is the variance due to linear relationship

                                W+H is the total variance

Decision rule: Reject      if P<0.05 otherwise accept       at the 5% level of significance



3.2.7 Chi-square Test

Hypothesis for Chi-square Test:

   : The two variables are independent 

 : The two variables are not independent

  Test statistic:

  Decision Rule:Reject if P<0.05 otherwise accept at the 5% level of significance.


3.2.8 Omnibus Chi-square Test

The omnibus Chi-square test is a log-likelihood ratio test for investigating the model coefficients in logistic regression. The test procedures are as follows:

Hypothesis for Omnibus Chi-square Test:

 : The model coefficients are not statistically significant

   : The model coefficients are statistically significant

Test statistic: 


Decision Rule:Reject  if p<0.05 otherwise acceptat the 5% level of significance i.e. significance of the logistic model.


3.2.9 Box M Test for the Equality of Covariance Matrices

Hypothesis for Box’s M Test:

: The two covariance matrices are equal

 : The two covariance matrices are not equal

Test Statistic:


Decision Rule: Reject   if P<0.05 otherwise acceptat the 5% level of significance.


3.2.10 Wald Test

The Wald test is used to test the statistical significance of each coefficient  in the logistic model. A Wald test calculates a statistic which is:


This value is squared which yields a chi-square distribution and is used as a Wald test statistic.

Decision rule: Reject  (the null hypothesis that the coefficient is equal to zero) when p-value of that coefficient is less than       level of significance.


4.0. RESULTS AND DISCUSSIONS

4.1. Discriminant Analysis

4.1.1 Model Measurement

 A discriminant model is identified as “Useful” if there is at least 25% more improvement achievable over the by chance Accuracy rate. Here, “By Chance Accuracy Rate” is 51% and 25% increase of this value equals to 63% and the cross validated accuracy rate is 95%.Hence, cross validated accuracy rate can be used to discriminate better. Moreover, Wilks' lambda is a measure of the usefulness of the model. 


Table 1: SPSS Output: Model Test :Wilks' Lambda


Test of Function(s)

Wilks' Lambda

Chi-square

Df

Sig.


1

.276

377.437

10

.000


The Wilk’s Lambda Probability of 0.000<0.05, means that predictors significantly discriminate the groups. This provides the proportion of total variability not explained, i.e.  27.6% unexplained.


Table 2 Canonical correlation.


Function

Eigenvalue

% of Variance

Cumulative %

Canonical Correlation


1

2.626a

100.0

100.0

.851




A canonical correlation of 0.851 suggests that the model explains 72.42% of the variation in the grouping variable, i.e. whether an applicant is a creditworthy or not.


4.1.2. Importance of Independent Variables:

Table 3: Discriminant Models




APPLICANT ATTRIBUTES

FUNCTIONS



STRUCTURE MATRIX

STANDARDIZED DISCRIMINANT COEFFICIENTS

CANONICAL DISCRIMINANT COEFFICIENTS


AGE

0.046

0.261

0.023


SEX

0.219

0.208

0.516


MARITAL STATUS

0.053

0.007

0.022


JOB 

0.819

0.957

1.273


PURPOSE

-0.038

-0.045

-0.038


CREDIT AMOUNT

-0.007

-0.280

0.000


ANNUAL EARNINGS

0.010

-0.138

0.000


REPAYMENT PLAN

0.137

-0.074

-0.228


APPLICATION TYPE

-0.355

-0.438

-1.202


APPLICATION PERIOD

0.109

0.100

0.247


CONSTANT



-2.438


The structure matrix shows the correlations of each variable with each discriminant function. Generally, just like factor loadings, 0.30 is seen as the cut-off between important and less important variables. The interpretation of the standardized discriminant function coefficients is like that in multiple regressions. The canonical discriminant function coefficients (unstandardized coefficients) are used to create the discriminant function (equation). It operates just like regression equation. There are specific characteristics determined by the discriminant model for the two groups (creditworthy applicants and non-creditworthy applicants). For example, on an average, a good customer is a junior civil servant and applying as an individual.


Table 4: SPSS Output: Group Statistics: Only the Good Group:


Status of the Credit Applicant

Mean

Std. Deviation

Valid N (list wise)





Unweight

Weighted


Good

Application type

1.0366

.18832

164

164.000



Job of the applicant

3.0488

.95805

164

164.000










Characteristics of the Creditworthy Applicants


On the other hand, the bad group holds some certain characteristics in contradictory with the good group. For example, on an average, a non-creditworthy applicant is self-reliant and is a group application.

Table 5: SPSS Output: Group Statistics: Only the Bad Group:



Status of the Credit Applicant

Mean

Std. Deviation

Valid N (listwise)





Unweighted

Weighted


Bad

Application type

1.4559

.49989

136

136.000



Job of the applicant

1.0515

.37158

136

136.000




Characteristics of the Non-creditworthy Applicant


4.2. Logistic Regression:

4.2.1 Measurement of Model Performance: The significance of the model can be checked in two ways. First one is the significance test. 


Table 6: SPSS Output: Model Test: Omnibus Tests of Model Coefficients




Chi-square

Df

Sig.


Step 1

Step

310.784

10

.000



Block

310.784

10

.000



Model

310.784

10

.000


Checking Usefulness of the Derived Model

In this analysis, the probability of the model chi-square (310.784) is 0.000, less than the level of significance of .05. The null hypothesis that there is no difference between the model with only a constant and the model with independent variables is rejected. The second way of confirming the usefulness of the model is to compare the classification accuracy rate. The independent variables can be characterized as useful predictors distinguishing between the two groups of the dependent variable if the classification accuracy rate is substantially higher than the accuracy attainable by chance alone.



Table 7: SPSS Output At Step 0: Classification Table:




Observed

Predicted




Applicant status

Percentage Correct




Good

Bad



Step 0

Applicant status

Good

164

0

100.0




Bad

136

0

.0



Overall Percentage



54.7


a. Constant is included in the model.


b. The cut value is .500


 Checking Usefulness of the Derived Model


Table 8: SPSS Output At Step 1: Classification Table:




Observed

Predicted




Applicant status

Percentage Correct




Good

Bad



Step 1

Applicant status

Good

156

8

95.1




Bad

4

132

97.1



Overall Percentage



96.0


a. The cut value is .500


Checking Usefulness of the Derived Model


4.2.2 Importance of Independent Variables: The significance test is the statistical evidence of the presence of a relationship between the dependent variable and each of the independent variables. The significance test is the Wald Statistic. Here, the null hypothesis is that the b coefficient for the particular independent variable is equal to zero









Table 9: SPSS Output: Significant Variables by logistic model 





B

S.E.

Wald

df

Sig.

Exp(B)


Step 1a

Age

-.007

.027

.074

1

.786

.993



Sex

-.883

.700

1.591

1

.207

.414



Marital Status

-.279

1.114

.063

1

.802

.756



Job

-3.638

.571

40.571

1

.000*

.026



Purpose

.031

.233

.018

1

.895

1.031



Credit Amount

.000

.000

3.006

1

.083

1.000



Annual earnings

.000

.000

.813

1

.367

1.000



Repayment plan

.882

.790

1.245

1

.264

2.416



Application type

2.469

.907

7.407

1

.006*

11.805



Application period

-.738

.727

1.029

1

.310

.478



Constant

3.804

3.045

1.561

1

.211

44.894


.


Reject the null hypothesis if the probability of the Wald statistic is less than or equal to the level of significance of .05

The individual coefficient represents the change in the odds of the modeled category associated with a one-unit change in the independent variable. Here, in this analysis, the modeled group is the “Bad Group” because of having the highest numerical code of 2. Individual coefficients are expressed in log units and are not directly interpretable

.

4.3. k-Nearest Neighbor Analysis:

4.3.1. Model measurement: The accuracy of k-NN model depends largely on the choice of the k (the number of the nearest neighbor to examine)

Table 10: SPSS Output: Error Summary (k=8, training sample=211, testing sample=89).


Partition

Percentage incorrectly classified cases


Training 

4.74%


Holdout

1.13%



4.3.2Importance of Independent Variable


Figure 2:  SPSS Output: Significant Variables

4.4 Classification and Regression Tree (CART) Analysis


Figure 3: SPSS Output: Classification Tree

Significant Variable Identified By the Classification Tree Model


4.4. Artificial Neural Network:

4.4.1 Measurement of Model Performance:

The following model summary table displays information about the results of the neural network training. Here, cross entropy error is displayed because the output layer uses the softmax activation function. This is the error function that the network tries to minimize during training.Moreover, the percentage of incorrect prediction is equivalent to 3.8% in the training samples. So, percentage of correct prediction is nearer to 96.2% that is quite high.If any dependent variable has scale measurement level, then the average overall relative error (relative to the mean model) is displayed. On the other hand, if the defined dependent variables are categorical, then the average percentage of incorrect predictions is displayed

Table 11: SPSS Output: Model Summary:



Training

Cross Entropy Error

31.851



Percent Incorrect Predictions

3.8%



Stopping Rule Used

1 consecutive step(s) with no decrease in errora



Training Time

0:00:00.375


Testing

Cross Entropy Error

6.795



Percent Incorrect Predictions

1.1%


Dependent Variable: Status of the Credit Applicant


a. Error computations are based on the testing sample.


Checking Usefulness of the derived Model

4.4.2 Importance of Independent Variables:

The following table performs an analysis, which computes the importance and the normalized importance of each predictor in determining the neural network.The analysis is based on the training and testing samples. The importance of an independent variable is a measure of how much the network’s model-predicted value changes for different values of the independent variable. Moreover, the normalized importance is simply the importance values divided by the largest importance values and expressed as percentages

Table 12: SPSS Output: Independent Variable Importance:




Importance

Normalized Importance


Attribute 2

.084

26.8%


Attribute3

.011

3.6%


Attribute 4

.314

100.0%


Attribute 5

.061

19.5%


Attribute 8

.035

11.1%


Attribute 9

.117

37.1%


Attribute 10

.054

17.2%


Attribute 6

.135

42.9%


Attribute 1

.087

27.6%


Attribute 7

.102

32.5%


Important Variables Identified By the Neural Network Model


4.5 Comparison of the Model’s Predictive Ability

4.5.1 Discriminant Analysis: 

Table 13: SPSS Output: Classification Results:





Attribute 11

Predicted Group Membership

Total





1.00

2.00



Original

Count

1.00

156

8

164




2.00

4

132

136



%

1.00

95.1

4.9

100.0




2.00

2.9

97.1

100.0


Cross-validateda

Count

1.00

155

9

164




2.00

6

130

136



%

1.00

94.5

5.5

100.0




2.00

4.4

95.6

100.0


Predictive Ability of the Discriminant Model

  The discriminant model is able to classify 156 good applicants as “Good Group” out of 164 good applicants. Thus, it holds 95% classification accuracy for the good group. On the other hand, the same discriminant model is able to classify 132 bad applicants as “Bad Group” out of 136 bad applicants. Thus, it holds 97.1% classification accuracy for the bad group. Thus, the model is able to generate 96.0%classification accuracy in combined groups.


4.5.2 Logistic Regression:

Table 14: SPSS Output: Classification Results:




Observed

Predicted




Attribute 11

Percentage Correct




1.00

2.00



Step 1

Attribute 11

1.00

156

8

95.1




2.00

4

132

97.1



Overall Percentage



96.0


Predictive Ability of the Logistic Model


The logistic model is able to classify 156 good applicants as “Good Group” out of 164 good applicants. Thus, it holds 95.1% classification accuracy for the good group. On the other hand, the same logistic model is able to classify 132 bad applicants as “Bad Group” out of 136 bad applicants. Thus, it holds 97.1% classification accuracy for the bad group. Thus, the model is able to generate 96.0% classification accuracy for the both groups.

4.5.3 k-Nearest Neighbor Analysis:

Table 15: SPSS Output: Classification Results:


Predictive Ability of the k-Nearest Neighbor Classifier

The k-Nearest Neighbor Classifier model is able to classify 110 good applicants as “Good Group” out of 116 good applicants. Thus, it holds 94.83% classification accuracy for the good group. On the other hand, the same k-Nearest Neighbor Classifier model is able to classify 91 bad applicants as “Bad Group” out of 95 bad applicants. Thus, it holds 95.79% classification accuracy for the bad group. Thus, the model is able to generate 95.27%classification accuracy in combined groups.


4.5.4 Classification and Regression Tree Analysis:


Table 16: SPSS Output: Classification Results:



Observed

Predicted



1.00

2.00

Percent Correct


1.00

157

7

95.7%


2.00

3

133

97.8%


Overall Percentage

53.3%

46.7%

96.7%


Growing Method: CHAID

Dependent Variable: Status of the Credit Applicant


Predictive Ability of the Classification Tree

The Classification Tree analysis model is able to classify 157 good applicants as “Good Group” out of 164 good applicants. Thus, it holds 95.7% classification accuracy for the good group. On the other hand, the same Classification Tree analysis model is able to classify 133 bad applicants as “Bad Group” out of 136 bad applicants. Thus, it holds 97.8% classification accuracy for the bad group. Thus, the model is able to generate 96.7% classification accuracy in combined group.


4.5.5 Artificial Neural Network: 

Table 17: SPSS Output: Classification Results:



Sample

Observed

Predicted




1.00

2.00

Percent Correct


Training

1.00

119

4

96.7%



2.00

4

81

95.3%



Overall Percent

59.1%

40.9%

96.2%


Testing

1.00

40

1

97.6%



2.00

0

51

100.0%



Overall Percent

43.5%

56.5%

98.9%


Dependent Variable: Status of the Credit Applicant


Predictive Ability of the Artificial Neural Network


The neural network model is able to classify 119 good applicants as “Good Group” out of 123 good applicants. Thus, it holds 96.7% classification accuracy for the good group. On the other hand, the same neural network model is able to classify 81 bad applicants as “Bad Group” out of 85 bad applicants. Thus, it holds 95.3% classification accuracy for the bad group. Thus, the model is able to generate 96.2% classification accuracy for the both groups. Here, the training sample is taken into account, because statistical models don’t use testing sample.


4.6. Optimization of Neural Network Performance & Future Research Scope

Genetic Algorithm can be used as an optimization method for improving NN performance. 

There are two main issues about the performance of the neural network. First of all, it is important to determine its structure and secondly, it is also vital to specify the weights of the neural network that help to minimize the total errors (Deb, Poli et al. 2004). These are optimization issues. Evolutionary algorithm (genetic algorithm) is a kind of optimization technique that uses selection and recombination as the main instruments to deal with optimization problems (Kamruzzaman, Begg et al. 2009). For example, genetic algorithm is the main available method that can be used to find well suited network architecture for a given task or problem (Patel, Honavar et al. 2001). According to the genetic algorithm theory, all combinations of the parameters of the possible solutions of a given problem (in this analysis, all the possible combinations of the parameters of the neural network architecture) must be coded into a gene and by a process of the selection of the fittest (in this case, the best neural network architecture that provides better classification) only the best solutions are selected for the reproduction, and after each subsequent generation, new solutions (only selected if they provide better classification accuracy) are generated by means of the reproduction between solutions and their related mutations (Mira, Cabestanyet al. 2009). 

Genetic algorithm is especially suitable for complex optimization problems. Many researchers tried to optimize the weights of the neural network using genetic algorithm alone (or, with backpropagation algorithm) and others attempted to find out a good network architecture or structure (the number of units and their interconnections) (Deb, Poliet al. 2004). Instruments or tools used by the genetic algorithm are selection, crossover and mutation (Larose, 2006). 

Selection indicates the technique of selecting chromosomes that will be used for reproduction. The fitness function appraises each of the chromosome (candidate solutions), and the better (fitter) the chromosome, the more probability that it will be selected for the reproduction purpose. The central task of crossover is to perform recombination, means that the creation of the two new offspring by randomly choosing a locus and exchanging subsequences to the left and right of that locus between two chromosomes chosen during the selection process. Mutation randomly modifies the bits or digits at a particular locus in a chromosome, most of the time with very low likelihood. Mutation brings new information to the genetic pool and protects against finding too quickly to a local optimum. The concepts are from Larose (2006). 

There are some unique benefits of using Genetic Algorithm as searching and optimization technique. For example, it is efficient, adaptive, and robust search technique, that is capable of generating optimal (or, near optimal) solutions (Pal and Wang 1996). For these characteristics and advantages, genetic algorithms applications in pattern recognition problems (which require robust, fast and close appropriate solutions) are perfect and almost natural. 

Although this study is aimed at providing a comparative ability of the neural network in the classification of the credit card customers, its scope of coverage is limited by the selection and comparison of the statistical models with neural network model. Future research efforts can certainly go beyond this limitation in order to have a more comprehensive review of genetic algorithm related literatures and to use the same dataset for the optimization of the network.

4.7. Findings and Conclusion


Appropriate predictor variables selection is one of the conditions for successful credit scoring models development. This study reviews several considerations regarding the selection of the predictor variables. Moreover, using the Multilayer Perceptron Algorithm of Neural Network, network architecture is constructed for predicting the probability that a given customer will default on a loan. The model results are comparable to those obtained using commonly used techniques like Logistic Regression, CART Discriminant Analysis or k-NN Classifier, as described in the following table:


Table: 18: Predictive Models Comparison

Credit Dataset


Models 

Good 

Accepted 

Good 

Rejected 

Bad 

Accepted 

Bad 

Rejected 

Success 

Rate 


Discriminant Analysis 

156 

132 

96.0%


Logistic Regression 

158 

8

132

4

96.0% 


k-NN Classifier 

110 

91 

95.79% 


CART

157

7

133

3

96.7%


Neural Network

119

51

81

4

96.3%



There are two noteworthy and interesting points about this table. First of all, it shows the predictive ability of each model. Here, the column 2 and 5 (“Good Accepted” and “Bad Rejected”) are the applicants that are classified correctly. Moreover, the column 3 and 4 (“Good Rejected” and “Bad Accepted”) are the applicants that are classified incorrectly. Furthermore, it shows that neural network gives slightly better results than discriminant analysis, Logistic regression and k-NN Classifier but, it gives slightly worst results than Classification and Regression Trees.  It should be noted that it is not possible to draw a general conclusion that neural network holds better predictive ability than k-NN Classifier, Logistic regression and discriminant analysis, worst predictive ability than  Classification and Regression Tree (CART) because this study covers only one dataset. On the other hand, statistical models can be used to further explore the nature of the relationship between the dependent and each independent variable, and statistical models are preferred than neural network if there is tiny difference in predictive ability of the neural network and the statistical models. 

Secondly, the above provided table gives an idea about the cost of misclassification. In order to introduce the costs of misclassification it is assumed that a “Bad Accepted” generates much higher costs than a “Good Rejected”, because there is a chance to lose the whole amount of credit while accepting a “Bad” and only losing the interest payments while rejecting a “Good” (Schader, Gaul et al. 2009). Here, in this analysis, it is apparent that neural network (equals to 81) acquired less amount of “Bad Accepted” than discriminant analysis (equals to 132) logistic regression (equals to 132) Classification and Regression Tree (equals 133) and k-NN Classifier (equals 91). So, neural network achieves less cost of misclassification. 

In the above table, statistical models contain the same number of cases for the model construction, but the NN and k-NN Classifier hold less number of cases because a portion of the cases is kept for the “Testing Sample” to prevent the network from overtraining/ over-fitting. Furthermore, statistical models don’t use anything like testing samples. So, the classification results are slightly incomparable because of the uneven number of cases used in all models. After re-running the analysis, without any testing sample in the NN, and k-NN Classifier with the equal number of sample sizes in all the models, it is found that "Bad Accepted" is decreased further to 3 from 81 for the NN. Because, neural network over trained itself. Moreover, overall percentage correct is increased to 98% from 96.3%. So, for the credit dataset, NN is giving good results always. But, for k-NN Classifier even though the whole sample is used for training the model, the “Bad Accepted” remained 4 but the overall percentage correct is increased to 96.34% from 96.3% which is very insignificant. 

In the final section, Genetic Algorithm is proposed to obtain better classification accuracy through the configurations of the neural network architecture and weights. Moreover, it is suggested as future research scope that the same dataset can be used for neural network performance improvement and the results can be compared with before optimization results.


 

References

Abrahams, C.R. and Zhang, M. (2009) Credit Risk Assessment: The New Lending System for Borrowers, Lenders, and Investors, Wiley.

Agresti, A. (2002) Categorical Data Analysis, Wiley- Interscience

Altman, E. (1968): Financial Ratios, Discriminant Analysis and the prediction of Corporate Bankruptcy. Journal of Finance, 23,589-609

Anderloni, L., Braga, M.D. and Carluccio, E.M. (2006) New Frontiers in Banking Services: Emerging Needs and Tailored Products for Untapped Markets, Springer.

Beares, P.R., Beck, R.E. and Siegel, S.M. (2001) Consumer Lending, American Bankers Association.

Blattberg, R.C., Kim, B.D. and Neslin, S.A. (2008) Database Marketing: Analyzing and Managing Customers, Springer.

Bocij, P., Chaffey, D., Greasley, A. and Hickies, S. (2009) Business Information Systems: Technology, Development and Management for the E-business, FT Press.

Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J. (1984): Classification and Regression Trees. Pacific Grove, CA: Wadsworth, 1984.

Campbell, T.S., Dietrich, J.K. (1983): The Determinants of Default on Insured Conventional Residential Mortgage Loans. Journal of Finance, 38,1569-1581

Charitou, A., Neophytou, E., Charalambous, C.(2004): Predicting Corporate Failure: Empirical Evidence for the UK. European Accounting Review, 13, 465-497.

Christine B., (2009) Logistic regression and its Application in Credit Scoring.

Chatterjee, S. Barcun, S. (1970): A Nonparametric Approach to Credit Screening. Journal of American Statistical Association, 65,50-154.

Colquitt, J. (2007) Credit Risk Management, McGraw-Hill.

Culp, C.L. (2001) The Risk Management Process: Business Strategy and Tactics, Wiley.

Davis, R.H., Edelman, D.B., and Gammerman, A.J. (1992). Machine learning algorithms for Credit-Card applications. IMA Journal of Mathematics Applied in Business and Industry, 4, 43-51.

Deb, K., Poli, R., Banzhaf, W., Beyer, H.G., Burke, E., Darwen, P., Dasgupta, D., Beyer, H.G., Foster, Harman, M., Holland, O., Lanzi, P.L., Spector, L., Tettamanzi, A. and Thierens, D.(2004) Genetic and Evolutionary Computation GECCO 2004, Springer. 

Desai, V.S. Crook, J.N., and Overstreet, G.A. (1996). A comparison of neural networks and linear scoring models in the credit union environment. European Journal of Operation Research, 95(1), 24-37

Denison, D.G.T., Mallick, B.K., Smith, A.F.M. (1998): A Bayesian CART Algorithm. Biometrics, 85,363-377.

Devaney, S. (1994): The Usefulness of Financial Ratios as Predictors of Household Insolvency: Two Perceptives. Financial  Counseling and planning, 5, 15-24.

Eisenbeis, R.A. (1977): Pitfalls in the Application of Discriminant Analysis in Business, Finance and Economics. Journal of Finance, 32, 875-900.

Esposito, F., Malerba, D., Semeraro, G.(1997): A comparative Analysis of Methods for Pruning Decision Trees. IEEE Transactions on pattern Analysis and Machine Intelligence, 19, 476-491.

Fabozzi, F.J., Davis, H.A. and Choudhry, M. (2006) Introduction to Structured Finance, Wiley.

Feldman, D., Gross, S. (2005): Mortgage Default: Classification Tree Analysis. Journal of Real Estate Finance and Economics, 30,2005 369-396.

Fisher, R.A. (1936): The use of Multiple Measurements in Taxonomic Problems. Annals of Eugenic, 7, 179-188.

Frydman, H., Altman, E., Kao, D.L. (1985): Introducing Recursive Partitioning for Financial Classification.

Gardner, M.J., Mills, D.L. (1989): Evaluating the likelihood of Default on Delinquency Loans: Financial Management, 18, 55-63.

Greenacre, M. and Blasius, J. (2006) Multiple Correspondence Analysis and Related Methods, Chapman and Hall/CRC.

Hand, D.J., Vinciotti, V. (2003): Choosing k for Two-class Nearest Neighbor Classifiers with Unbalanced Classes. Pattern Recognition Letters, 24, 1555-1562.

Henley, W.E., and Hand, D.J. (1996). A k-nearest neighbor Classifier for assessing Consumer Credit risk. Statistician, 44, 77-95. 

Holmes, C.C., Adams, N.M. (2002): A Probabilistic Nearest Neighbor Method for Statistical Pattern Recognition. Journal of Royal Statistical Society. Series B,64,295-306.

Jorion, P. (2000) Value at Risk: The New Benchmark for Managing Financial Risk, McGraw-Hill.

Kamruzzaman, J., Begg, R.K. and Sarker, R.A. (2006) Artificial  Neural Networks in Finance and Manufacturing, Idea Group Publishing.

Larose, D.T. (2006) Data Mining Methods and Models, Wiley-IEEE Press.

Lawrence, E., Arshadi, N. (1995): A MultinominalLogit Analysis of Problem Loan Resolution Choices in Banking. Journal of Money, Credit and Banking, 27, 202-216.

Leondes, C.T. (2005) Intelligent Knowledge- Based Systems: Business and Technology in the New Millennium, Kluwer Academic Publishers.

Liang, L., Desheng, W.(2005) An application of pattern recognition on scoring Chinese corporations financial conditions based on backpropagation neural network

Mays,E.(1998) Credit Risk Modeling: Design and Application, CRC.

Meyers, L.S. Gamst, G.C. and Guarino, A.J. (2005) Applied Multivariate Research: Design and Interpretation, Sage Publications.

Mira, J., Cabestany, J. and Prieto, A. (2009) New Trends in Neural Computation, Springer. 

Mu, C.C., Shih, H.H. (2003) Credit Scoring and rejected instances reassigning through evolutionary computation techniques. Journal of Expert Systems with Applications 24, 433-441.

Pal, S.K. and Wang, P.P. (1996) Genetic Algorithms for Pattern Recognition, CRC

Patel, M., Honavar, V. and Balakrishnan, K.(2001) Advances in the Evolutionary Synthesis of Intelligent Agents, The MIT Press.

Plewa, F.J. and Friedlob, G.T. (1995) Understanding Cash Flow, Wiley.

Reichert, A.K., Cho, C.C. and Wagner, G.M. (1983). An examination of the Conceptual issues involved in developing Credit-Scoring Models. Journal of Business and Economics Statistics ,1,101-114.

Rosenberger, L.E., Nash, J and Graham A. (2009) The Deciding Factor: The Power of Analytics to make Every Decision a Winner, Jossey-Bass

Samsul M.I., Lin Z. Fei Li (2009) Application of Artificial Intelligence (Artificial Neural Network) to Assess Credit Risk. A Predictive Model For Credit Card Scoring.

Servigny, A.D. and Renault, O. (2004) The Standard and Poor’s Guide to Measuring and Managing Credit Risk, McGraw-Hill.

Siddiqi, N. (2005) Credit Risk Scorecards: Developing and Implementing Intelligent Credit Scoring, Wiley.

Thomas, Edelman, D. and Crook, J. (2002) Credit Scoring and its Applications, Society for Industrial Mathematics.

Thomas, L.C., Edelman, D.B. and Crook, J.N. (2004) Readings in Credit Scoring: Foundations, Development and aims, Oxford University Press.

West, D. (2000). Neural Network Credit Scoring Models. Computers and Operation Research, 27,1131-1152.

Zhang, G.P. (2000). Neural Networks for Classification: a survey. IEEE Transactions on Systems, Man, and Cybernetics-Part: Applications and Reviews, 30(4), 451-462.

Zhang, G.P. (2003) Neural Networks in Business Forecasting, Information Science Publishing












0 Response to "A FREE SEMINAR ON DEFAULT PREDICTORS IN AGRICULTURAL CREDIT SCORING: EVIDENCE FROM BANK OF AGRICULTURE DATA SOKOTO"

Post a Comment