Offline Signature Biometric Verification with Length Normalization using Convolution Neural Network

: Offline handwritten signature is a type of behavioral biometric-based on an image. Its problem is the accuracy of the verification because once an individual signs, he/she seldom signs the same signature. This is referred to as intra-user variability. This research aims to improve the recognition accuracy of the offline signature. The proposed method is presented by using both signature length normalization and histogram orientation gradient (HOG) for the reason of accuracy improving. In terms of verification, a deep-learning technique using a convolution neural network (CNN) is exploited for building the reference model for a future prediction. Experiments are conducted by utilizing 4,000 genuine as well as 2,000 skilled forged signature samples collected from 200 individuals. This database is publicly distributed under the name of SIGMA for Malaysian individuals. The experimental results are reported as both error forms, namely False Accept Rate (FAR) and False Reject Rate (FRR), which achieved up to 4.15% and 1.65% respectively. The overall successful accuracy is up to 97.1%. A comparison is also made that the proposed methodology outperforms the state-of-the-art works that are using the same SIGMA database. Gradient (HOG), Image Processing, Length-normalization, Signature verification.


Introduction:
In general, the biometric system has been defined as an individual that is based on a characteristic vector extracted from the physiological or behavioral characteristics of the individual 1 . Normally, two modes of biometric operations are as follows: identification and verification 2 . First, Identification Mode is a comparison of the biometric target data with all of the other data available in the system, and can be represented in the following question: "Who are you?.". In other words, it performs a (1:n) match as it is a one-to-many comparison. This method of biometric authentication is widely used for forensic and surveillance purposes. Although the second mode of the biometric process is called the Verification Mode. This model is based on the following question: "Are you the one you claim to be?". Here, the target biometric data is compared to the original reference stored in the system to authenticate its identity. It can also be specified to fit one-to-one (1:1) data 3 .

Handwritten
Signature Verification Systems are used to automatically recognize whether the signature given by the author belongs to the individual in question. The genuine signatures are those produced by the claimed individual (original writer) and the forgeries are those made by the impostor (forger). Some of the problems related to offline hand-written signatures are a high number of users (classes) 4 . Also, the high-dimensional vector length, the limited number of training samples per writer with high intra-class variability.
The handwritten signature usually comprises the first with the last names of an individual, and it is often the case that the signature does not contain a full name, but rather a part of it. This type of signature is referred to as a paraph. The motivation is that the signature can be defined as a behavioral type of biometrics that acts as a non-intrusive as well as non-invasive authentication for the users 5 . Also, its implementation involves both company transactions and government agencies 6 . Besides, the biometric signature is deemed as one of the most accepted biometrics because most people have their signatures, which can be used as codes for them 7 . However, the main problem is the high intra-user variability properties, which decreases the total accuracy of the signature verification. That is because the individuals may not produce the same signature as one of the signed previously 7 . The method of signature authentication is either static or dynamic data format 8 . The static is also known as offline signature verification, which conducts signature verification using scanned images signed on a paper-based document as shown in Fig.1. While the dynamic one is referred to as an online signature verification process, signature samples are collected one point by one digitally, usually using computerized pens and visual tablets 8 .
It should be noted that the high correct matching (accuracy) of the signature authentication is difficult to be attained due to the issue named "high intra-user variability", in which the False Reject Rate (FRR) increases. Therefore, the objective of this research is to propose a verification system that increases the accuracy, which outperforms the existing works, by exploiting the deep-learning based on the convolutional neural network (CNN). The contribution of this paper is to improve the accuracy achieved until now with the SIGMA database by using both histogram orientation gradient (HOG) passed to deep-learning as a hybrid technique. Finally, to validate the proposed work, a comparison is made between the proposed method and the state-of-the-art works using the database called SIGMA databases.
The paper is structured as follows: Section 2 is devoted to related work for the authentication of signatures; Section 3 explains the system architecture by using a signature length normalization with a (HOG) and a convolutional neural network. The experiment and implementation are then defined in Section 4. The consequence and discussion of the proposed method are explained in Section 5. Then, the conclusion is provided in Section 6.

Related Work
In this section, the latest work of handwritten signature verification is reviewed regarding the method description of the work, a database used in that work with its characteristics, and the resulted accuracy of the method with any other comment that is required to be mentioned.
For instance, in the work in reference 8 , online signature verification is presented by using length normalization as Up-Sampling and Down-Sampling techniques. This is accomplished by the PCA (Principal Component Analysis) for feature extraction to build the feature vector that represents the signature sample. Then, passing this vector to ANN (Artificial Neural Network) for classification is performed by exploiting the Database of SIGMA (the same database will be used in this research paper). The result was obtained with a False Accept Rate (FAR) of up to 5.5 % and a False Reject Rate (FRR) of up to 8.75 %. Another online signature verification has been presented for online handwritten signature recognition by using the same database SIGMA as in reference 9 . This approach to online signature verification through the use of multilayer perceptron (MLP) on a subset of principal component analysis (PCA) features. The result is as follows: a false acceptance rate (FAR) of 7.4% and a false rejection rate (FRR) of 6.4%.
Another work is presented in reference 10 for an offline handwritten signature. In this method a feature vector of an offline handwritten signature by using Histogram Orientation Gradient (HOG) for the feature extraction and also used Support Vector Machine (SVM) for the classification. An experiment has been conducted to estimate the accuracy by using SIGMA database. The result achieved up to 96.8% as detailed: False Accept Rate (FAR) is 3% and False Reject Rate (FRR) is 3.35%.
Another work used scale-invariant feature transform (SIFT) and speeded-up robust features (SURF) features to build codebooks of the feature histograms as in reference 11 . This work used and support vector machines (SVMs) for the verification. Here, the Database used 1600 samples and a recognition rate is up to 95% by involving the ten-fold cross-validations.
Another idea of the signature verification in case there are few samples of trailing, samples generator is used based on the genuine signer to duplicates and generate the same image space as in reference 12 . The classifier is the SVM, as claimed the equal error rate (EER) has been decreased from 5.71% to 1.08%. The evaluation has been performed using many database signatures such as GPDS, MCYT-75, and CEDAR. Also, convolutional neural network (CNN) methods as deep-learning-based on four models VGG16, VGG19, ResNet50, and DenseNet121 have been used for offline handwritten signature verification as in reference 13 . Here, the best model achieved an accuracy of up to 98.06% in DenseNet121 by using the MCYT database.

Methodology
Operations are divided into two main phases: training (enrollment) and testing (verification) as shown in Figure 2. In the first place, the training operation begins by gathering a database of signature samples as images from individuals. These signatures are certainly are signed with different lengths. For this reason, the signature length normalization is then exploited based on the algorithm provided in reference 8 that is Approved to increase the verification rate if signature lengths are normalized with the same length (the idea of the normalization will be discussed in subsection A). Several processing techniques are then used to improve the signature images in terms of contrast and noise reduction. In this process, contrast enhancement is used based on the intensity improvement, Furthermore, a nonlinear filter (median filtering) 14  feature extraction called histogram orientation gradient (HOG) is used to increase the recognition rate for signature verification. Thus, the feature vector after the HOG phase is a one-dimensional vector (1-D) that will be fed to the convolutional neural network (CNN) for the classification procedure. CNN then generates a reference model that can be used for future verification (prediction). The second step, which is the authentication process, is carried out if there is a need to verify the signature whether it is genuine or forged. The operation is done by taking the signature from the entity and then converting it to an image. After that, the same steps that have been taken in the training operation will be implemented during the verification phase, except for the training as shown in Fig. 2. CNN is used to compare the registered and the queried signature attributes that need to be verified in the testing phase. Finally, a decision is made based on a threshold to determine if an offline signature must be approved or rejected.

A-Signature length normalization
To cope with the intra-user variability of handwritten signatures, which decreases the verification rate, the method of normalization is extended to the online handwritten signature as described in the reference 8 . After normalization in this study, the signature type format is converted from online (time series signature format ] [t X ) to offline (image signature format). Signature length normalization shall be accomplished concerning time, as a result, all signature samples in the SIGMA database shall consist of a defined period of signature length.
Accordingly, the normalization for the signature length is agreed to be 256 pixels (dotted) in this research. The reason for selecting this factor for all individuals of the database is because most of the signature samples in this database are close to the average length of the 261 pixels signal sampling.
The normalization method is accomplished by mapping the unknown length to the proper length (256) of the signature provided that the original signature sample will not be confused or distorted. The normalization method is focused on two main operations: Down-Sampling with Up-Sampling 15 . In terms of the Down-Sampling is explained as if a signature signal ] [t X , and it is required to run a 2-pixels down-sampling data, then the output is selecting only one from every two pixels as . In terms of the up-sampling operation, which is explained that the output ] [t Y is generated by interlacing between two old values a new value, which is derived by performing an interpolation process as in (1) Down-Sampling is applied to those signatures when the length of the signature exceeds the length of the desired signature. While Up-Sampling is applied to those signatures when the duration of the signature is less than the length of the desired signature.

B-HOG features for signature
Histogram Orientation Gradient (HOG) is used for the representation of the shape features introduced by Dalal and Triggs at the 2005 CVPR conference 16 . In this research, HOG was adopted as a feature extraction technique for authenticating signature images. HOG is implemented by selecting masks for edge detections to compute derivatives and gradients, then, splitting an image into cells and grouping cells into a block, block overlapping, and normalization parameters 16 . In the current work, HOG was implemented by setting the Cell-Size to [110x110] pixels. Then the size of the block is [2x2]. Block-Overlap between adjacent lines is selected 1 for better accuracy. Also, the number of bins selected in the orientation histograms is 9 bins as default. HOG depends on the gradient (∂) that can be computed by Eq.(2), while the orientation (θ) can be claculated using Eq.(3).
Accordingly, the overall feature vector length for is 432 the aforementioned configuration, which is used to represent each signature image sample. Figure 3 depicts two offline handwritten signatures from SIGMA database (having ID=119, ID=136) with a three kinds of cell size as [100x100], [110x110] and [120x120]. It is worth mentioning that the output of the HOG is a 1dimensional vector that contains the histogram of the ordination gradient for the image.

C-Signature verification by CNN
One of the most popular deep-learning algorithms is called convolutional neural network (CNN) explained in [17][18][19] that is used for classification and prediction. CNN contains different processing layers. the basic three layers are convolution, rectified linear unit (ReLU), and pooling layers. the procedure starts by inserting the input images and apply convolution operations with convolutional filters. several filters will be included that are used to highlight the edges of the input image. As far as the Rectified Linear Unit (ReLU) is concerned, it facilitates quicker and more efficient training by mapping negative values to zero and retaining positive values. This is often referred to as activation since only the active features are forwarded to the next layer 17 . Next, the Pooling operation, which is performing nonlinear down-sampling by selecting the value of a block size [2x2] throughout the image. These operations are repeated over tens or hundreds of layers, with each layer learning to identify different features.
After the extensive running of the experiment to choose a suitable number of layers, the proposed CNN design in this research paper has several layers as illustrated in Fig. 4. This design consists of three primary layers called rounds. Each round layer contains four processing operations, which are the convolution layer, normalization, activation function (Relu), and finally the maxpooling layer. The configuration of all rounds is the same except for the convolution operation, where the number of filters is different in each round. The number of the filter masks are as follows: 21, 42, and 84 for the Conv1, Conv2, and Conv3 respectively. The convolutions stride is [1x1] and the padding type is the same. it is worth mentioning that the input to the CNN is a 1-D feature vector having a length of either 288, 432, or 540.

Experiment Setup
Several experiments have been conducted on offline signatures to test the accuracy of the verification of the proposed process. The experiments are carried out using the signature samples in the SIGMA database related to Malaysian people 20 . Now, each individual will have a unique training matrix used to train a model specific for that individual. The training matrix is designed to have 20 samples, divided as following each individual has 10 genuine samples as well as 10 forged samples, which are combined in one matrix to form a training matrix. In which the 10 forged samples are further divided into two-part as five for skilled forged samples and the other five for randomly forged samples (taken from a different individual). Finally, the training matrix includes 20 signature samples. Each signature sample is represented by 432 features. these 432 features came from the HOG feature extraction.
The training is run by CNN for each individual's signatures. In other words, there will be 200 times training and testing operations in each experiment. Then, the evaluation of the signature verification system is stated by extracting the following two errors: FAR (False Accept Rate) and FRR (False Reject Rate) 21 for each person separately. Regarding the test matrix, which is designed similarly to the training matrix, must be built with the same number of features and classes. Once the training type of the proposed system is supervised learning, target scores for each class must be presented 22 . Accordingly, the +1 target has been assigned to the genuine signatures, while the -1 target has been assigned to the forged signature samples. Here, the target matrix has a size of [20x1], because the first 10 targets are for those genuine signature samples, and the remaining targets are for those the 10 forged signature samples as illustrated in Table 1: Now to calculate the FRR error, a result of the prediction of the CNN output will be evaluated. The size of the predicted matrix is the same as the sample numbers in the testing matrix, which is [20x432] (20 signature samples each one with a length of 432 features). To calculate the FRR, the first 10 output scores have to be as +1 target. If any of these 10 outputs are not +1, that means a counter named FR (False Rejection) will be added by one (increment) such as ) . Then, the rate of this error is calculated as in (4): Conversely, the predicted output vector indexed from 11~20 must be -1 target. If any one of these 10 predicted output is not -1, then a counter named (FA) will be increased by one such as . This means that the error FAR is appeared and computed as in (5): After extracting the FAR and FRR errors, the total accuracy related to that users is calculated as (6) : Ultimately, to calculate the overall verification accuracy of the SIGMA database, the average operation is applied to the 200 individuals' accuracy as in (7): To reproduce the work and experiments the following parameters are configured as follows: for the CNN training, optimization type is either the adam (adaptive moment estimation) 23 , or the sgdm (Stochastic Gradient Descent with Momentum) 24 in the experiment. However, adam is better than sgdm in terms of the recognition rate. Also, Learn Rate Schedule is configured as piecewise, Max Epochs (1 iteration / Epochs) is configured as 80, Learn Rate Drop Factor is selected to be 0.9, and Learn Rate Drop Period is 5. These hyper-parameters have been selected by using the try and error method to adjust the reference trained model.

Results and Discussions:
The experimental results of this research are reported in two forms. Firstly, tables illustrate the verification rates, and the second form is the graphical representation to show the successful accuracies for each user in the SIGMA database containing 200 users.
Accordingly, Table 2, illustrates the results achieved by the proposed verification methodologies by using ADAM training CNN type. In Table 2, the most important thing is the accuracy obtained from the two types of error FAR and FRR. Six experiments have been conducted with a different setting in terms of the HOG feature extraction by altering the Cell-size three times, as follows, [100x100], [110x110], and [120x120] that resulted in feature vector as 540, 432, and 288 respectively. Also, the signature image size for all of them is unified having size [460x613] gray-scale image type. For each feature vector length, the experiment was repeated 2 times to achieve the reliability and validity of the experiment execution. As it is obvious that the best-achieved accuracy is 97.1% because FAR is up to 4.15% and FRR is up to 1.65%. when the vector length is 432 coming from the cell-size [110x110] pixels. other experimental results as in Table 2 are near to the best-achieved accuracies. Next, to show the accuracies for all 200 users in the SIGMA database, Fig. 5 depicts the accuracies, FAR, and FRR for each user. As shown in Fig. 5(a), FAR rate error for 200 users, as it is clear that the errors start increasing from the user after 120. Also, Fig. 5 (b) is the FRR error type for all 200 users, for instance, that sample 140 has FRR is 0.4 (40%).
In terms of Fig. 5(c), all 200 users' accuracies have been depicted. For example, most of the users have accuracies up to 1, which means 100%. the average of them is calculated based on Eq. (7), which is reported in Table 2 for FAR, FRR, and accuracy 4.3%, 1.55%, and 97.07% respectively. Similarly, Table 3, illustrates the results achieved by the proposed verification methodologies, in which the optimizer is based on sgdm training CNN type. In Table 3, the most important thing is the accuracy obtained from the two types of error FAR and FRR. Also. six experiments have been conducted with a different setting in terms of the HOG feature extraction by altering the Cell-size three times such as [100x100], [110x110], and [120x120] that resulted in feature vector as 540, 432, and 288 respectively. Also, the signature image size for all of them is unified having size [460x613] gray-scale image type. For each feature vector length, two times of experiments have been conducted to achieve the reliability and validity of the experiment execution. As it is clear, the best accuracy that has been achieved is 97.05% when FAR is up to 3.6% and FRR is up to 2.3%, for the vector length is 288 coming from the cell-size [120x120] pixels. Other experimental results as in Table 3 are near to the best-achieved accuracy. The accuracies based on sgdm training for all 200 users in the SIGMA database, Fig. 5 depicts these accuracies as well as FAR and FRR for each user. Fig. 6(a), FAR rate error for 200 users. Also, Fig. 6 (b) is the FRR error type for all 200 users, for instance, that sample 199 has FRR is 0. 1 (10%). In terms of Fig.6(c), all 200 users' accuracies have been depicted as it is shown that most of the users have accuracies up to 1. Thus, the average of them is calculated based on Eq. (7), which is reported in Table 3 for FAR, FRR, and accuracy are 3.6%, 2.3%, and 97.05% respectively. It is worth mentioning that each training and testing time for one user by using the proposed method is taking around 3-5 seconds. As a result, the training time for all users of the SIGMA database (200 users) database consumes approximately 1000 seconds (16.7 minutes). The experiments have been conducted on workstation hardware as characterized on a single CPU having hardware specification up to 6G-RAM, Core-i3, 2.4GHz.
In terms of the benchmarking and comparison for the proposed work with the state-of-the-art works, Table 4 lists the accuracies of the handwritten signature verification existing in the literature used in the same SIGMA database. As it is clear that the proposed methodology of using the following steps: contrast enhancement preprocessing, signature length normalization, HOG, and CNN has outperformed the existing works used the same SIGMA signature database, in which this database is publically published online 20 . It is worth mentioning that the limitation of this work needs a high CPU ability to be able to run the convolution operations and other CNN layers. For instance, if the proposed algorithm needs to be uploaded to an embedded system that system needs to be included with a suitable processing speed to avoid any delay happened in the real-time execution.

Conclusion:
Offline handwritten signature verification has a high rate of intra-user variability, which decreases the general recognition rate. In this paper, the verification rate has been improved by using the proposed methodology as compared with the stateof-the-art works. This method is specified by the following steps: contrast enhancement and image noise filtering, signature length normalization, feature extraction by exploiting HOG, and finally the classifier using one-dimensional convolution CNN. The database used for the evaluation is named SIGMA handwritten signature database, which contains 6000 signature samples with the skilled forgeries samples and published online. The best successful accuracy achieved is up to 97.1% when the FAR and FRR are 4.15% and 1.65% respectively in which this accuracy outperforms all the existing works that are using this SIGMA database. These accuracies have been obtained when the training optimizer is ADAM, which is better than the SGDM optimizer, Also, the length of the feature vector is 432 features extracted by using the HOG method. In future work, trying to decrease the error rate as possible as could from 2.9% to zero by using different possible deep-learning algorithms.