Human Face Recognition Based on Local Ternary Pattern and Singular Value Decomposition

: There is various human biometrics used nowadays, one of the most important of these biometrics is the face. Many techniques have been suggested for face recognition, but they still face a variety of challenges for recognizing faces in images captured in the uncontrolled environment, and for real-life applications. Some of these challenges are pose variation, occlusion, facial expression, illumination, bad lighting, and image quality. New techniques are updating continuously. In this paper, the singular value decomposition is used to extract the features matrix for face recognition and classification. The input color image is converted into a grayscale image and then transformed into a local ternary pattern before splitting the image into the main sixteen blocks. Each block of these sixteen blocks is divided into more to thirty sub-blocks. For each sub-block, the SVD transformation is applied, and the norm of the diagonal matrix is calculated, which is used to create the 16x30 feature matrix. The sub-blocks of two images, (thirty elements in the main block) are compared with others using the Euclidean distance. The minimum value for each main block is selected to be one feature input to the neural network. Classification is implemented by a backpropagation neural network, where a 16-feature matrix is used as input to the neural network. The performance of the current proposal was up to 97% when using the FEI (Brazilian) database. Moreover, the performance of this study is promised when compared with recent state-of-the-art approaches and it solves some of the challenges such as illumination and facial expression.


Introduction:
One of the aspects that distinguish people is the ability of individuals to identify faces. The face plays an important part in determining a person's identity. Paying attention to the human ability to face recognition without knowing how faces are decoded by the human brain is an important aspect of designing an algorithm for face detection and recognition 1 . People may recognize a wide range of faces and recognize recognizable faces through numerous variations in viewing environments, speech, age, gender, and glasses as distractions or hair modifications 2 . Face recognition has been the subject of a lot of experiments and studies over the last few decades 3,4 , and curiosity about theories and algorithms for face recognition is already increasing rapidly. A significant number of recent incidents, such as illegal tracing and terrorist attacks 5 , have revealed surveillance infrastructure flaws. Unmanned and automated automobiles, video monitoring, suspect detection, building access control, and autonomous and automated vehicles are only a few instances of concrete applications that are gaining popularity across industries. Governments are now interested in inventing and developing a novel computer-processing scheme based on behavioral or physical characteristics, known as biometrics. Biometrics is a useful device; it may connect to other applications that need access to control or protection. Many human characteristics, such as speech, signature, face, and finger/palm 6 geometry, have been examined in recent years 7,8 .
Analysis of images with faces that captured by digital camera still faces many challenges that reduce the recognition accuracy such as: The head may move by some angles (egocentric rotation). The position of the camera will affect the facial appearance and/or the shape. Face occlusions and the variation of the face due to facial expression, beard, mustache, glasses, and aging (changing the face specifically when the time between images is significant) all add difficulties to face recognition. In addition, one important issue is the variation of illumination. The quality of face image has a significant effect on face recognition 9,10 .
Face recognition tends to be a reasonable match between efficiency, social approval, anonymity, and protection across all of these. Consequently, this analysis used the Local Ternary Pattern (LTP) and Singular Value Decomposition (SVD) to improve the accuracy of the face image and achieve stronger results for face recognition by overcoming the problems of variation in illumination and facial expressions.
The remainder of the article is laid out as follows: The second section briefly introduces similar works. The proposed method and details from the Brazilian face database are presented in Section 3. Section 4 contains the experimental findings as well as a discussion. Section 5 involves future work and Section 6 concludes with some closing remarks.

Related Works:
In this section, a brief review of the Levenberg-Marquardt backpropagation (LMBP), SVD, and LTP methods is provided: Wang et. al. 11 used SVD in the Fourier domain. The proposed method shows high performance for the face-matching task and is certainly useful for face detection and skin color detection. However, it is very sensitive to changes in illumination, which affects the accuracy of the results 11 .
Hu et. al. 12 merged SVD and local near neighbors for face recognition, although, the proposed method gives acceptable recognition rates, however, it needs to be improved to give better results to resist environmental changes because it only works in a standard environment 12 .
Loderer et. al. 13 presented a feature extraction method for a face recognition system based on a genetic algorithm; the best recognition accuracy was achieved by using the LTP patterns. The performance and accuracy of this method may vary, due to varied illumination, facial expression, and pose. The disadvantage of this method is that it is unstable 13 .
Harakannanavar et. al. 14 introduced an improved LTP method. This paper has good robustness for face recognition in a non-constrained environment, especially face recognition with illumination change. The disadvantage of this method is that it is weak in recognizing faces in facial expressions and pose face images 14 .
Kakarwal, et. al. 15 proposed a face recognition method based on using visible, thermal, and fused image databases. The artificial neural network is used as an efficient technology to enhance the results. The backpropagation and Levenberg-Marquardt algorithms were compared by using the same database. The Backpropagation algorithm achieved an accuracy of 83.92% compared with the Levenberg-Marquardt accuracy, which gives a better result, its accuracy has reached 97% 15 .
Baker, et. al. 16 suggested a face recognition method based on LMBP and using PCA. This work achieved high recognition accuracy, but, the disadvantage of this method is the requirement for a huge amount of time for training and testing 16 .

Local Ternary Pattern (LTP):
LTP was first proposed by Tan and Triggs 17 to use 3-valued codes (0, 1, and -1) instead of two values (0, 1) as in the Local Binary Pattern. The gray value will be changed to one of three values in LTP according to the suggested threshold which is used as a control for the assigned values instead of the gray image value as shown in equation 1.
where t is a user-specified threshold. LTP codes are more resistant to noise but no longer strictly invariant to gray-level transformations. The LTP encoding procedure is illustrated in (Fig. 1(a) and (b)). The threshold was set at 5.
Using 3n valued codes when using LTP for visual matching, but the uniform pattern statement still holds in the ternary situation. For convenience, the experiments described have used a coding scheme that splits each ternary pattern into positive and negative halves, as seen in (Fig. 1(c) and (d)), then process them as two different channels of LBP descriptors, where the separate histograms and similarity metrics are computed, the effects only being merged after the computation.

Singular Value Decomposition (SVD):
The SVD technique is well-known in applied mathematics, engineering, and scientists working on applied linear algebra, systems and control theory, signal processing, image processing (mostly used for face recognition), data mining and statistics, psychometrics, and chemometrics 18 . The Singular Value Decomposition (SVD) is a transformation process based on linear algebra that converts the matrix (A) into three product matrices, A=USV T , where the matrices (U and V) are orthogonal matrices and the matrix (S) is a diagonal matrix. So, UU T =1 and VV T = 1, and S is the summation of the diagonal entities λ1, λ2….. which represent the singular vectors (called singular values) of a matrix (A). So, where r is the rank of matrix A. Singular values specify the luminance of an image layer while the corresponding pair of singular vectors specify the geometry of the image layer 19 .

Levenberg-Marquardt Backpropagation Neural Network (LMBP):
The architecture of the backpropagation uses the design of the preprocessing algorithm 20 . It was re-propagation that was crafted by generalizing Widrow-Hoff's learning rules to multi-layered organizations and non-differential exchange work. Input vectors and associated target vectors are used to prepare an organization until it can approximate a resource. Associate input vectors with explicit target vectors or input vectors as defined in this study. A Network with bias, a sigmoid level, and a direct linear level is equipped to approximate any resource with a limited number of gaps and discontinuities. There are two ways to calculate propagation; back and forth. The forward way is to put a network at the forefront, simulation, starting or initializing weight, and prepare the network. The biases and weights of the network are updated on the contrary.
The Levenberg-Marquardt algorithm aimed to evolve towards the second query training speed without saving the Hessian framework matrix. By the time the presentation work contains an aggregate type of squares as is usual in the preparation of feed organizations, at this point the Hessian network can be approximated as: (3) and the gradient can be computed as: (4) Where J is the Jacobian matrix structure, and e is a vector of network errors and the first subordinates of the entity errors concerning weights and patterns. The Jacobian matrix can be programmed using a regular reprocessing method (see 21 ). This method is far simpler than the Hessian matrix realization. In the following Newton-like modification, the Levenberg-Marquardt algorithm uses this approximation of the Hessian matrix: At the point where the scalar μ is zero, this is just Newton's strategy, using the large Hessian matrix. When μ is huge, it becomes a small slope in advance. Newton's fastest and most accurate technique is at least almost a minimum error, so it's about switching to Newton's strategy as possible as expected, given the circumstances. Consequently, μ decreases after each successful advancement (decrease in work) and grows just as speculative advancement resumes. Therefore, the presentation capacity will be systematically reduced with each algorithm cycle 20 .

Figure 2. Some examples of image variations from the FEI face database.
The proposed method consists of three main steps: preprocessing, feature extraction, and image classification. In this classification step, there are two phases; training and testing. Fig. 3 shows a summary of all the steps used in the proposed method of three main steps (preprocessing, feature extraction, and classification of images).

Pre-processing:
Preprocessing is the first step that prepares the image to be suitable for processing. The input image is an RGB image, for this suggestion, the RGB image was converted into a grayscale image. Moreover, the image was resized to 120x100. The sizes of all images in the database and input face image are unified, as shown in Fig. 4 (b).

Feature Extraction:
The gray image will be processed to extract the LTP features, as shown in (Fig. 4 (c)).

Figure 4. (a) Original image (b) Grayscale image, which resized to 120x100 pixels, and (c) LTP Image.
For the segmentation step, the resulted image from the LTP was divided into sixteen equalsized non-overlapping blocks (each with size 30x25). Figure 5 shows the sixteen blocks. Consequently, each block is divided into many non-overlapping sub-blocks with a size of (5x5), where the result is 16x30 blocks. For each sub-block, the SVD transformation is applied, where the SVD transforms the block into three matrices. The norm of the diagonal matrix is calculated and stored in the feature matrix (16x30 matrix).
The norm for each sub-group is determined by using equation 6. The result is a vector of thirty features (norm).
For the sixteen blocks, the result will be a matrix of (30x16) features. This features matrix will be stored with a corresponding image in the image dataset, which will be used in the future for a test to reduce the time.

Image Classification:
The last step is to make a decision. The decisions can be either matching or non-matching. First, the LMBP was trained on the images from the database. In this work, the Brazilian database was used which consisted of 200 face images each image with two poses. LMBP used 140 images for training and 60 images for testing. In the training step for each person, the features of the two poses were reduced to sixteen features by using the Euclidian distance in equation 7. For the thirty subblocks of two images, the distance is measured between each sub-block in two input images, then the minimum distance is selected as the feature for that main block. Then, the input to the LMBP was sixteen features, and the output was, in this case, either one (for matching) or zero (for nonmatching).
= √∑ (( − ) 2 ) =1 (7) Where p and q are pixels of two different images of the same size, n are all the pixels of an image.
The architecture of the LMBP used in this proposal includes eighteen hidden layers, sixteen nodes as input nodes, the number of epochs is twelve and the activation function was (symmetric saturating linear transfer function). In the testing process, the output can be any number between zero and one, where the system can regard the value greater than or equal to the threshold as a matching value, otherwise, it is not matching, as in relation 8. In this work, we found that the best threshold is 0.4.

Results and Discussion:
Experimental results from the proposed method will be evaluated via specificity, accuracy, precision, recall, and F1-Score. The first step in this section is the test of how the parameters are selected and used in the proposed method and how they affect the recognition accuracy.
The LMBP has three important parameters that have highly affected recognition accuracy. These parameters are learning rate, number of hidden layers, and activation function. Fig. 6 shows various values of the learning rate tested to find the best accuracy, which concludes that the learning rate with a value of 0.6 has produced the highest accuracy. The number of hidden layers is tested. Different numbers of hidden layers are selected and with each number, the accuracy is determined. The results are shown in Fig. 7, it can be concluded that the best number of hidden layers is eighteen layers.  The third parameter tested is the activation function. Many activation functions can be used in this research work and each one has a specific effect on the accuracy of the recognition. The activation functions can be divided into two groups, and each group was tested alone. The first group output values ranged from -1 to 1, while the second group output values ranged from 0 to 1. Each activation function group is tested to find which one gives the best accuracy. As shown in (Fig. 8 and Fig. 9) the activation function symmetric saturating linear transfer function (tansing) is the best.  Another test is implemented to find the best threshold that produces better accuracy. We found that the threshold equal to 0.4 is the best as shown in Fig. 10. The Root Mean Squared Error (RMSE) for the proposed method is determined with a specific dataset and a random dataset. The minimum RMSE for the specific dataset is 0.0242 and the random dataset is 0.0065 as shown in Fig.  11 using a random dataset gives better results than using a specific dataset.  The other test was to determine the best number of the epoch that gives the best accuracy, at this test various numbers of the epoch were used and various testing error tools (of confusion matrix) were used as shown in Table  1. The performance of the proposed method is measured according to the structure similarity, where it is measured between the images. In this test, 200 images were used as shown in Fig. 12. The results showed that most of the images performed (similarity) near one value. This is a good result. The confusion matrix is used to measure the algorithm performance. The results are summarized in Fig. 13, it is clear that the proposed algorithm works well and has very good performance.   The images in Fig. 15 represent the limitations of this algorithm. The proposed algorithm faces a problem with recognizing the images with faces rotated at an angle of more than 45 degrees. Finally, the proposed algorithm is compared with other similar works as shown in Table 2 15, 23-28 .

Conclusion:
The suggested method uses a hybrid method for face recognition. It combines LTP, SVD, and LMBP. Use of LTP assists in increasing the recognition rates. Using a small block size with SVD gives better results than using a large image size. The accuracy of face recognition is very promising. The contribution of this work is to solve the problems of image illumination variation, and the problem of facial expressions, which both reduce the accuracy. In addition, the proposed algorithm classified faces with different poses. It is possible to recognize rotated faces at an angle of 45 degrees, but it fails at more than 45 degrees.

Future Work:
The following are some perspectives that may help extend the work of this research:  Increase images size for increasing validity of the results and it improves, allowing subsampling of certain groups of data.  This work can be extended by using 3D images.  Using different algorithms or the same algorithm and with edge detection for detection of the facial expression and extraction.