An improved neurogenetic model for recognition of 3D kinetic data of human extracted from the Vicon Robot system

a


Introduction
To tackle complicated problems in our lives, it needs a technique that includes analytical components and is capable of processing and providing potential solutions.In our suggested model, it proposes using an artificial neural network (ANN).The artificial neural network is the cornerstone of artificial intelligence and can solve a wide range of complicated issues that are challenging for people or statistical techniques to handle 1 .Deep learning neural networks have been extensively utilized for classification, detection, and identification 2,3 .which is a strong tool for data forecasting in a range of systems 4,5 .Additionally, it was suggested to use a genetic algorithm to improve the learning process 6,7 .Choosing a genetic algorithm to enhance work performance is more potent.The efficiency of these evolutionary-based algorithms in resolving optimization issues is well recognized 8,9 .The goal of our research is to create a system that combines the strengths of the two algorithms, which may then be employed in many practical parts of our lives, such as control, diagnosis, and analysis 10,11 .The proposed method was utilized to distinguish between normal and aggressive human action by creating an efficient tool that combines two crucial algorithms.Reviewing the earlier studies, the suggested approach put forth in this study has already overcome several challenges by using sufficient data in the proposed model and enhancing the learning process in an artificial neural algorithm with a genetic algorithm by choosing the best layers and enhancing the other layers through crossover and mutation.
Several academics are interested in the process of identifying and recognizing the kinds of human movement because of its significance and requirement for such systems [12][13][14] .Academics are interested in the usage of wireless equipment networks because of the numerous applications for identifying human physical motions, including healthcare, rehabilitation, athletics, elder monitoring, sports, and human-robot interaction [15][16][17] .Because human activity is a work sequence that includes temporal information, the hybrid method of assessing activity is preferred [18][19][20] .
Human Activity Recognition (HAR) is a technique used in health care to monitor the actions of patients and the elderly who have illnesses such as dementia, Parkinson's disease, and others.As a result, HAR can aid in spotting abnormalities in routine daily tasks, avoiding any negative consequences.Since HAR systems are utilized in healthcare and elder monitoring, compromising their accuracy is risky 21 .Safety is becoming increasingly critical, and video capture and storage equipment are always evolving to meet this demand.These devices, however, must be supplied with a system capable of consistently recognizing a wide range of anomalous conditions while minimizing human error 22 .
The rapid development of artificial intelligence (AI) has paved the way for human activity recognition (HAR).It focuses on learning about a person's individual behavior (or movement) 23 .Several studies on the identification of various human activities have been conducted 24 .End-to-end deep learning models are increasingly being used in safety-critical human activity recognition (HAR) applications, such as healthcare monitoring and smart home control, to minimize developer load while improving prediction model performance and resilience 25 .As people age, they are more likely to have issues including cognitive decline, memory loss, chronic disease, vision, and hearing impairment, as well as many other disorders.They may also find it challenging to interact socially.The patients' capacity to carry out their everyday tasks is hampered by all these challenges, which also lower their quality of life and necessitate their asking for assistance from others 26 .
Thus, the ability to recognize human actions is necessary, as is the need to establish natural contact between humans and robots.Among the systems capable of analyzing human action is the Vicon robot system, which serves as a support function as it can read and analyze human movement and allows interaction between humans and robots.As shown in Fig. 1, our proposed method was used to evaluate the 3D data that indicated normal and aggressive human movements acquired from the Vicon robot system 27- 29 .
The paper is structured in the following sections: The Related Work Studies section describes the review of literature in the past related to the studies related to the proposed work.The dataset section describes the description of using data to apply our model and the features of the data.The neuro-genetic model section describes the mechanism of our proposed model through the algorithm and the flowchart.The method section describes the implementation of our proposed technique.The result section describes the outcomes of the implementation system.The conclusion section describes the summary of the proposed work and its implementation in the paper.

Related work studies
Researchers recently emphasized the effectiveness of hybrid designs that integrate ANN with metaheuristic approaches.In one of the studies 1 , the goal was to evaluate the accuracy and consistency of the feed-forward-back propagation neural network (FFBPNN) approach, improving the performance of the solar transformer-consumption (STC) system.The MATLAB code for the FFBP method was designed to enhance the values of reliability and cost function by reducing the error to 0.0001 accuracy.Its gradient descent learning mechanism may update the neural weights.In the other study 2 , the classification framework for potato tuber diseases was provided.
The FFNN was built and set up with a sigmoid transfer function.The rate of success for training potato picture identification was 100%.Test results for the batch of potato photos were 91.3% successful.
There were some implementations addressing the problem of prediction by neuro-genetic algorithms; the evolutionary operations investigated in one of studies 10 in interneuron synaptic structure search were performed using a structure synthesis technique designed for a feed-forward multilayer network trained by a teacher.To overbuild networks with one hidden layer, crossover operations over neural network architectures were utilized, where the number of neurons is randomly chosen within a certain range.The created neural network handles diagnostic problems with 72-99% accuracy for the compositions mentioned in the study.In one of researches 11 where the genetic algorithm employed the neural model to imitate the parameters, the computing complexity was less than the other optimization methods, and the neuro-genetic algorithm offered the ideal design for the parallel robot.The purpose of one of the works 12  In one of the researches 14 provided a unique, dataguided paradigm for complicated HAR and automated labeling based on cell phones.The findings reveal that the hierarchical k-medoids (Hkmedoids) method produces high-quality labels (93.89%).Machine and deep learning approaches, as well as other classic dimensionality reduction and TDA feature extraction techniques, are proposed in other works 23 to tackle the HAR problem.The tests are carried out using two publicly available datasets (WISDM and UCI-HAR).The system's accuracy was between 95.34% and 100% when applied to the datasets and the three experiments.

Data Set
The data set in our suggested model was connected to physical activities.The experiment comprised seven male and three female participants (ages 25 to 30) who had witnessed violence in the form of physical fighting.During 20 different studies, each subject was instructed to do 10 normal and 10 aggressive activities.In a 3D intelligent environment, a human is seen undertaking some physical activity.As a perception-to-action unit, two external devices-a 3D tracker (the Vicon system) and a mobile robot (the SCITOS G5)-collaborate to create accomplished activity classification and mechanical attribute-based data 29 .

Vicon Robot Module
The 3D data and pictures are obtained in this system by a low-level machine that captures and combines data from nine high-resolution infrared cameras.
Then use the motion model extraction module to obtain 3D models based on the alignment of markers on an item.Because the endpoints of the limbs are those that create physical actions, these kinematic models focus largely on the limbs and, more particularly, on the terminal stimuli, and the sampling is repeated to capture the picture per second to construct the time series at the end.
Vicon is a collection of activity information expressed in time series to be supplied to the robot for classification processing and assessment evaluation, ranging from image processing to time series creation.It categorizes the data and replicates a portion of the data to demonstrate the action classification performance.

The data set features
The suggested data for the proposed system's implementation are three-dimensional data, which are numerical values that describe a human's physical motions.This information was gathered using the Vicon robot system, which detects bodily motions, and it was preserved and classified for use in training the intended hybrid system.
There were two sorts of physical acts in the data set: normal and aggressive.
Each of these motions has a unique set of qualities, which are represented by ten different forms of movement for each sort of physical action, which are normal and aggressive.
The following is the data that reflects the type of normal physical action: Normal: bowing, clapping, handshaking, hugging, jumping, running, seating, standing, walking, and waving.
And here are the data that describe the sort of aggressive physical action: aggressive: elbowing, front kicking, hammering, headering, kneeing, pulling, punching, pushing, side kicking, and slapping.
Each of the ten types of physical movements described the characteristics that are used to train the system and design it for this type of movement.
These characteristics are represented by parts of the body that indicate the type of movement, and their three-dimensional measurements are taken, and each of them is organized and arranged in the manner required to be included in the system.The right arm, the left arm, the right leg, and the left leg-each body part contains a pair of markers that can be used to give the type of movement (except for the head), and each of them is three-dimensional data.

Arm signs: wrist and elbow
Leg signs: ankle and knee CRDS: The three coordinates (x, y, and z) that define the position of each 3D marker within space.
x: x coordinate y: y coordinate z: z coordinate Thus, each procedure has nine markers, and with each marker there are three coordinates, as shown in table 1. x y z x y z x Y z x y z x y z x y z x y z x y z x y z

Neuro-Genetic model
In our work, a hybrid smart model was proposed, which is a system consisting of two algorithms, namely the feedforward artificial neural network and the genetic algorithm, to combine the characteristics of the two algorithms utilize their advantages and integrate their work mechanisms to produce an effective system that is more efficient and accurate to be used to give you the value of predicting the required solution.Artificial neural networks are an exact replica of a biological neural network that effectively correlates many parameters with a larger number of unknown data points.And that this algorithm has the characteristics of efficient construction that can be trained based on the input data to adapt to it to produce a system capable of predicting the solution.
The proposed algorithm consists of an input layer, 12 hidden layers (each layer has three neurons), and an output layer.There are two datasets used in the input layer in ANN which are aggressive and normal datasets represented as two neurons, these data are processed through 12 hidden layers.Using the sigmoid function in each neuron, produce the activity type recognition result for the output layer, and give the output as aggressive or normal represented as two neurons, and in the same time we get the result from each layer to keep it for evaluation the layers in the selection stage in GA.Then the result is evaluated by comparing it to the real result of a system check, to be improved after that by means of the genetic algorithm.
The GA algorithm is based on the theory of biological evolution.Survival of the fittest is the main process that is mimicked in GA.And used this algorithm to take advantage of its characteristics in preserving the individuals with good efficiency, which can give the appropriate solution for use in the next generation.It is also possible, using this algorithm, to change the characteristics of the incompetent individuals to make them better at giving the solution and the result.
To do the optimization, it needs to check each layer in the ANN to evaluate the weights of each layer by knowing the error value for each of them.To do this, it proposed to break up the original ANN algorithm into multiple ANNs based on the number of hidden layers, as shown in Fig. 3. To consider that after each hidden layer there is an output layer to evaluate the weights in each layer, in this way after each hidden layer it considers new ANN, and find out the result from each ANN, verify it, and calculate the resulting error value, it needs to keep the error values generated from each ANN in a list and consider this list as a fitness list in the genetic stage In the selection stage, it performed a sorting procedure on the fitness list from low to high, as the lower error value represents the best solution, and this layer can be kept for use in the new generation.This sorting was adopted to arrange the artificial neural networks as shown in Fig. 4. The selection process was based on the first values from the list considered to be the best ANN to be used in the next generation.
As shown in Fig. 5, the first four items from the sorted fitness list were selected to perform the crossover between the final layers, this is done by exchanging the weights between the final layers in the selected ANNs.It is important for the rest of the artificial neural networks that are inefficient to give the solution, so it took the opportunity to improve their weights through the mutation process.
The mutation process works on neurons as well as on layers, as shown in Figs. 6 and 7, respectively, but this mutation focused on updating the weights in a way that reduces the error value.This is implemented by adjusting the state in the negative direction of the obtained error value; the limits are determined by which weights are randomly generated and whose values minimize the error value.
After completing the necessary steps in GA, then combine the multiple ANNs into one ANN algorithm with multiple hidden layers and check the error generated from them after the merge.If the solution is not satisfactory; and continue the process by creating a new generation until discover a good solution.And that is by breaking up the algorithm again in the next generation and repeating the same operations, and the limits required in the mutation process require that they be set again in proportion to the final error value resulting from the previous generation for the next generation, and the result is calculated, and continue to create other generations until a Satisfying result is obtained.The following algorithm and the pseudocode explain the steps that should follow depending on the Neuro-Genetic model, and in depicted as a flowchart in Fig. 1.

Neuro-Genetic Algorithm
Step 1: Send the extracted data from the Vicon robot system from the input layer to the feedforward ANN.
Step 2: Do the sum of products with the required weights, then do the processing within each neuron by sigmoid function in the first layer and send the output to the rest of the hidden layers.
Step 3: Do the sum of the product with the required weights, then do the processing within each neuron by sigmoid function in each layer, then do the Breaking up process on the ANN to multiple ANNs by computing the output and error value from each hidden layer.
Step 4: Keep the error from each ANN in a list, get the output from the output layer, and consider it the result from the original ANN.
Step 5: Determine the outcome of the last ANN and compare it to the intended outcome; if the result does not satisfy the desired result, do the optimization process using genetic algorithm operations.Step 6: Sort the list of ANN errors from low to high to do the selection stage in GA, the enhancing phase, and rearrange the layers according to the sorting list.
Step 7: Do the crossover step in the first four layers after arranging them by exchanging the weights.
Step 8: Give an opportunity to the rest of the hidden layers by doing a mutation on the weights, changing them randomly in the negative direction of error. Step

Materials and Methods
In our work, it employed the feedforward ANN method (input layer, 12 hidden layers, three neurons in each layer, and an output layer).As depicted in Fig. 2, the two datasets (aggressive and normal) feed into ANN through the input layer with two neurons and acquire the output of the neurons using the sigmoid activation function to get the result of aggressive or normal action from the two neurons' output.Then employed the genetic algorithm for the purposes of learning and training it.
To do the implementation, one of the two datasets is fed from the input layer to the first hidden layer in ANN and processed in each of the three neurons, then fed to the rest of the hidden layers in ANN.

Breaking up process
The idea put forth in our hybrid approach's first stage for learning systems is to divide the ANN into multiple ANNs to evaluate the weights in each layer by computing the output from each layer, comparing the result with the actual output, as shown in Fig. 3, and computing the error by comparing the result with the actual output.The comparing is done by doing the Mean Squared Error process.So, it assumed that there was an output layer after each layer and assessed each method after breaking it up.Then the error values for each ANN algorithm are recorded in a list.
The genetic algorithm now plays a part.Following the division of an ANN into multiple ANNs and the keeping of the error values in the list, it must carry out the following phases of the genetic algorithm: The selection At this stage of the genetic algorithm, it needs to reorder the fitness list of error values obtained from each ANN after the breaking up process from low to high by sorting process, as the lowest value, which is actually the first four values after rearranging the order, is considered the best ANN and has the opportunity to survive in the next generation, and the higher value indicates that these values are the worst and that it cannot be chosen but rather requires a chance to improve in order to be a better ANN.The selection procedure according to Table 2a is shown in Fig. 4. Give an opportunity to the rest of the hidden layers by doing a mutation on the weights, changing them randomly in the negative direction of error.

Yes No
Do the sum of products with the required weights, then do the processing within each neuron by sigmoid function in the first layer and send the output to the rest of the hidden layers.
Keep the error from each ANN in a list, get the output from the output layer, and consider it the result from the original ANN.

The crossover
In this phase, it operates on layers that have competence and a higher chance of survival and that were selected through the selection process because they had the lowest error value when they were evaluated.This stage of the genetic algorithm is a crucial one in the optimization process because it depends on the previous stage, which is the selection stage, which is represented by rearranging the errors from low to high, and the rearrangement of the ANNs depends on this sorting.
The top four ANNs have been chosen for the crossover procedure, and at least at this point, they exchange the weights between the final layers, as illustrated in Fig. 5, and as shown in Eq. 1, the crossover process depends on the exchange of neurons' weights in the last layer between the two chosen ANNs.
Na, Nb: The two selected ANNs W: The coefficient weights for Na G: The coefficient weights for Nb i, j, k: The rows and columns for W and G in last layer k

Mutation process
After the two processes of selection and crossover, it performs an important process in the optimization step, which is the mutation process.It performs the mutation process on the rest of the ANNs, which do not have the opportunity to survive in the next generation.The mutation occurs in neurons and layers in ANNs.The weights of neurons were randomly chosen to change it.The process of mutation proceeds in a way with the negative direction of error obtained from ANN by setting the limit of the range that decreases the obtained error, then choosing the value of new weights randomly.The range is set to be within the bounds of the limit based on the actual output.
In Figs. 6 a and b, the mutation process occurs on neurons and layers, respectively, and in Fig. 7, the random choice of neurons and layers in ANN is shown.
This method requires preparing the necessary conditions to set the limit of range with a negative direction of error from the previous generation, and this is done by putting a restriction on the choice of the values randomly so that when comparing the result of the current generation with the actual output, the error decreases to be close to zero.

Combining ANNs process
After completing the enhancing phases required in GA, which depend on breaking up the original ANN into multiple ANNs, the combining process takes place on the multiple ANNs to make a single ANN by merging the last layer from each ANN depending on the fitness list order to make one ANN.
The necessary operations of feedforward ANN are performed in the resulting combined ANN, and the algorithm's result is computed and compared with the real output.If it does not achieve a satisfying result, it needs to be generated again with the necessary phases by GA to improve it till it reaches the desired output. Whereas: Cann: The combination process of ANNs

Results and Discussion
In our system, 20 different study datasets (10 aggressive datasets and 10 normal datasets) were applied to 20 models.Each experiment's data were fed into the hybrid model, and the hybrid system was trained by a genetic algorithm to determine the type of activity to be distinguished.It achieved favorable outcomes after optimizing the genetic algorithm via crossover and mutation on the feedforward ANN algorithm.It produced satisfactory results by training the algorithm to acquire the needed outcomes in the following generation after optimization.
Tables 2a and 3a, show the results obtained from the system trained on the first experiment's data type of aggressive activity and normal activity respectively; the data includes 10 types of the intended activity.The model was applied, and a satisfactory result was obtained from the last generation.
In this table, the first column shows the sequence of obtained ANNs algorithms after breaking up the original ANN, and each field in the second column shows the error value obtained corresponding to ANNs.These values are kept in a list to be sorted from low to high, as explained in the method part of the selection phase in GA, and thus it may know the  The output value of the hybrid system designed for aggressive activity is represented by the value 1, while the output value represented by normal activity is 0.
As a discussion of the proposed system, results were acquired in our work to differentiate between aggressive and normal types of activities.When Feedforward ANN was initially applied to the aggressive and normal activity data sets for each of the ten types of activities, it produced results that were unsatisfactory when compared to the actual output.As a result, the algorithm was improved using a genetic algorithm, which took the place of the backpropagation process for learning.
After the optimization process, in which the layers were assessed through the breaking process, it became clear that the best layers were adopted through the crossover process, and through the mutation process, the weights were changed in a way that reduced the error to obtain satisfactory results, and the error values became very close to zero, as shown in the tables.

Conclusion
The method of genetically modifying the feedforward ANN algorithm's structure led to an improvement in learning without the need for the backpropagation operation.
After dividing the ANN algorithm into multiple ANNs, merging them into one ANN, and then reevaluating it, the crossover and mutation processes were used to develop the ANN learning process in our model because it is possible to get better results through the generations produced from each population.The Vicon robot system's 3D data and physical behaviors were analyzed using the model, and the analysis produced a pleasing outcome.The system that is produced can also be applied to automation, control, and analytical systems.In the future, it might be possible to create a method that allows for repeated crossover and selection with better learning, as well as the option of using higherquality neurons from other layers to get the outcome.It might also be possible to suggest a random mutation based on a mathematical model that considers the inverse relationship between error and mutation.

Figure 1 .
Figure 1.The flowchart of the neuro-genetic model

Figure 5 .
Figure 5. Crossover operation between the top 4 ANN layers in layers

Table 3 . b) The error and output data results from the final generation of the normal model from 10 experiments
-Conflicts of Interest: None.-We hereby confirms that all the Figures and Tables in the manuscript are ours.Besides, the Figures and images, which are not ours, have been given the permission for re-publication attached with the manuscript.-Ethical Clearance: The project was approved by the local ethical committee in university of Рeoples' Friendship University of Russia (RUDN University)