New algorithms to Enhanced Fused Images from Auto-Focus Images

: Enhancing quality image fusion was proposed using new algorithms in auto-focus image fusion. The first algorithm is based on determining the standard deviation to combine two images. The second algorithm concentrates on the contrast at edge points and correlation method as the criteria parameter for the resulted image quality. This algorithm considers three blocks with different sizes at the homogenous region and moves it 10 pixels within the same homogenous region. These blocks examine the statistical properties of the block and decide automatically the next step. The resulted combined image is better in the contrast value because of the added edge points from the two combined images that depend on the suggested algorithms. This enhancement in edge regions is measured and reaches to double in enhancing the contrast. Different methods are used to be compared with the suggested method.


Introduction:
All-in-focus imaging idea depends on multiple focused images to get one sharp image (1,2). The actual fusion process can be performed at different levels of information representation like pixel, feature, and decision (3). The passive autofocus techniques depend on choosing the optimal focus plane in the scene, where a minimal set of images focused at different depths is automatically selected (4). The camera aperture is considered one of the most important parameters that affect image details like blurring the background. It affects the image which effects on an image by getting a photo darker or brighter (5). The first lesson for the photographer is that the pupil of a human being is the same as a camera aperture. This means the eye pupil changes its size to control the light income like a camera sensor. Aperture is like the "pupil" through closing and opening the shutter to let the light come in (6). Different researches are available online in the field of data fusion. Hengjun Zhao (7) considered a grey image as a two-dimensional surface and depends on neighbour distance to measure pixel's sharpness, where the smooth image surface is restored by kernel regression. Yu Liu et al., (8) proposed multifocus images with dense scale-invariant feature transform (SIFT). But the efficiency of the computation is lower when compared with the guided altering-based fusion method. Moreover, it needs high memory for this method of feature descriptors to need to be stored. Durga Prasad Bavirisetti et al., (9) employed an average filter for multi-scale image decomposition because it is limited to be used for focused and sharpened regions and convert them to the fused image. Yu Liu et al. (10) presented a deep convolution neural network (CNN) trained by high-quality image patches and their blurred versions. But this method provides a preliminary attempt aiming to show the great potential of CNN. It needs complicated strategies to improve fusion quality. Discrete wavelet transforms technique method is used to transfer a low and high-frequency domain with local saliency method to enhance fusion image (11).
In this study, propose a different multifocused method using new algorithms. The suggested algorithms depend on calculating the standard deviation for each colour band for two input images. Then the evaluation depends on the contrast measure in image edge points. Finally, it evaluates the homogenous regions depending on the correlation measure. After that, the resulted fused image is improved compared with other methods.

Autofocusing Image:
Recently, the autofocusing field has covered a wide range in the image capturing. Imaging captured systems depend on a fixed focus treatment to shoot a target. In the conventional imaging system, the out-focused is caused by many reasons like moving an object during the imaging with a still background and/or the camera is moving which causes a disturbance in the lens (12). All these problems are behind building a focusing system to get an optimally focused image. Technology developed camera properties, namely autofocusing, even while the scene or camera is moving. It depends on the focal point of the imaging system to give a high contrast image. In an imaging system, the autofocusing can be divided into two groups (i) Active Focus (AF) and (ii) Passive Focus (PF). Active autofocusing is used for the single-lens reflex camera (SLR), but it is not a good choice with a separate light source. Passive autofocusing is used widely with different scenes and this can be divided into two sub-categories (i) Contrast (ii) Phase. Passive autofocusing depends on the light reflected from the object. Therefore, the passive autofocusing is used in Space Imaging System (13). The demand for developing new techniques for image fusion meets the requirements of the industry.

Evaluate Fused Image
The statistical features describe image properties because these properties can characterize the optically modulate the natural image. For instance, an image statistic for a surface shows its enhanced color saturations, while its quality perception is reduced luminance contrasts (14).

Mean
Mean filter is an easy method of smoothing images; this can be by reducing the amount of intensity variation within the filter mask. It is used with the standard deviation method. The average mean ( ) of the brightness's over the image size (N×M) within a region (R) is given by (15,16): Standard deviation (σ) is a criterion value which finds the variation or dispersion of a set of data values. A low value of (σ) refers to be close to the mean (also called the expected value) of the set. The high value of (σ) refers to a spread value out of the set over a wider range of values (17). It can be represented as: Sobel operator Sobel operator deals with edge detection in image processing and computer vision. Sobel and Feldman displayed the idea of a "mask 3x3 Image Gradient Operator" at a talk at SAIL in 1968 (17). Sobel operator is based on the mathematical convolution image with a small, separable, and integer valued filter in the two directions, horizontal and vertical, so relatively not expensive in terms of computations, and the gradient approximation that it obtains is relatively crude, in particular for highfrequency variations in the image. The Sobel operator uses two 3×3 masks, which are mathematically convolved with the source image to get approximations of the derivatives for two changes, horizontal and vertical.

Contrast
The rationale behind this is that a small difference is negligible if the average luminance is high, while the same small difference matters if the average luminance is low (see Weber-Fechner law). A common definition to contrast (CT) is a dimensionless relative measure of the intensity of two adjoining objects (18). In psychophysical studies, the contrast can be represented between two intensities I max and I min (I max is for the brighter region) as the Michelson contrast(19): Correlation Correlation equation describes a relationship between data. The formulas return a value between -1 (indicates a strong negative relationship) and 1 (indicates a strong positive relationship). A result of zero indicates no relationship. The formula to compute the correlation between fused autofocus pixel (F) and original pixel (A) is (20): matching filtering techniques applied using a sliding window of dimensions 3×3 pixel.

Algorithm Approach
In this research, a new suggested method uses the pixel's contrast of the first image fused with the corresponding pixel's contrast value of the second image. Two algorithms are required to build up suggest controlled technique: Algorithm 1 the suggested technique Input: two autofocus A and B of size (r×c). Output: the output is a fused image C of the same size (r×c). Step1: calculate standard deviation ( , ) for each band for two images A and B.
Step2: Calculate the difference between the two standard deviation σ( , ) = ( , ) -( , ) Step3: Calculate the standard deviation sign as: sn = sign( σ( , )); Step4: the fusion depends on the result of the following equation: ( , ) then sn = 0 Step5: decide which data is the result. If the sign is positive, then image A data is used otherwise image B data is used. In case both data are the same, then it is optional to choose any data.
Step 6: the end.

Algorithm 2 Calculate Contrast and Correlation
Measure value Input: the input of the algorithm is: 1. Autofocus images, A, B, and C of size (r×c).
Output: the output is: 1. The contrast values of edge points. 2. The correlation coefficient values. Step1: Apply edge detection method (Sobel operator) for all input images.
Step 2: find the adding edges points to the fused image C from image A and B by using the following equations: Add edge points = A -C. Add edge points = B -C.
Step 3: Calculate contrast value to the edge image points of A, B, and C. Then evaluate the difference between edges in A with C, and B with C to get S AC and S BC .
Step 4: Compute the correlation coefficient in fused images C. Three blocks located at different homogeneity regions with different window size.
Step 5: the end.

Supported Image
From the national aerospace laboratories, the indigenous aircraft Synthetic Aperture Radar Advance Simulators (SARAS), a pair of multi-focus plane is used and it is chosen to be a well-known test image. Image size is (512×512) pixels and bit depth (8bits) shown in Fig. 1a (21). A clock test image focused in two halves where the image size is (131×131) pixels and bit depth (24bits) shown in Fig. 1b. A multi-focus dollar image used with size (123×123) pixels and bit depth (24bits) shown in Fig. 1c. A multi-focus book image used with size (122×119) pixels and bit depth (24bits) shown in Fig. 1d (22). The sample A and B are referring to the first and second image respectively.

Results and Discussion:
Within this paper, two images of the same scene but with a different focus on the components of these images are studied and shown in Fig. 1. After applying the suggested algorithm, the resulted fused image (C) can be represented in Fig. 2 for the (a) plane, (b) clock, (c) dollar, and (d) book. This research has three parts; the first part consists of fused the two tested images as in Algorithm 1. The second part focused on the estimating the image quality by using contrast criteria in algorithm 2 (step 2 and 3) before and after the fusion process to test the process quality. The added edge points are considered an important parameter to enhance the fused image, and the contrast of the added edge points shown as S AC and S BC . The results are shown in Fig. 3, where the contrast values show this value increases in the edge points in the fused image (C) and it is constant for the edge points for the S AC and S BC .

Figure 3. Relationship between contrast with autofocus image befor and after fusion process and difference between them AC and BC for (a) airplan (b) hour (c) dollar and (d) book
There are different methods within the fusion technique presented and used within this paper. Some of them are used for comparison like Brovey Transform (BT) Algorithm (23), color normalized transformation (24), Local mean matching (LMM), Wavelet Transformation (WT), and High Pass Filter Additive (HPFA) (25). All these methods are compared with the suggested method with mean, contrast, and standard deviation criteria. The suggested method showed good results and enhancement in image output because of the modification within the suggested algorithm. Fig. 4 shows the comparison results with the suggested method, where A and B are the input images. The third part used correlation criteria to estimate the image fused quality as in algorithm 2 (step 4). The correlation between the first autofocus image (A) and second autofocus image (B) is calculated. Three blocks are taken from different homogeneous regions within the image at a different size (40×40), (30×30) and (20×20) to find the correlation and the displacement of these blocks by 10 pixels within homogeneity region shown in Fig.  5. Figure 6 shows the relation between the For the note, Fig.6d has three plots because it is colored image and has three color bands, therefore, statistical process done for all bands. The correlation for input images A and B in the homogenous regions with the fusion image C are shown in Fig. 6. The correlation for the fusion image is better than the input images.

Conclusions:
This paper shows a new autofocus image fusion algorithm. The new idea is focusing on nonhomogenous regions with fine details of the input image. This can be achieved using the local standard deviation as a criterion. The fine details of the two tested images dependent on the standard deviation value. Then these details are kept in a resulted image until complete. The best value of STD choses the block from two images and add it to the resulted fused image. There are two methods used to evaluate the resulted image. The first contrast method is used to increase the contrast to the edge points of the resulted image. Edge points are added from both input images to the resulted fused image. The second correlation method is used to evaluate the homogenous region. Both methods show good results of the fused image. The suggested method works with different kinds of images and sizes. For the suggestion, it is recommended to use entropy for lightness, Naturalness Image Quality Evaluator (NIQE) and gradient in the edge regions to be a criterion. Comparison is added to show the efficiency of the suggested algorithm.