New Versions of Liu-type Estimator in Weighted and non-weighted Mixed Regression Model

This paper considers and proposes new estimators that depend on the sample and on prior information in the case that they either are equally or are not equally important in the model. The prior information is described as linear stochastic restrictions. We study the properties and the performances of these estimators compared to other common estimators using the mean squared error as a criterion for the goodness of fit. A numerical example and a simulation study are proposed to explain the performance of the estimators.


Introduction:
The linear regression model is given as the follows: where is an × 1 column of observations that explain the dependent variable, X is an × matrix of observations on p independent variables, is a × 1 column of unknown parameters and is an × 1 column of residuals, with an expected value equal to zero and a variancecovariance matrix equal to 2 . When all the assumptions of the linear model in (1) have been satisfied, the ordinary least squares estimator, denoted (OLS), will be the best linear unbiased estimator for (1) and is given as follows: where S = X′X. The OLS estimator is not always be a good estimator when the multicollinearity is present; consequently, the goodness of the OLS estimator will be missed. Neter, (1) said that in the process of fitting regression model, when one independent variable is nearly combination of other independent variables and this will affect parameter estimates. Multicollinearity may cause serious difficulties.
Department of Mathematics, College of Education for Pure Sciences, University of Anbar, Anbar, Iraq. E-mail:eps.mustafa.ismaeel@uoanbar.edu.iq * ORCID ID:0000-0002-1684-7682 Variances of parameter estimates may be unreasonably large, parameter estimates may not be significant and a parameter estimate may have a sign different from what is expected.
Thus, the detection of multicollinearity has to be made to reduce the effect on the estimation. The measures most applied to detect multicollinearity are the Variance Inflator Factor (VIF) and the Condition Number (CN) and the researchers are still working for this subject (2).
To reduce the effect of this problem, the biased estimation technique has been developed. Therefore, many new biased estimators have been proposed, such as the RR estimator (3). From the theory that the combination of two different estimators might inherit the advantages of both estimators, Liu (4) combined the Stein estimator with the RR estimator and proposed the Liu estimator (LE) as follows: where 0 < d < In addition to the sample information, some exact or stochastic restrictions may be available for the unknown parameter of the model under consideration, then this will help to overcome the multicollinearity problem. Therefore, suppose that we have some prior information about in the form of stochastic linear restrictions as follows: where ℎ is a × 1 matrix that may be interpreted as a random vector with E(ℎ) = H , H is a × known matrix and V is assumed to be a known and positive definite (pd). Additionally, it is assumed that is stochastically independent of . Theil and Goldberger (7,8) introduced the mixed estimation technique by unifying the sample and the prior information in equation (4) in a common model. These authors introduced what they called the ordinary mixed estimator (OME) as follows: To improve the performance of OME estimator, Hu Yang and Jianwen Xu (9) introduced what they called the stochastic restricted Liu estimator by combining the OME and LE as follows: (10) provided a new alternative estimator, the stochastic restricted Liu-type estimator, which is obtained by combining the OME and Liu-type estimator in the following way: When the sample information given by (1) and the prior information presented by (4) are assigned as not equally important, Schffrin and Toutenburg (11) introduced the weighted mixed estimator (WME) as follows: where w is a nonstochastic scalar weight, with 0 ≤ ≤ 1. where ̂( , ) = −1 ( ′ +̂). We want to mention here, that there are many authors who are working in biased estimation methods in regression models with prior information or without, see, for example Özkale (17), Huang and Yang (18) , Kristofer M., Kibria G. B.M. and Shukur G. (19) and Özbay and Kaçıranlar (20).
As it can be observed, all the estimators in (5) to (10) are still dealing with the −1 matrix. Therefore, if there is severe multicollinearity, then the estimators will be obtained but with high variance. For this reason, the goal of this paper is to propose new types of stochastic-restricted Liu estimators in the case of the prior and the sample information being either equally important or not which does not deal with −1 only. This paper is organized as follows. In Section 2, the statistical model and the new weighted and non-weighted mixed Liu estimators are introduced. Then, in Section 3, the superiority of the proposed estimator compared with some related estimators is given, and we list some lemmas needed for the theoretical discussions. Finally, a numerical example and a simulation study are provided to illustrate some of the theoretical results in Section 5.

Model (15) combines the prior information in
The following fact gives another form of ̂( ) in (17). Lemma 1 (21): Suppose that the square matrices : × and : × are not singular, and let : × and : × be any two matrices. Then, where −1 ( ) = ( + ) −1 . We call this estimator the stochastic restricted Liu-type estimator (SRLTE). Case of the prior and sample information are not equally important As mentioned in section (2-1), the proposed estimator is as follows: where = ( + −1 ). It is called the weightedmixed Liu-type estimator (WMLTE). In fact, ̂( ) is a general estimator, which includes the OLS, LE and SRLTE estimators as special cases. This estimator is as follows.
If w=0 and d=1, then

The properties of the proposed estimators
The properties of the proposed estimator in (19) will be obtained, and then the estimator is generalized using the proposed estimator in (17), by setting w=1.
It is well known that the performance of any estimator ̂* for depends on its properties. Therefore, it is necessary to study the properties of the proposed estimator as well as those of other estimators.
The expected value and the variancecovariance matrices of the WME and WSLE estimators are given as follows: Additionally, the expected value and the variance -covariance matrix of the WMLTE are as follows: where = (S + + ′ −1 ) −1 . If we are dealing with the biased estimators, the mean squared error matrix is the best criterion that can provide good information about the performance of an estimator. This matrix can describe the variancecovariance matrix and the biased vector of an estimator simultaneously as follows: (̂ * ) = Cov (̂ * ) +Biased(β * ) Biased(β * )' Therefore, (̂ ) = 2 * (S + 2 ′ −1 ) * …(26)

Superiority of the Proposed Estimator
In this section, the superiority of ̂ (d) will be studied to the other estimators by using the mean squared error matrix. Before that, to clarify the discussion, a definition and some lemmas are listed. Definition: (22): Let A: n × n and B: n × n be any two matrices.
Then, the roots = ( ) of the equation | − | = 0 are called the eigenvalues of A in the metric B. It is clear from the above definition that the roots of ( ) are the usual eigenvalues of the matrix

Optimal biased parameter d for ̂ ( )
The least mean squared error of ̂w m ( ) can be obtained by finding the optimal value of the biased parameter d. Therefore, we have to find the d that achieves the desired performance. Model (1) can be written in the canonical form as follows: = + , where = , = ′ and P is a × orthogonal matrix, such that ′ ′ = Λ. Λ is a × diagonal matrix and its elements 1 , 2 , … . , are the eigenvalues of ′ , such that 1 > 2 > ⋯ > and (̂ (d)) = 2 (Λ + + ) −1 (Λ + Λ −1 + 2 )(Λ + + ) −1 +( − 1) 2 (Λ + + ) −1 ′(Λ + + ) −1 . To find the optimal value of the biased parameter d that minimizes (̂ ( )), let be fixed, minimize the trace of the (̂ (d)) as a function and calculate the derivative with respect to d as follows: Thus, After some simplifications, the optimal will be given as follows: . … (29) The optimal value of d in (26) depends on two unknown parameters, 2 and 2 . Therefore, these parameters are replaced with their unbiased estimators ̂2 and ̂2 to get the following (see (24)): (30)

Numerical Example and Simulation Study
In this section, the performance of the new estimator is explained compared to the other estimators (WME and WSLE) using the scalar mean squared error (mse). We use the dataset on Portland Cement originally attributed to Woods et al. (25), which several researchers used in their studies, including Hu Yang and Jianwen Xu (9), Hu Yang et al. (26). Our computations were all performed using Matlab R2010b. The following stochastic linear restriction is considered to improve estimator: ℎ = +e, H=(1, 1,1,0) and ~(0, 2 ) (see (27)     For further explanation regarding the behaviour of the new estimator, a Monte Carlo simulation experiment was performed. Following Kibria and Banik (24) and Hua Huang et al. (18) to achieve various degrees of collinearity, the explanatory variables are generated by using the following equation.
are independent standard normal pseudorandom numbers, p=4 is the number of the explanatory variables, n=100 and 500, and is specified so that the correlation between any two explanatory variables is given by 2  , where * is the estimator of the ith parameter in the jth replication and is the true parameter value.   Table 4 and Table 5 show that the WMLTE estimator is better than the other estimators for different values of correlation and for both cases (n=100 and n=500). This result supports the goal of this article for finding or improving an estimator that is more accurate compared to other estimators. It is clear that the new estimator is meaningful in practice.

Conclusion:
In this paper, a new version of the weighted-mixed Liu-type estimator is introduced for the vector of parameters in a linear regression model by unifying the sample and the prior information in the case that they either are equally or are not equally important. Furthermore, the new estimator is superior to the weighted-mixed estimator and the weighted-stochastic-restricted Liu-type estimator in the mean squared error matrix under certain conditions. The optimal value of the biased parameter for the proposed estimator are obtained. Finally, a numerical example and a simulation study are given for the comparison of the new estimator with other estimators in this study.