DOI QR코드

DOI QR Code

2D-MELPP: A two dimensional matrix exponential based extension of locality preserving projections for dimensional reduction

  • Xiong, Zixun (School of Information Engineering, Nanjing Audit University) ;
  • Wan, Minghua (School of Information Engineering, Nanjing Audit University) ;
  • Xue, Rui (School of Electronics and Information Engineering, Beihang University) ;
  • Yang, Guowei (School of Information Engineering, Nanjing Audit University)
  • Received : 2020.12.15
  • Accepted : 2021.08.30
  • Published : 2022.09.30

Abstract

Two dimensional locality preserving projections (2D-LPP) is an improved algorithm of 2D image to solve the small sample size (SSS) problems which locality preserving projections (LPP) meets. It's able to find the low dimension manifold mapping that not only preserves local information but also detects manifold embedded in original data spaces. However, 2D-LPP is simple and elegant. So, inspired by the comparison experiments between two dimensional linear discriminant analysis (2D-LDA) and linear discriminant analysis (LDA) which indicated that matrix based methods don't always perform better even when training samples are limited, we surmise 2D-LPP may meet the same limitation as 2D-LDA and propose a novel matrix exponential method to enhance the performance of 2D-LPP. 2D-MELPP is equivalent to employing distance diffusion mapping to transform original images into a new space, and margins between labels are broadened, which is beneficial for solving classification problems. Nonetheless, the computational time complexity of 2D-MELPP is extremely high. In this paper, we replace some of matrix multiplications with multiple multiplications to save the memory cost and provide an efficient way for solving 2D-MELPP. We test it on public databases: random 3D data set, ORL, AR face database and Polyu Palmprint database and compare it with other 2D methods like 2D-LDA, 2D-LPP and 1D methods like LPP and exponential locality preserving projections (ELPP), finding it outperforms than others in recognition accuracy. We also compare different dimensions of projection vector and record the cost time on the ORL, AR face database and Polyu Palmprint database. The experiment results above proves that our advanced algorithm has a better performance on 3 independent public databases.

Keywords

1. Introduction

In pattern recognition, massive high dimensional image inputs are computationally challenging to analysis[1-6]. We hope to retain as much information as possible while reducing data dimensions to improve the accuracy and efficiency of later processing such as inference [7-10]. Generally, dimensionality reduction methods can be classified into two classes: feature extraction and feature selection. The former is focused on creating new features and the later is focused on selecting most relevant subsets. And there are two feature extractions classifications: (1)Nonlinear methods and (2)Linear methods. Nonlinear ones like locality linear embedding(LLE)[11], Laplacian egienmaps(LE)[12] pay attention to preserving local structure, while other methods like Isomap[13] is focused on global structure. It also tries to maintain geodesic distances between different samples. Linear methods, on the other hand, has become extremely important for itssimple and computationally efficient classification strategies. Principal component analysis(PCA)[14-15] as unsupervised method and linear discriminant analysis(LDA)[16-17] as supervised method are two representative linear methos dimensionality reduction. locality preserving projections(LPP)[18], which can be seen as linearization of LE, is an alternative to PCA. LPP is obtained during the process of searching for the optimized approximate value of eigenfunctions of Laplace Beltrami operator on the manifold when there is a low dimensional manifold embedded in relatively high dimensional spaces. It needs to point out that LPP possesses two non linear advantages while former linear algorithms don’t:

1. New minimal criterion preserves neighborhood structure, considering that Euclidean distances are meaningful only when they are local. It makes sense particularly when data lies on nonlinear manifold embedded in the ambient space.

2. For the ‘neighborhood structure preserved’ advantage above, LPP is also suitable for information retrieval applications

It is also very important that LPP is defined on more than just training data points and thus can be simply applied to new data points compared with traditional non-linear methods.

Two dimensional principal component analysis(2D-PCA)[19] saves the trouble of transforming images into one dimensional vectors and it inspires subsequent matrix based algorithm. And two dimensional linear discriminant(2D-LDA)[20] analysis extracts proper features directly from images based on Fisher’s linear discriminant analysis. In the process especially like image recognition, the data is usually insufficient and having high dimensions causing singularity problems which make algorithms (e.x. LPP) unable to be applied directly. Two dimensional locality preserving projections (2D-LPP) is therefore proposed. It works on 2D images directly. Two dimensional locality preserving projections(2D-LPP)[21-23]perfectly solves the singularity that traditional LPP faced and is proved to perform better on face recognition and palmprint recognition , it turns out to be faster than LPP and achieves higher recognition rates.

Matrix based exponential method is also used to solve such small sample sized problem (SSS problems). ELDE[24] is based on local discriminant embedding (LDE)[25], which was put forward to overcome the limits that global linear discriminant analysis method meets. Matrix based exponential method also helps to improve the flaw of LPP’s sensitivity to neighborhood sized k[26] and fix singularity problems.

According to[27], 2D methods don’t necessarily perform better than 1D methods. It implies the limitations that 2D method meets and the direction for feature work. 2D-LPP is simple and efficient, but it don’t perform well when training samples are limited and it seems to meet limitations in some public databases. We apply matrix exponential methods on 2D-LPP to improve its behavior and we get its enhanced version--2D-MELPP. However, the computational complexity of 2D-MELPP is extremely high, so we replace some of matrix multiplications with multiple multiplications to get a more efficient way for solving 2D-MELPP. We also use a more efficient way to calculate matrix exponential to save time. The new 2D-MELPP has high memory costs, so we replace some matrix multiplications with multiple multiplications and we investigate the enhanced version on a random 3D data set, ORL, AR Face Database and Polyu Palprint Database, 2D-PCA,2D-LDA and 2D-LPP are also tested.

The remainder of paper is arranged as follows: The mainly content is about reference to related works in Section 2. Section 3 is the retrospect of innovative work in this paper. Section 4 is composed of 4 experiments designed to show the efficiency and accuracy of our new algorithm. Section 5 is separated into two parts, respectively describing conclusions of experiments in section 4 and the direction of our feature work.

2. Related Work

Our work is based on 2D-LPP and matrix exponential methods, which both help to solve SSS problems and improve recognition accuracy.

2.1 2D-LPP method

2D-LPP applies the projection for each data as follows:

xi = Aiw(i = 1, 2, 3...n)       (1)

x ∈ Rr×d Aj ∈ Rr×c w ∈ Rc×l

where r, c are row and column dimension of Ai respectively. w is the target projection matrix a, while Ai is the ith source image for testing, and n is the number of images in the datasets. It chooses[21] the criterion as :

\(\begin{aligned}\min \sum_{i, j} S_{i j}\left\|X_{i}-X_{j}\right\|^{2}(i, j=1,2,3 \ldots)\end{aligned}\)       (2)

is generated as : Sij

(1). First, we have to construct adjacency matrix G ∈ Rmxn. For every Gij both k-nearest neighbors and ε-nearest neighbors are appropriate. When doing the experiment in this paper, we choose the former and add label Information;

(2). For each Gij not equal to zero, we suppose ith and jth image are “close”;

(3). For all “close” ones, we apply the Heat Kernel method to build the S matrix.

Looking back to the origin criterion, it can be written as:

\(\begin{aligned}\min \sum_{i, j}\left\|A_{i} W-A_{j} W\right\|^{2}(i, j=1,2,3 \ldots)\end{aligned}\)       (3)

where the target function can be simplified[23] as

minwT AT(L⊗In)Aw       (4)

where A is in the form like [A1T, A2T ...AnT]T, Dii is column sum of S, In is a n×n identity matrix and ⊗ is kronecker product.

To eliminate an arbitrary scaling element of the process, 2D-LPP dispose it as followed :

\(\begin{aligned}\sum_{i} D_{i j} X_{i}^{T} X_{i}=1\end{aligned}\)        (5)

which can also be presented as:

WT AT(D⊗In)Aw = 1       (6)

And the whole minimization problem is transformed into a classical generalized eigenvalue problem as follows:

AT(L⊗In)Aw = λAT(D⊗In)Aw       (7)

So we can get final target projection matrix w.

2.2. Matrix exponential Process

In the sub-section, we will briefly look into the definition and some basic properties of matrix exponential process for the prerequisite of 2D-MELPP:

The Matrix exponential for M(M ∈ Rmxn) is defined as follows:

\(\begin{aligned}\exp (M)=1+M+\frac{M^{2}}{2 !}+\ldots+\frac{M^{m}}{m !}+\ldots\end{aligned}\)        (8)

where 1 can be treated as an n × n identity matrix, the matrix exponential has properties as followed:

(1). exp(M) is the sum of a sequence of matrixes with finite numbers;

(2). exp(M) is a full rank matrix;

(3). For arbitrary square matrix M, there exists the inverse of its matrix exponential;

(4). Supposing that R is a nonsingular matrix, we can have:

exp(R-1MR ) = R-1exp(M)R

(5). For every eigenvectors of M(v1, v2...vn) corresponding to α1, α2..., αn there exists eα1, eα2..., eαn as eigenvalues of exp(M) having the same eigenvectors just like M.

3. 2D-Matrix exponential based discriminate locality preserving projection(2D-MELPP)

3.1. Notation used in 2D-MELPP

3.1.png 이미지

3.2. 2D-MELPP without further improvements

In 2D-MELPP, the objective function is:

minwT AT exp(L⊗In)Aw       (9)

To eliminate an arbitrary scaling element of the process, 2D-LPP dispose it as followed :

wT AT exp(D⊗In)Aw =1       (10)

With Matrix based exponential D⊗L operator included accordingly.

Then the whole optimal question is equal to the eigenvalue problem as follows :

exp(AT)(L⊗In)A)w = λexp(AT(D⊗In)A)w       (11)

Referring to the properties (5) in section B, it is mathematically similar and can solve SSS problem in classical eigenvalue questions for (4) in section B.

According to[26], it can also emphasize the geometry features by the same time.

3.3. Efficient procedure to solve 2D-MELPP

The computational complexities of 2D-MELPP is computed exp(AT(L⊗In)A) and O(r3) respectively. A paper[28] meets the same problem as working with one dimension framework as follows:

w = argmin(wT exp(XDXT)w)-1(wT exp(XLXT)w)       (12)

To solve this, we can apply the similar strategy.

As what it’s shown in the part followed, an efficient procedure formula is build to solve 2D-MELPP.

First, we assume the SVD decomposition of A is

A = PΣVT       (13)

Let Sa = PT exp(AT(L⊗Im)A)P, Sb = PT exp(AT(D⊗Im)A)P, suppose that the eigenvector is u and λ is the eigenvalue.

Then we obtain:

exp(AT(D⊗In)A)Pu = λexp(AT(L⊗In)A)Pu       (14)

According to (4) the computational complexity is reduced. However, the computational complexity of computing exp(A(D⊗In)A) and exp(AT(L⊗In)A) is still O(d3). Then, out target has changed into compute exp(AT(D⊗In)A) and exp(AT(L⊗In)A) efficiently.

The eigen-decomposition of H = D⊗In is supposed :

H = QHΣHQHT       (15)

Let MH = ATQHΣH0.5, and its economic decomposition is

MH = QMΣMVMT       (16)

Then we obtain:

AT HA = QMΣM2QMT       (17)

Let column orthogonal matrix QMτ be that [QM, QMτ] is orthogonal.

According to the definition, we can have that:

exp(AT HA) = [QM, QMτ][QM, QMτ]T + ...       (18)

\(\begin{aligned}+\frac{\left[Q_{M}, Q_{M}^{\tau}\right]\left[\begin{array}{cc}\Sigma_{M}^{2} & O_{r_{M} \times\left(r-r_{M}\right)} \\ O_{\left(r-r_{M}\right) \times r_{M}} & O_{\left(r-r_{M}\right) \times\left(r-r_{M}\right)}\end{array}\right]}{m !}+\ldots\end{aligned}\)

\(\begin{aligned}=Q_{M} \exp \left(\Sigma_{M}^{\tau}\right) Q_{M}^{T}+1-Q_{M} Q_{M}^{T}\end{aligned}\)

where Qc×d are zeros matrices, c,d are its row and column numbers respectively. And rM is the dimension of ΣM.

Then, we have:

PT exp(AT HA)P = PTexp(ΣH2)QMTP + I - PTQMQMTP       (19)

Similarly, the eigen-decomposition of L = L⊗In is assumed:

L = QLΣLQLT       (20)

Let XL= VTQLΣL0.5, and its economic decomposition is:

XL = QXΣXVXT       (21)

Then we obtain:

AT LA=PQXΣX2QXTPT       (22)

Let column orthogonal matrix PMτ be that [PQX, PXτ] is orthogonal. Then we have:

\(\begin{aligned}\begin{array}{l}\exp \left(A^{T} L A\right)=\left[P Q_{X}, P Q_{X}^{\tau}\right]\left[P Q_{X}, P Q_{X}^{\tau}\right]^{T}+ \\ \ldots+\frac{\left[P Q_{X}, P Q_{X}^{\tau}\right]\left[\begin{array}{cc}\Sigma_{M}^{2 m} & O_{r_{x} \times\left(d-r_{x}\right)} \\ O_{\left(r-r_{x}\right) \times r_{x}} & O_{\left(r-r_{x}\right) \times\left(r-r_{x}\right)}\end{array}\right]}{m !} \\\end{array}\end{aligned}\)

\(=P Q_{X} \exp \left(\Sigma_{X}^{2}\right) Q_{X}^{T} P^{T}+P_{X}^{\tau}\left(P_{X}^{\tau}\right)^{T}\)       (23)

Then we have:

PT exp(AT LA)P = QX exp(ΣX2)QXT + I - QXQXT       (24)

Now we have the following new process to solve 2D-MELPP:

Algorithm :2D-MELPP

Input: sample A

Output: The projected matrix of G

(1). Constructing the matrix A , H = exp(AT DA), L = exp(AT LA) just like normal 2D-LPP;

(2). Calculating the SVD of A as A = PΣVT and the matrix H as H = QHΣHQHT ;

(3). Compute the matrix MH = ATQHΣH0.5and its economic SVD as MH= QMΣMVMT ;

(4). Compute the eigen-decomposition of L as L = QLΣLQLT ;

(5). Calculate the matrices XL = VTQLΣL0.5 ,and its economic decomposition is XL = QXΣXVXT ;

(6). Compute the matrix

J = PT exp(AT HA)P = PT exp(ΣH2)QMTPT + PTQMQMTP

U = PT exp(AT HA)P = QX exp(ΣX2)QXT+ I - QXQXTrespectively;

(7). Solve the eigenvalue problem Jui = λiUui ;

(8). Let G = PUk where Uk is the matrix combined of k eigenvectors corresponding to k largest eigenvalues, and then orthogonalize the projection matrix and get the G we wanted(k is a constant value) .

Then the time complexity is reduced to O(rn2).

4. Experimental results

In the experiment, we will evaluate the performance of 2D-MELPP with PCA+LPP, 2D-PCA, ELPP, 2D-LDA, and 2D-LPP on the random 3D data set and three different public image databases. We pick up KNN as the classifier for three image databases. Four experimental results are completed in the same experimental environment (CPU: 2.04 GHz, RAM: 3.9GB).

4.1 The Random 3D data set

In the first experiment, we will propose 2D-MELPP and other methods on a synthetic 3D matrix M. The data set is made up of 20 matrices, each of which is generated by a 20 by 20 matrix having 400 points subject to normal distribution. And the data set is divided into 2 class as follows(The data set is normalized) :

\(\begin{aligned}\operatorname{class}\left(A_{k i j}\right)=\left\{\begin{array}{ll}\text { class }-a & A_{k i j} \geq 0.999 \\ \text { class }-b & A_{k i j} \leq 0.001\end{array}\right.\end{aligned}\)       (25)

where Akij is value of ith row and jth column of the kth matrix, (0 ≤ i, j, k ≤ 20), class(Akij) is the class of Akij and the class is either class-a or class-b. The distribution of two classes can be seen in Fig. 1.

E1KOBZ_2022_v16n9_2991_f0001.png 이미지

Fig. 1. The class distribution of 3D data set. Each point is represented by(xi, yi, zj) in Cartesian coordinates. Variable i ranges from 0 to 20. Two classes are marked by blue and red respectively. Each point is assigned with random values ranging from 0 to 1 and the rules to decide its class are shown in (25).

We test PCA+LPP, ELPP, 2D-PCA, 2D-LDA, 2D-LPP and 2D-MELPP on this random data set. And FIGURE 2 shows the result of dimension reduction with PCA+LPP, 2D-PCA, ELPP, 2D-LDA, 2D-LPP and 2D-MELPP (from left to right, up to down).

As can be seen from Fig. 2, 2D-MELPP provide projected data with excellent linear separability, while other methods fail to do so.

E1KOBZ_2022_v16n9_2991_f0002.png 이미지

Fig. 2. The projected 2D data set with (a) PCA+LPP, (b) ELPP, (c) 2D-PCA, (d) 2D-LDA, (e) 2D-LPP and (f) 2D-MELPP (from left to right, up to down). The X axis is the reduced dimension, Y is the serial number of the picture.

4.2 The ORL face database

The ORL face image database consists of 400 face images with 40 classes, each having 10 samples within. The background were all set in homogeneous darkness, and the images are featured by different light intensities, facial expressions and facial details(glassed and not) as it can be shown in Fig. 3.

E1KOBZ_2022_v16n9_2991_f0003.png 이미지

Fig. 3. The images of one person from ORL face database

We range dimensions of project vector from 1 to 20 in Table 1 and pick one with highest recognition accuracy each(from 2 to 6 training samples each class). Fig. 4 shows recognition accuracy varies under 6 samples with the changes of dimensions. In Table 2, we record the time cost on each algorithm under 6 samples on the ORL face database.

Table 1. highest mean value of rECOGNITION ACCURACY (IN PERCENT) ON ORL database. There is a one-to-one correspondence between colunms and fixed training numbers. values in each PARENTHESIS ranges between 1 and 20, they represent for corresponding projected matrix dimensions.

E1KOBZ_2022_v16n9_2991_t0001.png 이미지

E1KOBZ_2022_v16n9_2991_f0004.png 이미지

Fig. 4. The recognition accuracy with 6 samples per class on the ORL face database

Table 2. AVERAGE cost time(s) of PCA+LPP, ELPP, 2D-PCA, 2D-LDA, 2D-LPP and 2D-MELPP on The ON ORL FACE SET. EACH COLUMN CORRESPONDS TO A FIXED ALGORITHM.And THE 2D-MELPP IS THE IMPROVED VERSION.

E1KOBZ_2022_v16n9_2991_t0004.png 이미지

4.3 The AR face database

AR face database is made up of face images of 120 people with 26 images of each. In this experiment we take 10 each which don’t have too many occlusion problems while testing algorithm on a lager scale than ORL in Fig. 5.

E1KOBZ_2022_v16n9_2991_f0005.png 이미지

Fig. 5. 20 image samples of one person form AR database. Pictures in second line were token 2 weeks after the first line

We range dimensions of project vector from 1 to 20 in Table 3 and pick one with highest recognition accuracy each(from 2 to 6 training samples each class). Fig. 6 shows recognition accuracy varies under 6 samples per class with the changes of dimensions.

E1KOBZ_2022_v16n9_2991_f0006.png 이미지

Fig. 6. The recognition accuracy with 6 samples per class on the AR face database

Table 3. highest mean values of rECOGNITION ACCURACY (IN PERCENT) ON Ar database. There is a one-to-one correspondence between colunms and fixed training numbers. values in each PARENTHESIS ranges between 1 and 20, they represent for corresponding projected matrix dimensions.

E1KOBZ_2022_v16n9_2991_t0002.png 이미지

In Table 4, we record the time cost on each algorithm under 6 samples on the AR face database.

Table 4. AVERAGE cost time(s) of PCA+LPP, ELPP, 2D-PCA, 2D-LDA, 2D-LPP and 2D-MELPP on The ON ar FACE SET. EACH COLUMN CORRESPONDS TO A FIXED ALGORITHM.And THE 2D-MELPP IS THE IMPROVED VERSION.

E1KOBZ_2022_v16n9_2991_t0005.png 이미지

4.4. The Polyu palmprint database

Polyu Palmprint Database consists of 600 palmprint image with 100 classes, each having 6 samples within. Fig. 7 shows the background were also set in homogeneous darkness and the images are featured by different light intensities. Half of the original image is taken 2 month later than the other half, and we set the training and testing division according to that.

E1KOBZ_2022_v16n9_2991_f0007.png 이미지

Fig. 7. The palmprint samples from Polyu palmprint database

Fig. 8 shows recognition accuracy varies under 3 samples with the changes of dimensions ranging from 1 to 50, and Table 5 shows highest recognition accuracy of each algorithm, Table 6 records the time cost with 50 dimensions.

Table 5. highest mean value of rECOGNITION ACCURACY (IN PERCENT) ON polyu database. There is a one-to-one correspondence between colunms and methods.

E1KOBZ_2022_v16n9_2991_t0003.png 이미지

E1KOBZ_2022_v16n9_2991_f0008.png 이미지

Fig. 8. recognition accuracy with 3 samples per class on Polyu Palmprint database

Table 6. AVERAGE cost time(s) of PCA+LPP, ELPP, 2D-PCA, 2D-LDA, 2D-LPP and 2D-MELPP on PolyU palmprint database. EACH COLUMN CORRESPONDS TO A FIXED ALGORITHM.And THE 2D-MELPP IS THE IMPROVED VERSION.

E1KOBZ_2022_v16n9_2991_t0006.png 이미지

From results of the experiment above, we can make observations:

(1). According to Fig.s 2, 4 and 6, provided with identical experimental conditions, the recognition accuracy of the advanced algorithm has better performances. The ability of the algorithm to preserve more discrimination than others may account for it.

(2). Generally, 2D-MELPP achieves higher recognition accuracy than other algorithms with small projection dimensions as shown in Table 1, 3, 5.

(3). 2D-MELPP takes less time than ELPP in tree databases, which provides supports for our development on computational complexities does reduce the it. It can be seen from Table 2, 4, 6.

5. Conclusion

5.1. Results and advanced nature of experiments

We are inspired by the idea of ELPP and comparison works between 2D-LDA and LDA to employee matrix exponential on 2D-LPP and work algebra procedure to promote its behavior both in theory and in test on random 3D data set, two public face databases and one palmprint database. We also develop a way to save memory cost for solving 2D-MELPP. Our advanced 2D-MELPP performs better in recognition accuracy and it cost less time than ELPP.

2D-MELPP is an efficient solution for the SSS problem elegantly since the matrix exponential of a symmetric matrix is positive definite all the time. Since 2D-MELPP is based directly on image matrix and the matrix exponential process has distances enlarged, it can preserve more information and achieve higher accuracy.

5.2. Spectrum of future work

Although it’s easy to apply linear methods for dimensionality reduction, it’s nonetheless hard to attain linear data in real world feature extraction application. In view of above-mentioned reason, improvements can be made with nolinear methods or struct based methods. We will verify them in future work.

Acknowledgement

This work is partially supported by the National Science Foundation of China under Grant Nos. 61876213, 62172229, 6177227, 61976117, 71972102, 61861033, 61976118, the Key R&D Program Science Foundation in Colleges and Universities of Jiangshu Province Grant Nos.18KJA520005, 19KJA360001, 20KJA520002, the Natural Science Fund of Jiangsu Province under Grants Nos. BK20201397, BK20211295, BK20221349. Jiangsu Key Laboratory of Image and Video Understanding for Social Safety of Nanjing University of Science and Technology under Grants J2021-4. Funded by the Qing Lan Project of Jiangsu University, Funded by the Future Network Scientific Research Fund Project SRFP-2021-YB-25, and China’s Jiangxi Province Natural Science Foundation (No. 20202ACBL202007). This work is funded in part by the “Qinglan Project” of Jiangsu Universities under Grant D202062032.

References

  1. Jia, Y. T., Law, S. S., & Yang, N., "Response and reliability analysis of a high-dimensional stochastic system," Probabilistic Engineering Mechanics, vol. 61, 2020.
  2. Wei Z, Guo W, Li B, et al, "High-dimensional metric combining for non-coherent molecular signal detection," IEEE Transactions on Communications, Vol.68, No. 3, pp.1479-1493, 2019. https://doi.org/10.1109/tcomm.2019.2959354
  3. Pevny, Tomas, Filler, Tomas, & Bas, Patrick, "Using high-dimensional image models to perform highly undetectable steganography," in Proc. of International Workshop on Information Hiding, pp.161-177, 2010.
  4. Jia, Y. T., Law, S. S., & Yang, N., "Response and reliability analysis of a high-dimensional stochastic system," Probabilistic Engineering Mechanics, 103077, 2020.
  5. Bin, Gu, Victor, S, Sheng, & Keng, et al, "Incremental support vector learning for ordinal regression," IEEE Transactions on Neural Networks & Learning Systems, vol. 26, no. 7, pp. 1403-1416, 2015. https://doi.org/10.1109/TNNLS.2014.2342533
  6. Gu, B., Sun, X., & Sheng, V. S., "Structural minimax probability machine," IEEE Transactions on Neural Networks & Learning Systems, Vol.28, No.7, pp.1646-1656, 2017. https://doi.org/10.1109/TNNLS.2016.2544779
  7. Laurens Van Der Maaten, Eric Postma, & Jaap Van Den Herik, "Dimensionality reduction: a comparative review," Review Literature & Arts of the Americas, Vol.10, No.1, 2009.
  8. Chapelle, O., Bernhard Scholkopf, & Zien, A., "Spectral Methods for Dimensionality Reduction," Semi-Supervised Learning, MIT Press.
  9. Yan, S., Xu, D., Zhang, B., Zhang, H. J., Yang, Q., & Lin, S., "Graph embedding and extensions: a general framework for dimensionality reduction," IEEE Transactions on Pattern Analysis & Machine Intelligence, Vol.29, pp. 40-51, 2007. https://doi.org/10.1109/TPAMI.2007.250598
  10. Zhang, T., Tao, D., Li, X., & Yang, J., "Patch alignment for dimensionality reduction," IEEE Transactions on Knowledge & Data Engineering, Vol.21, No.9, pp.1299-1313, 2009. https://doi.org/10.1109/TKDE.2008.212
  11. Roweis, Sam, T., Saul, Lawrence, & K, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, n0, 5500, pp. 2323-2326, 2000. https://doi.org/10.1126/science.290.5500.2323
  12. Belkin, M., & Niyogi, P., "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Computation, vol. 15, no. 6, pp. 1373-1396, 2003. https://doi.org/10.1162/089976603321780317
  13. J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000. https://doi.org/10.1126/science.290.5500.2319
  14. Turk, M., "Eigenfaces for recognition," J. Cognitive Neuroscience, 3(1), 71-86, 1991. https://doi.org/10.1162/jocn.1991.3.1.71
  15. Kirby, M., & Sirovich, L., "Application of the karhunen-loeve procedure for the characterization of human faces," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.12, No.1, pp.103-108, 1990. https://doi.org/10.1109/34.41390
  16. Swets, D. L., & Weng, J. J., "Using discriminant eigenfeatures for image retrieval," IEEE Transactions on Pattern Analysis & Machine Intelligence, Vol.18, No.8, pp.831-836, 1996. https://doi.org/10.1109/34.531802
  17. Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J., "Eigenfaces vs. fisherfaces: recognition using class specific linear projection," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.19, No.7, pp.711-720, 1997. https://doi.org/10.1109/34.598228
  18. He, X., "Locality preserving projections," Advances in Neural Information Processing Systems, Vol.16, No.1, pp.186-197, 2003.
  19. Yang, J., Zhang, D. D., Frangi, A. F., & Yang, J. Y., "Two-dimensional pca: a new approach to appearance-based face representation and recognition," IEEE Trans Pattern Anal Mach Intell, Vol.26, No.1, pp.131-137, 2004. https://doi.org/10.1109/TPAMI.2004.1261097
  20. Li, M., & Yuan, B., "2d-lda: a statistical linear discriminant analysis for image matrix," Pattern Recognition Letters, Vol.26, No.5, pp.527-532, 2005. https://doi.org/10.1016/j.patrec.2004.09.007
  21. Chen, S., Zhao, H., Kong, M., & Luo, B., "2d-lpp: a two-dimensional extension of locality preserving projections," Neurocomputing, Vol.70, No.4-6, pp.912-921, 2007. https://doi.org/10.1016/j.neucom.2006.10.032
  22. Zhi, R., & Ruan, Q., "Facial expression recognition based on two-dimensional discriminant locality preserving projections," Neurocomputing, No.71, Vol.7-9, pp.1730-1734, 2008. https://doi.org/10.1016/j.neucom.2007.12.002
  23. Hu, D., Feng, G., & Zhou, Z., "Two-dimensional locality preserving projections (2dlpp) with its application to palmprint recognition," Pattern Recognition, Vol.40, No.1, pp.339-342, 2007. https://doi.org/10.1016/j.patcog.2006.06.022
  24. Ran, R., Fang, B., Xuegang, W. U., & Zhang, S., "A simple and effective generalization of exponential matrix discriminant analysis and its application to face recognition," ICE Transactions on Information and Systems, E101.D(1), pp.265-268, 2018. https://doi.org/10.1587/transinf.2017EDL8198
  25. Chen, H. T., Chang, H. W., & Liu, T. L., "Local discriminant embedding and its variants," in Proc. of CVPR, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp.846-853, 2005.
  26. Wang, S. J., Chen, H. L., Peng, X. J., & Zhou, C. G., "Exponential locality preserving projections for small sample size problem," Neurocomputing, Vol.74, No.17, pp.3654-3662, 2011. https://doi.org/10.1016/j.neucom.2011.07.007
  27. Zheng, W. S., Lai, J. H., & Li, S. Z., "1d-lda vs. 2d-lda: when is vector-based linear discriminant analysis better than matrix-based?," Pattern Recognition, 41(7), 2156-2172, 2008. https://doi.org/10.1016/j.patcog.2007.11.025
  28. Lu, G. F., Wang, Y., Zou, J., & Wang, Z., "Matrix exponential based discriminant locality preserving projections for feature extraction," Neural Networks, vol. 97, pp.127-136, 2018. https://doi.org/10.1016/j.neunet.2017.09.014