• Communications for Statistical Applications and Methods
• /
• v.22 no.2
• /
• pp.147-157
• /
• 2015

We consider a sparse high-dimensional linear regression model. Penalized methods using LASSO or non-convex penalties have been widely used for variable selection and estimation in high-dimensional regression models. In penalized regression, the selection and prediction performances depend on which penalty function is used. For example, it is known that LASSO has a good prediction performance but tends to select more variables than necessary. In this paper, we propose an additive sparse penalty for variable selection using a combination of LASSO and minimax concave penalties (MCP). The proposed penalty is designed for good properties of both LASSO and MCP.We develop an efficient algorithm to compute the proposed estimator by combining a concave convex procedure and coordinate descent algorithm. Numerical studies show that the proposed method has better selection and prediction performances compared to other penalized methods.

• Communications for Statistical Applications and Methods
• /
• v.15 no.1
• /
• pp.43-50
• /
• 2008

Multinomial logistic regression is a well known multiclass classification method in the field of statistical learning. More recently, the development of sparse multinomial logistic regression model has found application in microarray classification, where explicit identification of the most informative observations is of value. In this paper, we propose a sparse multinomial kernel logistic regression model, in which the sparsity arises from the use of a Laplacian prior and a fast exact algorithm is derived by employing a bound optimization approach. Experimental results are then presented to indicate the performance of the proposed procedure.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.11 no.7
• /
• pp.3594-3607
• /
• 2017

The traditional feature extraction methods such as principal component analysis (PCA) cannot obtain the local structure of the samples, and locally linear embedding (LLE) cannot obtain the global structure of the samples. However, a common drawback of existing PCA and LLE algorithm is that they cannot deal well with the sparse problem of the samples. Therefore, by integrating the globality of PCA and the locality of LLE with a sparse constraint, we developed an improved and unsupervised difference algorithm called Sparse Difference Embedding (SDE), for dimensionality reduction of high-dimensional data in small sample size problems. Significantly differing from the existing PCA and LLE algorithms, SDE seeks to find a set of perfect projections that can not only impact the locality of intraclass and maximize the globality of interclass, but can also simultaneously use the Lasso regression to obtain a sparse transformation matrix. This characteristic makes SDE more intuitive and more powerful than PCA and LLE. At last, the proposed algorithm was estimated through experiments using the Yale and AR face image databases and the USPS handwriting digital databases. The experimental results show that SDE outperforms PCA LLE and UDP attributed to its sparse discriminating characteristics, which also indicates that the SDE is an effective method for face recognition.

• Genomics & Informatics
• /
• v.20 no.4
• /
• pp.48.1-48.7
• /
• 2022

Penalized regression has been widely used in genome-wide association studies for joint analyses to find genetic associations. Among penalized regression models, the least absolute shrinkage and selection operator (Lasso) method effectively removes some coefficients from the model by shrinking them to zero. To handle group structures, such as genes and pathways, several modified Lasso penalties have been proposed, including group Lasso and sparse group Lasso. Group Lasso ensures sparsity at the level of pre-defined groups, eliminating unimportant groups. Sparse group Lasso performs group selection as in group Lasso, but also performs individual selection as in Lasso. While these sparse methods are useful in high-dimensional genetic studies, interpreting the results with many groups and coefficients is not straightforward. Lasso's results are often expressed as trace plots of regression coefficients. However, few studies have explored the systematic visualization of group information. In this study, we propose a multi-level polar Lasso (MP-Lasso) chart, which can effectively represent the results from group Lasso and sparse group Lasso analyses. An R package to draw MP-Lasso charts was developed. Through a real-world genetic data application, we demonstrated that our MP-Lasso chart package effectively visualizes the results of Lasso, group Lasso, and sparse group Lasso.

• Journal of Digital Contents Society
• /
• v.16 no.4
• /
• pp.605-613
• /
• 2015

This paper addresses the problem to make sparse the projection matrix in pattern recognition method. Recently, the size of computer program is often restricted in embedded systems. It is very often that developed programs include some constant data. For example, many pattern recognition programs use the projection matrix for dimension reduction. To improve the recognition performance, very high dimensional feature vectors are often extracted. In this case, the projection matrix can be very big. Recently, RSR(roated sparse regression) method[1] was proposed. This method has been proved one of the best algorithm that obtains the sparse matrix. We propose three methods to improve the RSR; outlier removal, sampling and elastic net RSR(E-RSR) in which the penalty term in RSR optimization function is replaced by that of the elastic net regression. The experimental results show that the proposed methods are very effective and improve the sparsity rate dramatically without sacrificing the recognition rate compared to the original RSR method.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.11 no.5
• /
• pp.2539-2554
• /
• 2017

Regression-based image super resolution (SR) methods have shown great advantage in time consumption while maintaining similar or improved quality performance compared to other learning-based methods. In this paper, we propose a novel single image SR method based on hierarchical regression to further improve the quality performance. As an improvement to other regression-based methods, we introduce a hierarchical scheme into the process of learning multiple regressors. First, training samples are grouped into different clusters according to their geometry similarity, which generates the structure layer. Then in each cluster, a compact dictionary can be learned by Sparse Coding (SC) method and the training samples can be further grouped by dictionary atoms to form the detail layer. Last, a series of projection matrixes, which anchored to dictionary atoms, can be learned by linear regression. Experiment results show that hierarchical scheme can lead to regression that is more precise. Our method achieves superior high quality results compared with several state-of-the-art methods.

• Journal of the Korean Statistical Society
• /
• v.27 no.4
• /
• pp.515-529
• /
• 1998

Burman (1987) and Hall and Titterington (1987) studied kernel smoothing for sparse multinomial data in detail. Both of their estimators for cell probabilities are sparse asymptotic consistent under some restrictive conditions on the true cell probabilities. Dong and Simonoff (1994) adopted boundary kernels to relieve the restrictive conditions. We propose a local linear kernel estimator which is popular in nonparametric regression to estimate cell probabilities. No boundary adjustment is necessary for this estimator since it adapts automatically to estimation at the boundaries. It is shown that our estimator attains the optimal rate of convergence in mean sum of squared error under sparseness. Some simulation results and a real data application are presented to see the performance of the estimator.

• Proceedings of the Korean Information Science Society Conference
• /
• 2011.06c
• /
• pp.373-376
• /
• 2011

얼굴 영상에서 구성요소(눈썹, 눈, 코, 입 등)의 존재에 따라 보는 사람의 얼굴 인식 정확도는 큰 영향을 받는다. 이는 인간의 뇌에서 얼굴 정보를 처리하는 과정은 얼굴 전체 영역 뿐만 아니라, 부분적인 얼굴 구성요소의 특징들도 고려함을 말한다. 비음수 행렬 분해(NMF: Non-negative Matrix Factorization)는 이러한 얼굴 영역에서 부분적인 특징들을 잘 표현하는 기저영상들을 찾아내는데 효과적임을 보여주었으나, 각 기저영상들의 중요도는 알 수 없었다. 본 논문에서는 NMF로 찾아진 기저영상들에 대응되는 인코딩 정보를 SLR(Sparse Logistic Regression)을 이용하여 성별 인식에 중요한 부분 영역들을 찾고자 한다. 실험에서는 주성분분석(PCA)과 비교를 통해 NMF를 이용한 기저영상 및 특징 벡터 추출이 좋은 성능을 보여주고, 대표적 이진 분류 알고리즘인 SVM(Support Vector Machine)과 비교를 통해 SLR을 이용한 특징 벡터 선택이 나은 성능을 보여줌을 확인하였다. 또한 SLR로 확인된 각 기저영상에 대한 가중치를 통하여 인식 과정에서 중요한 얼굴 영역들을 확인할 수 있다.

• The Korean Journal of Applied Statistics
• /
• v.20 no.3
• /
• pp.561-571
• /
• 2007

Local linear regression estimator is the most widely used nonparametric regression estimator which has a number of advantages over the traditional kernel estimators. It is well known that local linear estimator can produce erratic result in sparse regions in the realization of the design and the interpolation method of Hall and Turlach (1997) is the very efficient way to resolve this problem. However, it has been never pointed out that Hall and Turlach's interpolation method is very sensitive to outliers. In this paper, we propose the robust version of the interpolation method for adapting to sparse design. The finite sample properties of the method is compared with Hall and Turlach's method by the simulation study.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.18 no.12
• /
• pp.2946-2952
• /
• 2014

Among the Example based Super Resolution(SR) techniques, Neighbor embedding(NE) has been inspired by manifold learning method, particularly locally linear embedding. However, the poor generalization of NE decreases the performance of such algorithm. The sizes of local training sets are always too small to improve the performance of NE. We propose the Learning Sparse-Neighbor Image Representation baesd on SVR having an excellent generalization ability to solve this problem. Given a low resolution image, we first use bicubic interpolation to synthesize its high resolution version. We extract the patches from this synthesized image and determine whether each patch corresponds to regions with high or low spatial frequencies. After the weight of each patch is obtained by our method, we used to learn separate SVR models. Finally, we update the pixel values using the previously learned SVRs. Through experimental results, we quantitatively and qualitatively confirm the improved results of the proposed algorithm when comparing with conventional interpolation methods and NE.