• Title/Summary/Keyword: sparsity

Search Result 328, Processing Time 0.033 seconds

The Effect of Data Sparsity on Prediction Accuracy in Recommender System (추천시스템의 희소성이 예측 정확도에 미치는 영향에 관한 연구)

  • Kim, Sun-Ok;Lee, Seok-Jun
    • Journal of Internet Computing and Services
    • /
    • v.8 no.6
    • /
    • pp.95-102
    • /
    • 2007
  • Recommender System based on the Collaborative Filtering has a problem of trust of the prediction accuracy because of its problem of sparsity. If the sparsity of a preference value is large, it causes a problem on a process of a choice of neighbors and also lowers the prediction accuracy. In this article, a change of MAE based on the sparsity is studied, groups are classified by sparsity and then, the significant difference among MAEs of classified groups is analyzed. To improve the accuracy of prediction among groups by the problem of sparsity, We studied the improvement of an accurate prediction for recommending system through reducing sparsity by sorting sparsity items, and replacing the average preference among them that has a lot of respondents with the preference evaluation value.

  • PDF

Sparsity Effect on Collaborative Filtering-based Personalized Recommendation (협업 필터링 기반 개인화 추천에서의 평가자료의 희소 정도의 영향)

  • Kim, Jong-Woo;Bae, Se-Jin;Lee, Hong-Joo
    • Asia pacific journal of information systems
    • /
    • v.14 no.2
    • /
    • pp.131-149
    • /
    • 2004
  • Collaborative filtering is one of popular techniques for personalized recommendation in e-commerce sites. An advantage of collaborative filtering is that the technique can work with sparse evaluation data to predict preference scores of new alternative contents or advertisements. There is, however, no in-depth study about the sparsity effect of customer's evaluation data to the performance of recommendation. In this study, we investigate the sparsity effect and hybrid usages of customers' evaluation data and purchase data using an experiment result. The result of the analysis shows that the performance of recommendation decreases monotonically as the sparsity increases, and also the hybrid usage of two different types of data; customers' evaluation data and purchase data helps to increase the performance of recommendation in sparsity situation.

Data Sparsity and Performance in Collaborative Filtering-based Recommendation

  • Kim Jong-Woo;Lee Hong-Joo
    • Management Science and Financial Engineering
    • /
    • v.11 no.3
    • /
    • pp.19-45
    • /
    • 2005
  • Collaborative filtering is one of the most common methods that e-commerce sites and Internet information services use to personalize recommendations. Collaborative filtering has the advantage of being able to use even sparse evaluation data to predict preference scores for new products. To date, however, no in-depth investigation has been conducted on how the data sparsity effect in customers' evaluation data affects collaborative filtering-based recommendation performance. In this study, we analyzed the sparsity effect and used a hybrid method based on customers' evaluations and purchases collected from an online bookstore. Results indicated that recommendation performance decreased monotonically as sparsity increased, and that performance was more sensitive to sparsity in evaluation data rather than in purchase data. Results also indicated that the hybrid use of two different types of data (customers' evaluations and purchases) helped to improve the recommendation performance when evaluation data were highly sparse.

Multiview-based Spectral Weighted and Low-Rank for Row-sparsity Hyperspectral Unmixing

  • Zhang, Shuaiyang;Hua, Wenshen;Liu, Jie;Li, Gang;Wang, Qianghui
    • Current Optics and Photonics
    • /
    • v.5 no.4
    • /
    • pp.431-443
    • /
    • 2021
  • Sparse unmixing has been proven to be an effective method for hyperspectral unmixing. Hyperspectral images contain rich spectral and spatial information. The means to make full use of spectral information, spatial information, and enhanced sparsity constraints are the main research directions to improve the accuracy of sparse unmixing. However, many algorithms only focus on one or two of these factors, because it is difficult to construct an unmixing model that considers all three factors. To address this issue, a novel algorithm called multiview-based spectral weighted and low-rank row-sparsity unmixing is proposed. A multiview data set is generated through spectral partitioning, and then spectral weighting is imposed on it to exploit the abundant spectral information. The row-sparsity approach, which controls the sparsity by the l2,0 norm, outperforms the single-sparsity approach in many scenarios. Many algorithms use convex relaxation methods to solve the l2,0 norm to avoid the NP-hard problem, but this will reduce sparsity and unmixing accuracy. In this paper, a row-hard-threshold function is introduced to solve the l2,0 norm directly, which guarantees the sparsity of the results. The high spatial correlation of hyperspectral images is associated with low column rank; therefore, the low-rank constraint is adopted to utilize spatial information. Experiments with simulated and real data prove that the proposed algorithm can obtain better unmixing results.

Adaptive Adjustment of Compressed Measurements for Wideband Spectrum Sensing

  • Gao, Yulong;Zhang, Wei;Ma, Yongkui
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.1
    • /
    • pp.58-78
    • /
    • 2016
  • Compressed sensing (CS) possesses the potential benefits for spectrum sensing of wideband signal in cognitive radio. The sparsity of signal in frequency domain denotes the number of occupied channels for spectrum sensing. This paper presents a scheme of adaptively adjusting the number of compressed measurements to reduce the unnecessary computational complexity when priori information about the sparsity of signal cannot be acquired. Firstly, a method of sparsity estimation is introduced because the sparsity of signal is not available in some cognitive radio environments, and the relationship between the amount of used data and estimation accuracy is discussed. Then the SNR of the compressed signal is derived in the closed form. Based on the SNR of the compressed signal and estimated sparsity, an adaptive algorithm of adjusting the number of compressed measurements is proposed. Finally, some simulations are performed, and the results illustrate that the simulations agree with theoretical analysis, which prove the effectiveness of the proposed adaptive adjusting of compressed measurements.

Low Dimensional Multiuser Detection Exploiting Low User Activity

  • Lee, Junho;Lee, Seung-Hwan
    • Journal of Communications and Networks
    • /
    • v.15 no.3
    • /
    • pp.283-291
    • /
    • 2013
  • In this paper, we propose new multiuser detectors (MUDs) based on compressed sensing approaches for the large-scale multiple antenna systems equipped with dozens of low-power antennas. We consider the scenarios where the number of receiver antennas is smaller than the total number of users, but the number of active users is relatively small. This prior information motivates sparsity-embracing MUDs such as sparsity-embracing linear/nonlinear MUDs where the detection of active users and their symbol detection are employed. In addition, sparsity-embracing MUDs with maximum a posteriori probability criterion (MAP-MUDs) are presented. They jointly detect active users and their symbols by exploiting the probability of user activity, and it can be solved efficiently by introducing convex relaxing senses. Furthermore, it is shown that sparsity-embracing MUDs exploiting common users' activity across multiple symbols, i.e., frame-by-frame, can be considered to improve performance. Also, in multiple multiple-input and multiple-output networks with aggressive frequency reuse, we propose the interference cancellation strategy for the proposed sparsity-embracing MUDs. That first cancels out the interference induced by adjacent networks and then recovers the desired users' information by exploiting the low user activity. In simulation studies for binary phase shift keying modulation, numerical evidences establish the effectiveness of our proposed MUDs exploiting low user activity, as compared with the conventional MUD.

Cooperative Spectrum Sensing Utilizing Sub-Nyquist Sampling in Cognitive Radio Networks (인지 무선 네트워크에서 Sub-Nyquist 샘플링을 활용한 협력 스펙트럼 센싱 기법)

  • Jung, Honggyu;Kim, Kwangyul;Shin, Yoan
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.40 no.7
    • /
    • pp.1234-1238
    • /
    • 2015
  • We propose cooperative spectrum sensing schemes based on sub-Nyquist sampling. As compressed sensing has recently attracted great attention, sparsity order estimation techniques also has been widely investigated. Thus, assuming that the sparsity order of channel occupancy can be obtained, we mathematically analyze the detection performance of sub-Nyquist sampling schemes according to various sampling rates and cooperative spectrum sensing schemes. Simulation results verify the performance of the proposed schemes.

Compressed Sensing Based Dynamic MR Imaging: A Short Survey (Compressed Sensing 기법을 이용한 Dynamic MR Imaging)

  • Jung, Hong;Ye, Jong-Chul
    • Journal of the Institute of Electronics Engineers of Korea SP
    • /
    • v.46 no.5
    • /
    • pp.25-31
    • /
    • 2009
  • The recently developed sampling theory, "compressed sensing" is gathering huge interest in MR reconstruction area because of its feasibility of high spatio-temporal resolution of dynamic MRI which has been limited in conventional methods based on Nyquist sampling theory. Since dynamic MRI usually has high redundant information along temporal direction, this can be very sparsely represented in most of cases. Therefore, compressed sensing that exploits the sparsity of unknown images can be effectively applied in most of dynamic MRI. This review article briefly introduces currently proposed compressed sensing based dynamic MR imaging algorithms and other methods exploiting sparsity. By comparing them with conventional methods, you may have insight how the compressed sensing based methods can impact nearly every area of clinical dynamic MRI.

Power Failure Sensitivity Analysis via Grouped L1/2 Sparsity Constrained Logistic Regression

  • Li, Baoshu;Zhou, Xin;Dong, Ping
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.8
    • /
    • pp.3086-3101
    • /
    • 2021
  • To supply precise marketing and differentiated service for the electric power service department, it is very important to predict the customers with high sensitivity of electric power failure. To solve this problem, we propose a novel grouped 𝑙1/2 sparsity constrained logistic regression method for sensitivity assessment of electric power failure. Different from the 𝑙1 norm and k-support norm, the proposed grouped 𝑙1/2 sparsity constrained logistic regression method simultaneously imposes the inter-class information and tighter approximation to the nonconvex 𝑙0 sparsity to exploit multiple correlated attributions for prediction. Firstly, the attributes or factors for predicting the customer sensitivity of power failure are selected from customer sheets, such as customer information, electric consuming information, electrical bill, 95598 work sheet, power failure events, etc. Secondly, all these samples with attributes are clustered into several categories, and samples in the same category are assumed to be sharing similar properties. Then, 𝑙1/2 norm constrained logistic regression model is built to predict the customer's sensitivity of power failure. Alternating direction of multipliers (ADMM) algorithm is finally employed to solve the problem by splitting it into several sub-problems effectively. Experimental results on power electrical dataset with about one million customer data from a province validate that the proposed method has a good prediction accuracy.

On dual transformation in the interior point method of linear programming (내부점 선형계획법의 쌍대문제 전환에 대하여)

  • 설동렬;박순달;정호원
    • Proceedings of the Korean Operations and Management Science Society Conference
    • /
    • 1996.10a
    • /
    • pp.289-292
    • /
    • 1996
  • In Cholesky factorization of the interior point method, dense columns of A matrix make dense Cholesky factor L regardless of sparsity of A matrix. We introduce a method to transform a primal problem to a dual problem in order to preserve the sparsity.

  • PDF