• Communications for Statistical Applications and Methods
• /
• 제17권6호
• /
• pp.899-907
• /
• 2010

무응답 대체(non-response imputation) 방법에 관한 많은 이론과 방법이 제안되었으며 실제 자료 분석에 이용되고 있다. 흔히 횡단면 무응답 대체를 위하여 다중대체법(multiple imputation)이 사용되고 있으며 2차년도 이상의 패널자료에는 종시점회귀대체법(cross-wave regression imputation)이 사용되고 있다. 본 연구에서는 패널자료 분석을 위하여 종시점회귀대체법의 일반형태인 시계열 대체법과 횡단면 무응답 대체법을 결합한 시계열-횡단면 다중 대체법을 제안하였다. 노동부의 매월노동통계 자료를 이용하여 제안한 방법과 기존의 종시점회귀대체법을 비교하여 우수함을 보였다.

• Communications for Statistical Applications and Methods
• /
• 제10권2호
• /
• pp.575-583
• /
• 2003

We proposed imputation strategies for the Population and Housing Census 2000 in Korea. The total area of floor space and marital status which have relatively high non-response rates in the Census are considered to develope the effective missing value imputation procedures. The Classification and Regression Tree(CART) is employed to construct the imputation cells for hot-deck imputation, as well as to predict missing value by model-based approach. We compare three imputation methods which include CART model-based imputation, hot-deck imputation based on CART and logical hot-deck imputation proposed by The Korea National Statistical Office. The results suggest that the proposed hot-deck imputation based on CART is very efficient and strongly recommendable.

• Journal of Information Processing Systems
• /
• 제9권4호
• /
• pp.633-650
• /
• 2013

All the imputation techniques proposed so far in literature for data imputation are offline techniques as they require a number of iterations to learn the characteristics of data during training and they also consume a lot of computational time. Hence, these techniques are not suitable for applications that require the imputation to be performed on demand and near real-time. The paper proposes a computational intelligence based architecture for online data imputation and extended versions of an existing offline data imputation method as well. The proposed online imputation technique has 2 stages. In stage 1, Evolving Clustering Method (ECM) is used to replace the missing values with cluster centers, as part of the local learning strategy. Stage 2 refines the resultant approximate values using a General Regression Neural Network (GRNN) as part of the global approximation strategy. We also propose extended versions of an existing offline imputation technique. The offline imputation techniques employ K-Means or K-Medoids and Multi Layer Perceptron (MLP)or GRNN in Stage-1and Stage-2respectively. Several experiments were conducted on 8benchmark datasets and 4 bank related datasets to assess the effectiveness of the proposed online and offline imputation techniques. In terms of Mean Absolute Percentage Error (MAPE), the results indicate that the difference between the proposed best offline imputation method viz., K-Medoids+GRNN and the proposed online imputation method viz., ECM+GRNN is statistically insignificant at a 1% level of significance. Consequently, the proposed online technique, being less expensive and faster, can be employed for imputation instead of the existing and proposed offline imputation techniques. This is the significant outcome of the study. Furthermore, GRNN in stage-2 uniformly reduced MAPE values in both offline and online imputation methods on all datasets.

• Communications for Statistical Applications and Methods
• /
• 제30권3호
• /
• pp.331-341
• /
• 2023

Handling missing values in data analysis is essential in constructing a good prediction model. The easiest way to handle missing values is to use complete case data, but this can lead to information loss within the data and invalid conclusions in data analysis. Imputation is a technique that replaces missing data with alternative values obtained from information in a dataset. Conventional imputation methods include K-nearest-neighbor imputation and multiple imputations. Recent methods include missForest, missRanger, and mixgb ,all which use machine learning algorithms. This paper compares the imputation techniques for datasets with mixed datatypes in various situations, such as data size, missing ratios, and missing mechanisms. To evaluate the performance of each method in mixed datasets, we propose a new imputation performance measure (IPM) that is a unified measurement applicable to numerical and categorical variables. We believe this metric can help find the best imputation method. Finally, we summarize the comparison results with imputation performances and computational times.

• Communications for Statistical Applications and Methods
• /
• 제30권6호
• /
• pp.577-587
• /
• 2023

Conventional categorical data imputation techniques, such as mode imputation, often encounter issues related to overestimation. If the variable has too many categories, multinomial logistic regression imputation method may be impossible due to computational limitations. To rectify these limitations, we propose a two-stage imputation method. During the first stage, we utilize the Boruta variable selection method on the complete dataset to identify significant variables for the target categorical variable. Then, in the second stage, we use the important variables for the target categorical variable for logistic regression to impute missing data in binary variables, polytomous regression to impute missing data in categorical variables, and predictive mean matching to impute missing data in quantitative variables. Through analysis of both asymmetric and non-normal simulated and real data, we demonstrate that the two-stage imputation method outperforms imputation methods lacking variable selection, as evidenced by accuracy measures. During the analysis of real survey data, we also demonstrate that our suggested two-stage imputation method surpasses the current imputation approach in terms of accuracy.

• Journal of the Korean Statistical Society
• /
• 제36권3호
• /
• pp.423-434
• /
• 2007

Imputation using a regression model is a method to preserve the correlation among variables and to provide imputed point estimators. We discuss the implementation of regression imputation using fractional imputation. By a suitable choice of fractional weights, the fractional regression imputation can take the form of hot deck fractional imputation, thus no artificial values are constructed after the imputation. A variance estimator, which extends the method of Kim and Fuller (2004), is also proposed. Results from a limited simulation study are presented.

• 한국조사연구학회지:조사연구
• /
• 제11권1호
• /
• pp.1-18
• /
• 2010

본 논문에서는 패널조사에서 발생하는 웨이브 무응답을 대체하는 방법을 고찰하였다. 패널조사에서는 이전 조사 데이터를 무응답 대체에 활용할 수 있기 때문에 이러한 성질을 이용하면 횡단면 무응답 대체보다 더 효과적인 웨이브 무응답 대체법을 찾을 수 있다. 먼저 웨이브 무응답 대체를 사용하는 해외의 주요 패널조사를 살펴보고, 웨이브 무응답 대체방법 중 종단면 회귀대체법, 이월대체법, 최근방 회귀대체법, 그리고 행렬대체법을 고찰하였다. 그리고 웨이브 무응답 대체법의 성능을 비교하기 위하여 한국복지패널 데이터를 대상으로 모의실험을 실시하였다. 성능을 비교하기 위하여 평균대체, 회귀대체, 비대체, 최근방 대체, 핫덱 대체를 고려하였고 성능평가 지표로는 예측 정확성 지표와 추정 정확성 지표를 이용하였다. 모의실험 결과 비대체, 행렬대체는 두 지표 모두 우수했고, 회귀대체, 종단면 회귀대체, 이월대체는 예측 정확성은 우수한 반면 추정 정확성은 다소 떨어졌으며, 반대로 최근방 회귀대체, 최근방 대체, 핫덱 대체는 예측 정확성은 떨어지나 추정 정확성은 높은 것으로 나타났다. 마지막으로 평균 대체는 두 지표 모두 좋지 않았다.

• Journal of the Korean Data and Information Science Society
• /
• 제14권2호
• /
• pp.143-152
• /
• 2003

A data set having missing observations is often completed by using imputed values. In this paper, performances and accuracy of five imputation procedures are evaluated when missing values exist only on the response variable in the exponential regression model. Our simulation results show that adjusted exponential regression imputation procedure can be well used to compensate for missing data, in particular, compared to other imputation procedures. An illustrative example using real data is provided.

• Journal of the Korean Statistical Society
• /
• 제29권4호
• /
• pp.385-394
• /
• 2000

Jackknife variance estimation based on adjusted imputed values when nonresponse is nonrandom and follow-up data are available for a subsample of nonrespondents is provided. Both hot-deck and ratio imputation method are considered as imputation method. The performance of the proposed variance estimator under nonrandom response mechanism is investigated through numerical simulation.

• 한국데이터정보과학회:학술대회논문집
• /
• 한국데이터정보과학회 2003년도 춘계학술대회
• /
• pp.135-144
• /
• 2003

A data set having missing observations is often completed by using imputed values. In this paper, performances and accuracy of five imputation procedures are evaluated when missing values exist only on the response variable in the exponential regression model. Our simulation results show that adjusted exponential regression imputation procedure can be well used to compensate for missing data, in particular, compared to other imputation procedures. An illustrative example using real data is provided.