Advanced SearchSearch Tips
Outlier detection and treatment in industrial sampling survey
facebook(new window)  Pirnt(new window) E-mail(new window) Excel Download
 Title & Authors
Outlier detection and treatment in industrial sampling survey
Joo, Young Sun; Cho, Gyo-Young;
  PDF(new window)
Outliers in surveys can have a large effect on estimates of totals. This is especially true in business surveys where the populations are drawn are typically skewed. In this paper, we discussed the practical development and implementation of methods to identify and deal with outliers. A detection method is based on quartile method and detected outlier is processed in various ways. The study examines two versions of winsorised estimators with three different cut-off thresholds for each one. For the simulation study, four types of weight transformation function have been considered.
Outlier detection;outlier treatment;winsorization;weight reduction;
 Cited by
Chambers, R., Kokic, P., Smith, P. and Cruddas, M. (2000). Winsorization for identifying and treating outliers in business surveys. Proceedings of the Second International Conference on Establishment Surveys, 717-726, American Statistical Association Alexandria, Virginia.

Eltinge, J. L. and Cantwell, P. J. (2006). Outliers and influential observations in establishment surveys, Federal Economic Statistics Advisory Committee,

Hidiroglou, M. A. and Berthelot, J. M. (1986). Statistical editing and imputation for periodic business surveys. Survey Methodology, 12, 73-83.

Ishikawa, A., Endo, S. and Shiratori, T. (2010). Treatment of outliers in business surveys : The case of short-term economic survey of enterprises in Japan (Tankan), 10-E-8, Bank of Japan, Japan.

Kim, J. (2006). Weight reduction method for ourlier in survey sampling. Communications for Statistical Applications and Methods, 13, 19-27. crossref(new window)

Kim, J. T. (2014). Lowness and outlier analysis of biological oxygen demand on Nakdong main stream river. Journal of the Korean Data & information Science Society, 25, 119-130. crossref(new window)

Kokic, P. N. and Bell, P. A. (1994). Optimal winsorizing cutoffs for a stratified finite population estimator. Journal of Official Statistics, 10, 419-435.

Lee, H. (1995). Outliers in business surveys. Chapter 26 in Business Survey Methods (B. Cox et al., eds.), Wiley, New York.

Mattews, S. and Berard, H. (2002). The outlier detection and treatment strategy for the monthly wholesale and retail trade survey of statistics Canada. in Proceedings of the Survey Methods Section, 63-68, Statistical Society of Canada.

Sohn, K. C. and Shin, I. H. (2012). Outlier detection using Grubb and Cochran test in clinical data. Journal of the Korean Data & information Science Society, 23, 657-663. crossref(new window)

Song, G. M., Moon, J. E. and Park, C. (2011). Realization of an outlier detection algorithm using R. Journal of the Korean Data & information Science Society, 22, 449-458.

Tukey, J. W. (1977). Exploratory Data Analysis, Addison-Wesley, California.