• Title/Summary/Keyword: Data normalization

Search Result 480, Processing Time 0.039 seconds

An Iterative Normalization Algorithm for cDNA Microarray Medical Data Analysis

  • Kim, Yoonhee;Park, Woong-Yang;Kim, Ho
    • Genomics & Informatics
    • /
    • v.2 no.2
    • /
    • pp.92-98
    • /
    • 2004
  • A cDNA microarray experiment is one of the most useful high-throughput experiments in medical informatics for monitoring gene expression levels. Statistical analysis with a cDNA microarray medical data requires a normalization procedure to reduce the systematic errors that are impossible to control by the experimental conditions. Despite the variety of normalization methods, this. paper suggests a more general and synthetic normalization algorithm with a control gene set based on previous studies of normalization. Iterative normalization method was used to select and include a new control gene set among the whole genes iteratively at every step of the normalization calculation initiated with the housekeeping genes. The objective of this iterative normalization was to maintain the pattern of the original data and to keep the gene expression levels stable. Spatial plots, M&A (ratio and average values of the intensity) plots and box plots showed a convergence to zero of the mean across all genes graphically after applying our iterative normalization. The practicability of the algorithm was demonstrated by applying our method to the data for the human photo aging study.

Normalization of Microarray Data: Single-labeled and Dual-labeled Arrays

  • Do, Jin Hwan;Choi, Dong-Kug
    • Molecules and Cells
    • /
    • v.22 no.3
    • /
    • pp.254-261
    • /
    • 2006
  • DNA microarray is a powerful tool for high-throughput analysis of biological systems. Various computational tools have been created to facilitate the analysis of the large volume of data produced in DNA microarray experiments. Normalization is a critical step for obtaining data that are reliable and usable for subsequent analysis such as identification of differentially expressed genes and clustering. A variety of normalization methods have been proposed over the past few years, but no methods are still perfect. Various assumptions are often taken in the process of normalization. Therefore, the knowledge of underlying assumption and principle of normalization would be helpful for the correct analysis of microarray data. We present a review of normalization techniques from single-labeled platforms such as the Affymetrix GeneChip array to dual-labeled platforms like spotted array focusing on their principles and assumptions.

Study on Normalization Method of LED Degradation Data (LED 열화 데이터에 대한 정규화 방법에 대한 연구)

  • Jeong, Uihyo;Lim, Hongwoo;Hyung, Jaephil;Jung, Changuk;Cho, Jeongha;Jang, Joongsoon
    • Journal of Applied Reliability
    • /
    • v.18 no.1
    • /
    • pp.49-55
    • /
    • 2018
  • Purpose: To propose improved method for normalization, compare to the de facto international standard which is IESNA TM-21 or conventional normalization methods. Methods: Firstly, we analysed conventional methods and specified the problem of normalization method which is based on first measured data. Secondly, we proposed our approach which is based on the design specification. Lastly, we studied a real degradation data which is conducted for 15,000 hours. Conclusion: Proposed normalization method is better approach because it can reflect real data and design specification, and reduce distortion when analysing degradation data. Also, It is appliable to other long-life reliability items.

Comparison of Normalization Methods for Defining Copy Number Variation Using Whole-genome SNP Genotyping Data

  • Kim, Ji-Hong;Yim, Seon-Hee;Jeong, Yong-Bok;Jung, Seong-Hyun;Xu, Hai-Dong;Shin, Seung-Hun;Chung, Yeun-Jun
    • Genomics & Informatics
    • /
    • v.6 no.4
    • /
    • pp.231-234
    • /
    • 2008
  • Precise and reliable identification of CNV is still important to fully understand the effect of CNV on genetic diversity and background of complex diseases. SNP marker has been used frequently to detect CNVs, but the analysis of SNP chip data for identifying CNV has not been well established. We compared various normalization methods for CNV analysis and suggest optimal normalization procedure for reliable CNV call. Four normal Koreans and NA10851 HapMap male samples were genotyped using Affymetrix Genome-Wide Human SNP array 5.0. We evaluated the effect of median and quantile normalization to find the optimal normalization for CNV detection based on SNP array data. We also explored the effect of Robust Multichip Average (RMA) background correction for each normalization process. In total, the following 4 combinations of normalization were tried: 1) Median normalization without RMA background correction, 2) Quantile normalization without RMA background correction, 3) Median normalization with RMA background correction, and 4) Quantile normalization with RMA background correction. CNV was called using SW-ARRAY algorithm. We applied 4 different combinations of normalization and compared the effect using intensity ratio profile, box plot, and MA plot. When we applied median and quantile normalizations without RMA background correction, both methods showed similar normalization effect and the final CNV calls were also similar in terms of number and size. In both median and quantile normalizations, RMA backgroundcorrection resulted in widening the range of intensity ratio distribution, which may suggest that RMA background correction may help to detect more CNVs compared to no correction.

Rotation Angle Estimation of Multichannel Images (다채널 이미지의 회전각 추정)

  • Lee Bong-Kyu;Yang Yo-Han
    • The Transactions of the Korean Institute of Electrical Engineers D
    • /
    • v.51 no.6
    • /
    • pp.267-271
    • /
    • 2002
  • The Hotelling transform is based on statistical properties of an image. The principal uses of this transform are in data compression. The basic concept of the Hotelling transform is that the choice of basis vectors pointing the direction of maximum variance of the data. This property can be used for rotation normalization. Many objects of interest in pattern recognition applications can be easily standardized by performing a rotation normalization that aligns the coordinate axes with the axes of maximum variance of the pixels in the object. However, this transform can not be used to rotation normalization of color images directly. In this paper, we propose a new method for rotation normalization of color images based on the Hotelling transform. The Hotelling transform is performed to calculate basis vectors of each channel. Then the summation of vectors of all channels are processed. Rotation normalization is performed using the result of summation of vectors. Experimental results showed the proposed method can be used for rotation normalization of color images effectively.

Effectiveness of Normalization Pre-Processing of Big Data to the Machine Learning Performance (빅데이터의 정규화 전처리과정이 기계학습의 성능에 미치는 영향)

  • Jo, Jun-Mo
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.14 no.3
    • /
    • pp.547-552
    • /
    • 2019
  • Recently, the massive growth in the scale of data has been observed as a major issue in the Big Data. Furthermore, the Big Data should be preprocessed for normalization to get a high performance of the Machine learning since the Big Data is also an input of Machine Learning. The performance varies by many factors such as the scope of the columns in a Big Data or the methods of normalization preprocessing. In this paper, the various types of normalization preprocessing methods and the scopes of the Big Data columns will be applied to the SVM(: Support Vector Machine) as a Machine Learning method to get the efficient environment for the normalization preprocessing. The Machine Learning experiment has been programmed in Python and the Jupyter Notebook.

Analysis of normalization effect for earthquake events classification (지진 이벤트 분류를 위한 정규화 기법 분석)

  • Zhang, Shou;Ku, Bonhwa;Ko, Hansoek
    • The Journal of the Acoustical Society of Korea
    • /
    • v.40 no.2
    • /
    • pp.130-138
    • /
    • 2021
  • This paper presents an effective structure by applying various normalization to Convolutional Neural Networks (CNN) for seismic event classification. Normalization techniques can not only improve the learning speed of neural networks, but also show robustness to noise. In this paper, we analyze the effect of input data normalization and hidden layer normalization on the deep learning model for seismic event classification. In addition an effective model is derived through various experiments according to the structure of the applied hidden layer. As a result of various experiments, the model that applied input data normalization and weight normalization to the first hidden layer showed the most stable performance improvement.

Robust Feature Normalization Scheme Using Separated Eigenspace in Noisy Environments (분리된 고유공간을 이용한 잡음환경에 강인한 특징 정규화 기법)

  • Lee Yoonjae;Ko Hanseok
    • The Journal of the Acoustical Society of Korea
    • /
    • v.24 no.4
    • /
    • pp.210-216
    • /
    • 2005
  • We Propose a new feature normalization scheme based on eigenspace for achieving robust speech recognition. In general, mean and variance normalization (MVN) is Performed in cepstral domain. However, another MVN approach using eigenspace was recently introduced. in that the eigenspace normalization Procedure Performs normalization in a single eigenspace. This Procedure consists of linear PCA matrix feature transformation followed by mean and variance normalization of the transformed cepstral feature. In this method. 39 dimensional feature distribution is represented using only a single eigenspace. However it is observed to be insufficient to represent all data distribution using only a sin91e eigenvector. For more specific representation. we apply unique na independent eigenspaces to cepstra, delta and delta-delta cepstra respectively in this Paper. We also normalize training data in eigenspace and get the model from the normalized training data. Finally. a feature space rotation procedure is introduced to reduce the mismatch of training and test data distribution in noisy condition. As a result, we obtained a substantial recognition improvement over the basic eigenspace normalization.

A Single Index Approach for Subsequence Matching that Supports Normalization Transform in Time-Series Databases (시계열 데이터베이스에서 단일 색인을 사용한 정규화 변환 지원 서브시퀀스 매칭)

  • Moon Yang-Sae;Kim Jin-Ho;Loh Woong-Kee
    • The KIPS Transactions:PartD
    • /
    • v.13D no.4 s.107
    • /
    • pp.513-524
    • /
    • 2006
  • Normalization transform is very useful for finding the overall trend of the time-series data since it enables finding sequences with similar fluctuation patterns. The previous subsequence matching method with normalization transform, however, would incur index overhead both in storage space and in update maintenance since it should build multiple indexes for supporting arbitrary length of query sequences. To solve this problem, we propose a single index approach for the normalization transformed subsequence matching that supports arbitrary length of query sequences. For the single index approach, we first provide the notion of inclusion-normalization transform by generalizing the original definition of normalization transform. The inclusion-normalization transform normalizes a window by using the mean and the standard deviation of a subsequence that includes the window. Next, we formally prove correctness of the proposed method that uses the inclusion-normalization transform for the normalization transformed subsequence matching. We then propose subsequence matching and index building algorithms to implement the proposed method. Experimental results for real stock data show that our method improves performance by up to $2.5{\sim}2.8$ times over the previous method. Our approach has an additional advantage of being generalized to support many sorts of other transforms as well as normalization transform. Therefore, we believe our work will be widely used in many sorts of transform-based subsequence matching methods.

Energy Feature Normalization for Robust Speech Recognition in Noisy Environments

  • Lee, Yoon-Jae;Ko, Han-Seok
    • Speech Sciences
    • /
    • v.13 no.1
    • /
    • pp.129-139
    • /
    • 2006
  • In this paper, we propose two effective energy feature normalization methods for robust speech recognition in noisy environments. In the first method, we estimate the noise energy and remove it from the noisy speech energy. In the second method, we propose a modified algorithm for the Log-energy Dynamic Range Normalization (ERN) method. In the ERN method, the log energy of the training data in a clean environment is transformed into the log energy in noisy environments. If the minimum log energy of the test data is outside of a pre-defined range, the log energy of the test data is also transformed. Since the ERN method has several weaknesses, we propose a modified transform scheme designed to reduce the residual mismatch that it produces. In the evaluation conducted on the Aurora2.0 database, we obtained a significant performance improvement.

  • PDF