• Title/Summary/Keyword: Stream Data Reduction

Search Result 95, Processing Time 0.033 seconds

A PCA-based Data Stream Reduction Scheme for Sensor Networks (센서 네트워크를 위한 PCA 기반의 데이터 스트림 감소 기법)

  • Fedoseev, Alexander;Choi, Young-Hwan;Hwang, Een-Jun
    • Journal of Internet Computing and Services
    • /
    • v.10 no.4
    • /
    • pp.35-44
    • /
    • 2009
  • The emerging notion of data stream has brought many new challenges to the research communities as a consequence of its conceptual difference with conventional concepts of just data. One typical example is data stream processing in sensor networks. The range of data processing considerations in a sensor network is very wide, from physical resource restrictions such as bandwidth, energy, and memory to the peculiarities of query processing including continuous and specific types of queries. In this paper, as one of the physical constraints in data stream processing, we consider the problem of limited memory and propose a new scheme for data stream reduction based on the Principal Component Analysis (PCA) technique. PCA can transform a number of (possibly) correlated variables into a (smaller) number of uncorrelated variables. We adapt PCA for the data stream of a sensor network assuming the cooperation of a query engine (or application) with a network base station. Our method exploits the spatio-temporal correlation among multiple measurements from different sensors. Finally, we present a new framework for data processing and describe a number of experiments under this framework. We compare our scheme with the wavelet transform and observe the effect of time stamps on the compression ratio. We report on some of the results.

  • PDF

Evaluation of Multivariate Stream Data Reduction Techniques (다변량 스트림 데이터 축소 기법 평가)

  • Jung, Hung-Jo;Seo, Sung-Bo;Cheol, Kyung-Joo;Park, Jeong-Seok;Ryu, Keun-Ho
    • The KIPS Transactions:PartD
    • /
    • v.13D no.7 s.110
    • /
    • pp.889-900
    • /
    • 2006
  • Even though sensor networks are different in user requests and data characteristics depending on each application area, the existing researches on stream data transmission problem focus on the performance improvement of their methods rather than considering the original characteristic of stream data. In this paper, we introduce a hierarchical or distributed sensor network architecture and data model, and then evaluate the multivariate data reduction methods suitable for user requirements and data features so as to apply reduction methods alternatively. To assess the relative performance of the proposed multivariate data reduction methods, we used the conventional techniques, such as Wavelet, HCL(Hierarchical Clustering), Sampling and SVD (Singular Value Decomposition) as well as the experimental data sets, such as multivariate time series, synthetic data and robot execution failure data. The experimental results shows that SVD and Sampling method are superior to Wavelet and HCL ia respect to the relative error ratio and execution time. Especially, since relative error ratio of each data reduction method is different according to data characteristic, it shows a good performance using the selective data reduction method for the experimental data set. The findings reported in this paper can serve as a useful guideline for sensor network application design and construction including multivariate stream data.

Study in the integrated watershade management for conservation of water resources (II) - Water quality modeling and simulation of Oship stream - (수자원 보전을 위한 유역통합관리 방안에 관한 연구(II) - 오십천 수계의 수질모델링 및 수질 예측 -)

  • 허인량;정의호;권재혁
    • Journal of Environmental Health Sciences
    • /
    • v.28 no.2
    • /
    • pp.61-69
    • /
    • 2002
  • Oship stream is located nearby south eastern coasts. This study was performed to find out waters quality modeling and then to predict water quality of Oship stream. Based on survey data, BOD, T-N, T-P calibration and verification result were in good agreement with measured value within mean coefficient variance(MSE) value, which were 13.9%, 9.0%, 26.5% and 19.5%, 12.0%, 16.5%, respectively. Sectional water quality predictions of the main stream of Oship stream are executed on the basis of the following cases 1) with sewage treatment of Dogye-eup 2) reduction of mine wastewater treatment of 80% in th basin. As a result, BOD, T-P improvement rates at down stream of Oship stream, case 1) were appeared 12.2%, 22.2%, case 2) maximum sulfate ion and conductivity reduction removal rate of Oship stream were 58%, 68%. The main pollution sources of Oship-stream were almost domestic wastewater and mine wastewater discharged from Dogye-eup which located in uppers stream. The large effects will appear after the construction of Dogye sewage water treatment plant which remove the organic matter and nutrients in these sewage water. The waste water from mine can not easily to treat for characteristics of effluence and economic problems. However, to achieve the goal of water quality in Oship-stream water system, treatments of those are necessary.

Analysis of Drying Stream Characteristics in the Rural Area (농촌하천 건천화 특성조사 및 분석 -경기 진위천 중심-)

  • Park, Ki-Wook;Yoon, Yeo-Jung;Ju, Uk-Jong
    • Proceedings of the Korean Society of Agricultural Engineers Conference
    • /
    • 2005.10a
    • /
    • pp.68-73
    • /
    • 2005
  • The purpose of this study are to insure flow rate for rural stream in the rural area. The analysis of drying stream characteristics for two streams(Bong-mu, Wang-jang stream) from field survey data. Also, a study area has been determined, and criteria of estimation has been established : criteria such as, available hydrological data, the size of stream lengths and basin areas and regional characteristics. The spacial analysis is applied to stream slopes for upstream and downstream at weirs, stream networks and ground water pump stations, sinuosity of drying stream. As a result of drying streams survey analysis, drying stream characteristics are followed; levee types are earth and natural, cross sectional shapes are trapezoid, stream bed materials are gravels and sands, facilities in streams are weirs. The cause of the reduction analyzed by investigation of the current status of facilities for agricultural water use. Agricultural reservoirs block up the stream and water does not flow over the reservoirs except by storm. They also discharge water through diversion channels and the water diverted does not flow through the natural stream. Farmers directly take water from the stream by weirs.

  • PDF

Stream Data Processing based on Sliding Window at u-Health System (u-Health 시스템에서 슬라이딩 윈도우 기반 스트림 데이터 처리)

  • Kim, Tae-Yeun;Song, Byoung-Ho;Bae, Sang-Hyun
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.4 no.2
    • /
    • pp.103-110
    • /
    • 2011
  • It is necessary to accurate and efficient management for measured digital data from sensors in u-health system. It is not efficient that sensor network process input stream data of mass storage stored in database the same time. We propose to improve the processing performance of multidimensional stream data continuous incoming from multiple sensor. We propose process query based on sliding window for efficient input stream and found multiple query plan to Mjoin method and we reduce stored data using backpropagation algorithm. As a result, we obtained to efficient result about 18.3% reduction rate of database using 14,324 data sets.

Fast Speaker Adaptation Using Sub-Stream Based Eigenvoice (Sub-Stream 기반의 Eigenvoice를 이용한 고속 화자적응)

  • Song, Hwa-Jeon;Lee, Jong-Seok;Kim, Hyung-Soon
    • MALSORI
    • /
    • v.55
    • /
    • pp.93-102
    • /
    • 2005
  • In this paper, sub-stream based eigenvoice method is proposed to overcome the weak points of conventional eigenvoice and dimensional eigenvoice. In the proposed method, sub-streams are automatically constructed by the statistical clustering analysis that uses the correlation information between dimensions. To obtain the reliable distance matrix from covariance matrix for dividing into optimal sub-streams, MAP adaptation technique is employed to the covariance matrix of training data and the sample covariance of adaptation data. According to our experiments, the proposed method shows $41\%$ error rate reduction when the number of adaptation data is 50.

  • PDF

A Data Processing Mechanism in Sensor Network Environment (센서 네트워크 환경에서의 데이터 처리 메커니즘)

  • Park, Dae-Hyun;Kim, Young-Jun;Lee, Jeong-Hoom;Chong, Il-Young
    • Proceedings of the IEEK Conference
    • /
    • 2007.07a
    • /
    • pp.133-134
    • /
    • 2007
  • The effective data processing mechanism in the sensor network means data stream model and real-time query processing model for real-time processing of stream data. This mechanism can improve satisfaction of users and reduce delay rate of data processing. In this paper, we explain the problem which is occurred when users need to search certain information among information of stream data and describe reduction model of delay rate according to data transmission.

  • PDF

Assessing Impact of Reduction of Non-Point Source Pollution by BASINS/HSPF (HSPF를 이용한 비점오염원 삭감에 따른 효과 분석)

  • Bae, Dae-Hye;Ha, Sung-Ryong
    • Journal of Environmental Impact Assessment
    • /
    • v.20 no.1
    • /
    • pp.71-78
    • /
    • 2011
  • This paper aims to assessing impact of reduction of non-point source pollution in the Bokha Stream watershed. The BASINS/HSPF model was calibrated and verified for water flow and water qualities using Total Maximum Daily Load 8days data from 2006 to 2007. Accuracy of the BASINS/HSPF models in simulating hydrology and water quality was compared and there were somewhat differences of statistical results, but water flow and water quality were simulated in good conditions over the study period. The applicability of models was tested to evaluate non-point source control scenarios to response hydrology and water quality in the Bokha stream using various measures which include BMPs approach and change of landuse. The evaluation of reduction of non-point source pollution was developed using load-duration curve. Despite strong reduction of non-point source, there are not satiated target quality at low flow season.

Analysis of the Characteristics of Thermal Environment Change Due to Urban Stream Restoration (도심 하천 복원에 따른 주변지역 열환경 변화 특성 분석)

  • Do, Woo-Gon;Jung, Woo-Sik
    • Journal of Environmental Science International
    • /
    • v.28 no.2
    • /
    • pp.235-248
    • /
    • 2019
  • The purpose of this study is to quantitatively analyze the effects of a restoration project on the decrease in the temperature in the surrounding areas. The thermal environment characteristics of the investigation area were analyzed using the meteorological data from the Busanjin Automatic Weather System which is closest to the target area. The terrain data of the modeling domain was constructed using a digital map and the urban spatial information data, and the numerical simulation of the meteorological changes before and after the restoration of the stream was performed using the Envi-met model. The average temperature of the target area in 2016 was $15.2^{\circ}C$ and was higher than that of the suburbs. The monthly mean temperature difference was the highest at $1.1^{\circ}C$ in November and the lowest in June, indicating that the temperatures in the urban areas were high in spring and winter. From the Envi-met modeling results, reductions in temperature due to stream restoration were up to $1.7^{\circ}C$ in winter, and decreased to $3.5^{\circ}C$ in summer. The effect of temperature reduction was seen in the entire region where streams are being restored.

Improvement of Stream Water Quality by Applying Best Management Practices to Chungjudam Watershed using SWAT Model (SWAT 모형을 이용한 최적관리기법 적용에 따른 충주댐 유역의 하천수질 개선연구)

  • Yu, Yung-Seok;Park, Jong-Yoon;Shin, Hyung-Jin;Kim, Saet-Byul;Kim, Seong-Joon
    • Journal of The Korean Society of Agricultural Engineers
    • /
    • v.54 no.1
    • /
    • pp.55-62
    • /
    • 2012
  • This study is to assess the reduction of nonpoint source pollution by applying Best Management Practice (BMP) in Chungju-dam watershed (6,585.1 $km^2$) using Soil and Water Assessment Tool (SWAT). The model was calibrated using 3 years (1998-2000) daily streamflow at 3 locations and monthly water quality of sediment (SS), total nitrogen (T-N) and total phosphorus (T-P) data at 2 locations and validated for another 3 years (2001-2003) data. The 5 BMPs of streambank stabilization, porous gully plugs, recharge structures, terrace, and contour farming were applied to stream and area with the specific criteria of previous researches. Through the parameter sensitivity analysis, the farming practice P-factor and Manning's roughness of stream were sensitive. Overall, the NPS reduction effect was high for streambank stabilization, terrace, and contour farming. At the watershed outlet, the SS, T-P, and T-N were reduced by 64.4 %, 62.8 % and 17.6 % respectively.