• Title/Summary/Keyword: Data Stream Processing

Search Result 445, Processing Time 0.032 seconds

DISSECTION TECHNIQUE FOR EFFICIENT JOIN OPERATION ON SEMI-STRUCTURED DOCUMENT STREAM

  • Seo, Dong-Hyeok;Lee, Dong-Gyu;Ryu, Keun-Ho
    • Proceedings of the KSRS Conference
    • /
    • 2007.10a
    • /
    • pp.11-13
    • /
    • 2007
  • There has been much interest in stream query processing. Various index techniques and advanced join techniques have been proposed to efficiently process data stream queries. Previous proposals support rapid and advanced response to the data stream queries. However, the amount of data stream is increasing and the data stream query processing needs more speedup than before. In this paper, we proposed novel query processing techniques for large number of incoming documents stream. We proposed Dissection Technique for efficient query processing in the data stream environment. We focused on the dissection technique in join query processing. Our technique shows efficient operation performance comparing with the other proposal in the data stream. Proposed technique is applied to the sensor network system and XML database.

  • PDF

An Adaptive Query Processing System for XML Stream Data (XML 스트림 데이타에 대한 적응력 있는 질의 처리 시스템)

  • Kim Young-Hyun;Kang Hyun-Chul
    • Journal of KIISE:Databases
    • /
    • v.33 no.3
    • /
    • pp.327-341
    • /
    • 2006
  • As we are getting to deal with more applications that generate streaming data such as sensor network, monitoring, and SDI (selective dissemination of information), active research is being conducted to support efficient processing of queries over streaming data. The applications on the Web environment like SDI, among others, require query processing over streaming XML data, and its investigation is very important because XML has been established as the standard for data exchange on the Web. One of the major problems with the previous systems that support query processing over streaming XML data is that they cannot deal adaptively with dynamically changing stream because they rely on static query plans. On the other hand, the stream query processing systems based on relational data model have achieved adaptiveness in query processing due to query operator routing. In this paper, we propose a system of adaptive query processing over streaming XML data in which the model of adaptive query processing over streaming relational data is applied. We compare our system with YFiiter, one of the representative systems that provide XML stream query processing capability, to show efficiency of our system.

Predictive Memory Allocation over Skewed Streams

  • Yun, Hong-Won
    • Journal of information and communication convergence engineering
    • /
    • v.7 no.2
    • /
    • pp.199-202
    • /
    • 2009
  • Adaptive memory management is a serious issue in data stream management. Data stream differ from the traditional stored relational model in several aspect such as the stream arrives online, high volume in size, skewed data distributions. Data skew is a common property of massive data streams. We propose the predicted allocation strategy, which uses predictive processing to cope with time varying data skew. This processing includes memory usage estimation and indexing with timestamp. Our experimental study shows that the predictive strategy reduces both required memory space and latency time for skewed data over varying time.

A Data-Driven Query Processing Method for Stream Data (스트림 데이터를 위한 데이터 구동형 질의처리 기법)

  • Min, Mee-Kyung
    • Journal of Digital Contents Society
    • /
    • v.8 no.4
    • /
    • pp.541-546
    • /
    • 2007
  • Traditional query processing method is not efficient for continuous queries with large continuous stream data. This paper proposes a data-driven query processing method for stream data. The structure of query plan and query execution method are presented. With the proposed method, multiple query processing and sharing among queries can be achieved. Also query execution time can be reduced by storing partial results of query execution. This paper showed an example of query processing with XML data and XQuery query.

  • PDF

DPICM subprojectile counting technique using image analysis of infrared camera (적외선 영상해석을 이용한 이중목적탄 자탄계수 계측기법연구)

  • Park, Won-Woo;Choi, Ju-Ho;Lyou, Joon
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 1997.10a
    • /
    • pp.11-16
    • /
    • 1997
  • This paper describes the grenade counting system developed for DPICM submunition analysis using the infrared video streams, and its some video stream processing technique. The video stream data processing procedure consists of four sequences; Analog infrared video stream recording, video stream capture, video stream pre-processing, and video stream analysis including the grenade counting. Some applications of this algorithms to real bursting test has shown the possibility of automation for submunition counting.

  • PDF

WT-Heuristics: An Efficient Filter Operator Ordering Technology in Stream Data Environments (WT-Heuristics: 스트림 데이터 환경에서의 효율적인 필터 연산자 순서화 기법)

  • Min, Jun-Ki
    • The KIPS Transactions:PartD
    • /
    • v.15D no.2
    • /
    • pp.163-170
    • /
    • 2008
  • Due to the proliferation of the Internet and intranet, a new application domain called stream data processing has emerged. Stream data is real-timely and continuously generated. In this paper, we focus on the processing of stream data whose characteristics vary unpredictably by over time. Particularly, we suggest a method which generates an efficient operator execution order called WT-Heuristics. WT-Heuristics efficiently determines the operator execution order since it considers only two adjacent operators in the operator execution order. Also, our method changes the execution order with respect to the change of data characteristics with minimum overheads.

A PCA-based Data Stream Reduction Scheme for Sensor Networks (센서 네트워크를 위한 PCA 기반의 데이터 스트림 감소 기법)

  • Fedoseev, Alexander;Choi, Young-Hwan;Hwang, Een-Jun
    • Journal of Internet Computing and Services
    • /
    • v.10 no.4
    • /
    • pp.35-44
    • /
    • 2009
  • The emerging notion of data stream has brought many new challenges to the research communities as a consequence of its conceptual difference with conventional concepts of just data. One typical example is data stream processing in sensor networks. The range of data processing considerations in a sensor network is very wide, from physical resource restrictions such as bandwidth, energy, and memory to the peculiarities of query processing including continuous and specific types of queries. In this paper, as one of the physical constraints in data stream processing, we consider the problem of limited memory and propose a new scheme for data stream reduction based on the Principal Component Analysis (PCA) technique. PCA can transform a number of (possibly) correlated variables into a (smaller) number of uncorrelated variables. We adapt PCA for the data stream of a sensor network assuming the cooperation of a query engine (or application) with a network base station. Our method exploits the spatio-temporal correlation among multiple measurements from different sensors. Finally, we present a new framework for data processing and describe a number of experiments under this framework. We compare our scheme with the wavelet transform and observe the effect of time stamps on the compression ratio. We report on some of the results.

  • PDF

Frequent Patten Tree based XML Stream Mining (빈발 패턴 트리 기반 XML 스트림 마이닝)

  • Hwang, Jeong-Hee
    • The KIPS Transactions:PartD
    • /
    • v.16D no.5
    • /
    • pp.673-682
    • /
    • 2009
  • XML data are widely used for data representation and exchange on the Web and the data type is an continuous stream in ubiquitous environment. Therefore there are some mining researches related to the extracting of frequent structures and the efficient query processing of XML stream data. In this paper, we propose a mining method to extract frequent structures of XML stream data in recent window based on the sliding window. XML stream data are modeled as a tree set, called XFP_tree and we quickly extract the frequent structures over recent XML data in the XFP_tree.

The Framework of Stream Data Processing System for Realtime Health Care Service (실시간 헬스케어 서비스를 위한 스트림 데이터 시스템 프레임워크의 설계)

  • Wu, Zejun;Lee, Yeon;Bae, Hae-Young
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2011.06a
    • /
    • pp.21-22
    • /
    • 2011
  • The growth of using smartphone and tablet pc has enabled variety kinds of realtime applications. In these applications, the data which we called data stream is multidimensional, continuous, rapid, and time-varying. However the traditional Database Management System (DBMS) suffers from processing the real time and complex application, in this paper we proposed the framework for CCR Data Stream Server's design and implementation that compiled with Data Stream Database Management System (DSMS) and DBMS in EMR system. The system enables users not only to query stored CCR information from DBMS, but also to execute continues query for the real-time CCR Data Stream.

  • PDF

A Multi-dimensional Query Processing Scheme for Stream Data using Range Query Indexing (범위 질의 인덱싱을 이용한 스트림 데이터의 다중 질의처리 기법)

  • Lee, Dong-Un;Rhee, Yun-Seok
    • Journal of the Korea Society of Computer and Information
    • /
    • v.14 no.2
    • /
    • pp.69-77
    • /
    • 2009
  • Stream service environment demands real-time query processing for voluminous data which are ceaselessly delivered from tremendous sources. Typical R-tree based query processing technologies cannot efficiently handle such situations, which require repetitive and inefficient exploration from the tree root on every data event. However, many stream data including sensor readings show high locality, which we exploit to reduce the search space of queries to explore. In this paper, we propose a query processing scheme exploiting the locality of stream data. From the simulation, we conclude that the proposed scheme performs much better than the traditional ones in terms of scalability and exploration efficiency.