• Title/Summary/Keyword: Data integration

Search Result 3,354, Processing Time 0.028 seconds

Performing Data Integration: Handed-code Approach vs. Tool-based Approach

  • Koo, Heung-Seo
    • Journal of the Korea Society of Computer and Information
    • /
    • v.24 no.7
    • /
    • pp.39-44
    • /
    • 2019
  • Data integration technology is one of the key elements in building data warehouses or big data, and is used to combine data from multiple sources and provide an integrated view to users. Traditionally, the performance of data integration uses a handed-code approach or a tool-based approach that utilizes data integration tools such as ETL. There is a debate about which methods are efficient. This study is conducted to give practitioners preparing for a data integration project an insight into how to perform data integration. This paper examines the views of experts on the controversy over the adoption of ETL tools that have been on the agenda of the data integration area for over a decade.

The Development of an Integration Tool for the Data Sharing Among the Enterprise information Systems (기업 정보 시스템간 효율적인 데이터 공유를 위한 통합 도구 개발)

  • 한관희;박찬우;최운집;이상한
    • Proceedings of the Korean Society of Precision Engineering Conference
    • /
    • 2004.10a
    • /
    • pp.782-787
    • /
    • 2004
  • Recently, many enterprises are introducing EAI(Enterprise Application Integration) technologies for integrating heterogeneous enterprise information systems. Among EAI levels, data-level integration is relatively straightforward and most popular. However, current commercial solutions have complex functionalities and are expensive for implementing the data integration tasks. Also, they have their own proprietary architectures and have a restricted interoperability. Proposed in this paper is the development of data integration middleware for facilitating data exchanges between the heterogeneous information systems. The main feature of this middleware is a explicit mapping of meta data about the relationships between source and target data. Based on this mapping, users who do not have expertise in information technology at the small & medium enterprise can easily handle data exchange tasks between information systems.

  • PDF

The Development of a Data Integration Middleware for Enterprise Information Systems (기업 정보 시스템 간 데이터 통합을 위한 미들웨어 개발)

  • Han, K.H.;Park, C.W.;Bae, S.M.
    • IE interfaces
    • /
    • v.17 no.4
    • /
    • pp.407-413
    • /
    • 2004
  • Recently, many enterprises are adopting EAI (Enterprise Application Integration) technologies for integrating heterogeneous enterprise information systems. Among EAI levels, data-level integration is relatively straightforward and most popular. However, most commercial solutions provide complex functionalities and are expensive for implementing the data integration tasks at the small & medium enterprises. Also, they have their own proprietary architectures and have a restricted interoperability. Proposed in this paper is the development of a data integration middleware for facilitating data exchanges between the heterogeneous information systems. The main feature of this middleware is a explicit mapping of meta data about the relationships between source and target data. Based on this explicit mapping, users who do not have expertise in information technology at the small & medium enterprises can easily execute data exchange tasks among various information systems.

Intelligent Data Governance for the Federated Integration of Air Quality Databases in the Railway Industry (철도 산업의 공기 질 데이터베이스 연합형 통합을 위한 지능형 데이터 거버넌스)

  • Minjeong, Kim;Jong-Un, Won;Sangchan, Park;Gayoung, Park
    • Journal of Korean Society for Quality Management
    • /
    • v.50 no.4
    • /
    • pp.811-830
    • /
    • 2022
  • Purpose: In this paper, we will discuss 1) prioritizing databases to be integrated; 2) which data elements should be emphasized in federated database integration; and 3) the degree of efficiency in the integration. This paper aims to lay the groundwork for building data governance by presenting guidelines for database integration using metrics to identify and evaluate the capabilities of the UK's air quality databases. Methods: This paper intends to perform relative efficiency analysis using Data Envelope Analysis among the multi-criteria decision-making methods. In federated database integration, it is important to identify databases with high integration efficiency when prioritizing databases to be integrated. Results: The outcome of this paper aims not to present performance indicators for the implementation and evaluation of data governance, but rather to discuss what criteria should be used when performing 'federated integration'. Using Data Envelope Analysis in the process of implementing intelligent data governance, authors will establish and present practical strategies to discover databases with high integration efficiency. Conclusion: Through this study, it was possible to establish internal guidelines from an integrated point of view of data governance. The flexiblity of the federated database integration under the practice of the data governance, makes it possible to integrate databases quickly, easily, and effectively. By utilizing the guidelines presented in this study, authors anticipate that the process of integrating multiple databases, including the air quality databases, will evolve into the intelligent data governance based on the federated database integration when establishing the data governance practice in the railway industry.

A Study on Hybrid Database Integration Model for Product Data Management (PDM을 위한 하이브리드 데이터베이스 통합 모델에 관한 연구)

  • Lee, Kang-Chan;Lee, Sang;Yoo, Jung-Yeon;Lee, Kyu-Chul
    • The Journal of Society for e-Business Studies
    • /
    • v.3 no.1
    • /
    • pp.23-41
    • /
    • 1998
  • In a centralized database system, all system components reside at a single platform. In recent years there has been a rapid trend toward the integration of information systems over multiple sites that are interconnected via a communication network, and users' needs are changed to integration of multiple information sites. Multi database System is one of solutions for integrating distributed heterogeneous databases. However the problems in multi database system are restriction in distributed environment support, limitation in integrating heterogeneous media type data, static integration, and data-only of integration. In order to solve these problems, we propose a hybrid database integration model, HyDIM. HyDIM is used for the integrating legacy multimedia data, adopting CORBA, MDS, and mediator. We demonstrate a prototype system far PDM application domain.

  • PDF

Introduction and Utilization of Time Series Data Integration Framework with Different Characteristics (서로 다른 특성의 시계열 데이터 통합 프레임워크 제안 및 활용)

  • Jisoo, Hwanga;Jaewon, Moon
    • Journal of Broadcast Engineering
    • /
    • v.27 no.6
    • /
    • pp.872-884
    • /
    • 2022
  • With the development of the IoT industry, different types of time series data are being generated in various industries, and it is evolving into research that reproduces and utilizes it through re-integration. In addition, due to data processing speed and issues of the utilization system in the actual industry, there is a growing tendency to compress the size of data when using time series data and integrate it. However, since the guidelines for integrating time series data are not clear and each characteristic such as data description time interval and time section is different, it is difficult to use it after batch integration. In this paper, two integration methods are proposed based on the integration criteria setting method and the problems that arise during integration of time series data. Based on this, integration framework of a heterogeneous time series data was constructed that is considered the characteristics of time series data, and it was confirmed that different heterogeneous time series data compressed can be used for integration and various machine learning.

Development of Pointcloud Data Integration Technology in Construction Sites via Drone Photogrammetry and MMS LiDAR (드론 및 MMS를 활용한 건설현장 점군 데이터 통합 기술 개발)

  • Jae-Woo Park;Dong-Jun Yeom
    • Journal of the Korean Society of Industry Convergence
    • /
    • v.26 no.6_2
    • /
    • pp.1145-1153
    • /
    • 2023
  • This study presents the development of pointcloud data integration technology in construction sites via drone photogrammetry and MMS LiDAR. The integration of pointcloud data from drones and MMS technology can provide precise and accurate 3D digital maps of construction sites, which can benefit the development of smart construction and BIM. The advantages of using both drones and MMS technology for pointcloud data acquisition in construction sites are discussed, along with the limitations and challenges of using drone photogrammetry and MMS LiDAR for pointcloud data integration. The results of this study can contribute to the advancement of pointcloud data integration technology in construction sites and improve the efficiency and accuracy of construction projects.

ERS-1 AND CCRS C-SAR Data Integration For Look Direction Bias Correction Using Wavelet Transform

  • Won, J.S.;Moon, Woo-Il M.;Singhroy, Vern;Lowman, Paul-D.Jr.
    • Korean Journal of Remote Sensing
    • /
    • v.10 no.2
    • /
    • pp.49-62
    • /
    • 1994
  • Look direction bias in a single look SAR image can often be misinterpreted in the geological application of radar data. This paper investigates digital processing techniques for SAR image data integration and compensation of the SAR data look direction bias. The two important approaches for reducing look direction bias and integration of multiple SAR data sets are (1) principal component analysis (PCA), and (2) wavelet transform(WT) integration techniques. These two methods were investigated and tested with the ERS-1 (VV-polarization) and CCRS*s airborne (HH-polarization) C-SAR image data sets recorded over the Sudbury test site, Canada. The PCA technique has been very effective for integration of more than two layers of digital image data. When there only two sets of SAR data are available, the PCA thchnique requires at least one more set of auxiliary data for proper rendition of the fine surface features. The WT processing approach of SAR data integration utilizes the property which decomposes images into approximated image ( low frequencies) characterizing the spatially large and relatively distinct structures, and detailed image (high frequencies) in which the information on detailed fine structures are preserved. The test results with the ERS-1and CCRS*s C-SAR data indicate that the new WT approach is more efficient and robust in enhancibng the fine details of the multiple SAR images than the PCA approach.

GIS-based Spatial Integration and Statistical Analysis using Multiple Geoscience Data Sets : A Case Study for Mineral Potential Mapping (다중 지구과학자료를 이용한 GIS 기반 공간통합과 통계량 분석 : 광물 부존 예상도 작성을 위한 사례 연구)

  • 이기원;박노욱;권병두;지광훈
    • Korean Journal of Remote Sensing
    • /
    • v.15 no.2
    • /
    • pp.91-105
    • /
    • 1999
  • Spatial data integration using multiple geo-based data sets has been regarded as one of the primary GIS application issues. As for this issue, several integration schemes have been developed as the perspectives of mathematical geology or geo-mathematics. However, research-based approaches for statistical/quantitative assessments between integrated layer and input layers are not fully considered yet. Related to this niche point, in this study, spatial data integration using multiple geoscientific data sets by known integration algorithms was primarily performed. For spatial integration by using raster-based GIS functionality, geological, geochemical, geophysical data sets, DEM-driven data sets and remotely sensed imagery data sets from the Ogdong area were utilized for geological thematic mapping related by mineral potential mapping. In addition, statistical/quantitative information extraction with respective to relationships among used data sets and/or between each data set and integrated layer was carried out, with the scope of multiple data fusion and schematic statistical assessment methodology. As for the spatial integration scheme, certainty factor (CF) estimation and principal component analysis (PCA) were applied. However, this study was not aimed at direct comparison of both methodologies; whereas, for the statistical/quantitative assessment between integrated layer and input layers, some statistical methodologies based on contingency table were focused. Especially, for the bias reduction, jackknife technique was also applied in PCA-based spatial integration. Through the statistic analyses with respect to the integration information in this case study, new information for relationships of integrated layer and input layers was extracted. In addition, influence effects of input data sets with respect to integrated layer were assessed. This kind of approach provides a decision-making information in the viewpoint of GIS and is also exploratory data analysis in conjunction with GIS and geoscientific application, especially handing spatial integration or data fusion with complex variable data sets.

A Database Schema Integration Method Using XML Schema (XML Schema를 이용한 이질의 데이터베이스 스키마 통합)

  • 박우창
    • Journal of Internet Computing and Services
    • /
    • v.3 no.2
    • /
    • pp.39-56
    • /
    • 2002
  • In distributed computing environments, there are many database applications that should share data each other such as data warehousing and data mining with autonomy on local databases. The first step to such applications is the integration of heterogeneous database schema, but there is no accepted common data model for the integration and also are difficulties on the construction of integration program. In this paper, we use the XML Schema for the representation of common data model and exploit XSLT for reducing the programming difficulties. We define the schema integration operations and develop a methodology for the semi-automatic schema integration according to schema conflicts types. Our integration method has benefits on standardization, extendibility on schema integration process comparing to existing methodologies.

  • PDF