• Proceedings of the Korean Society of Computer Information Conference
• /
• 2017.01a
• /
• pp.75-78
• /
• 2017

This paper compares operational and network analysis of centralized and distributed repository for big data solutions in the IoT enabled Smart Grid environment. The comparative analysis clearly depicts that centralize repository consumes less memory consumption while distributed locality-based repository reduce network complexity issues than centralize repository in state-of-the-art Big Data Solution.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2003.05a
• /
• pp.473-476
• /
• 2003

Nowaday, Internet is doing the role of a large distributed information service tenter and various information and database servers managing it are in distributed network environment. However, the we have several difficulties in deciding the server to disposal input data depending on data properties. In this paper, we designed server mining mechanism and Intellectual data mining system architecture for the best efficiently dealing with input data pattern by using neural network among the various data in distributed environment. As a result, the new input data pattern could be operated after deciding the destination server according to dynamic binding method implemented by neural network. This mechanism can be applied Datawarehous, telecommunication and load pattern analysis, population census analysis and medical data analysis.

• International Journal of Internet, Broadcasting and Communication
• /
• v.14 no.2
• /
• pp.192-198
• /
• 2022

Companies are building and utilizing their own data analysis systems according to business characteristics in the distributed cloud. However, as businesses and data types become more complex and diverse, the demand for more efficient analytics has increased. In response to these demands, in this paper, we propose an unsupervised learning-based data analysis agent to which reinforcement learning is applied for effective data analysis. The proposal agent consists of reinforcement learning processing manager and unsupervised learning manager modules. These two modules configure an agent with k-means clustering on multiple nodes and then perform distributed training on multiple data sets. This enables data analysis in a relatively short time compared to conventional systems that perform analysis of large-scale data in one batch.

• International Journal of Internet, Broadcasting and Communication
• /
• v.16 no.2
• /
• pp.119-126
• /
• 2024

The volume of genomic data is constantly increasing in various modern industries and research fields. This growth presents new challenges and opportunities in terms of the quantity and diversity of genetic data. In this paper, we propose a distributed cloud system for integrating and managing large-scale gene databases. By introducing a distributed data storage and processing system based on the Hadoop Distributed File System (HDFS), various formats and sizes of genomic data can be efficiently integrated. Furthermore, by leveraging Spark on YARN, efficient management of distributed cloud computing tasks and optimal resource allocation are achieved. This establishes a foundation for the rapid processing and analysis of large-scale genomic data. Additionally, by utilizing BigQuery ML, machine learning models are developed to support genetic search and prediction, enabling researchers to more effectively utilize data. It is expected that this will contribute to driving innovative advancements in genetic research and applications.

• The Korean Journal of Applied Statistics
• /
• v.27 no.7
• /
• pp.1171-1185
• /
• 2014

Rhipe that integrates R and Hadoop environment, made it possible to process and analyze massive amounts of data using a distributed processing environment. In this paper, we implemented multiple regression analysis using Rhipe with various data sizes of actual data and simulated data. Experimental results for comparing the computing speeds of pseudo-distributed and fully-distributed modes for configuring Hadoop cluster, showed fully-distributed mode was more fast than pseudo-distributed mode and computing speeds of fully-distributed mode were faster as the number of data nodes increases. We also compared the performance of our Rhipe with stats and biglm packages available on bigmemory. The results showed that our Rhipe was more fast than other packages owing to paralleling processing with increasing the number of map tasks as the size of data increases.

• The KIPS Transactions:PartA
• /
• v.18A no.4
• /
• pp.123-128
• /
• 2011

As the global grid has grown in size, large-scale distributed data analysis schemes have gained momentum. Over the last few years, a number of methods have been introduced for allocating data intensive tasks across distributed and heterogeneous computing platforms. However, these approaches have a limited potential for scaling up computing nodes so that they can serve more tasks simultaneously. This paper tackles the scalability and communication delay for computing nodes. We propose a distributed data node for storing and allocating the data. This paper also provides data provisioning method based on the steady states for minimizing the communication delay between the data source and the computing nodes. The experimental results show that scalability and communication delay can be achieved in our system.

• International Journal of CAD/CAM
• /
• v.3 no.1_2
• /
• pp.51-60
• /
• 2003

The growth of the World Wide Web and the advances in high-speed network access have greatly changed existing CAD/CAE environment. The WWW has enabled us to share various distributed product data and to collaborate in the design process. An international standard for the product model data, STEP, and a standard for the distributed object technology, CORBA, are very important technological components for the interoperability in the advanced design and manufacturing environment. These two technologies provide background for the sharing of product data and the integration of applications on the network. This paper describes a distributed CAD/CAE environment that is integrated on the network by CORBA and product model data standard STEP. Several prototype application modules were implemented to verify the proposed concept and the test result is discussed. Finite element analysis server are further distributed into several frontal servers for the implementation of distributed parallel solution of finite element system equations. Distributed computation of analysis server is also implemented by using CORBA for the generalization of the proposed method.

• Spatial Information Research
• /
• v.19 no.4
• /
• pp.55-71
• /
• 2011

Recently, interest in cloud computing which provides IT resources as service form in IT field is increasing. As a result, much research has been done on the distributed data processing that store and manage a large amount of data in many servers. Meanwhile, in order to effectively utilize the spatial data which is rapidly increasing day by day with the growth of GIS technology, distributed processing of spatial data using cloud computing is essential. Therefore, in this paper, we review the representative distributed data processing techniques and we analyze the optimization requirements for performance improvement of the distributed processing techniques for a large amount of data. In addition, we uses the Hadoop and we evaluate the performance of the distributed data processing techniques for their optimization requirements.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.12
• /
• pp.4508-4530
• /
• 2021

Today distributed data storage service are being widely used. However lack of proper means of security makes the user data vulnerable. In this work, we propose a Randomized Block Size (RBS) model for secure data storage in distributed environments. The model work with multifold block sizes encrypted with the Chinese Remainder Theorem-based RSA (C-RSA) technique for end-to-end security of multimedia data. The proposed RBS model has a key generation phase (KGP) for constructing asymmetric keys, and a rand generation phase (RGP) for applying optimal asymmetric encryption padding (OAEP) to the original message. The experimental results obtained with text and image files show that the post encryption file size is not much affected, and data is efficiently encrypted while storing at the distributed storage server (DSS). The parameters such as ciphertext size, encryption time, and throughput have been considered for performance evaluation, whereas statistical analysis like similarity measurement, correlation coefficient, histogram, and entropy analysis uses to check image pixels deviation. The number of pixels change rate (NPCR) and unified averaged changed intensity (UACI) were used to check the strength of the proposed encryption technique. The proposed model is robust with high resilience against eavesdropping, insider attack, and chosen-plaintext attack.

• The Journal of Bigdata
• /
• v.4 no.2
• /
• pp.73-83
• /
• 2019

Since the emergence of the fourth industrial revolution, data analysis is being conducted in various fields. Distributed data processing has already become essential for the fast processing of large amounts of data. However, in the defense sector, simulation used cannot fully utilize the unstructured data which are prevailing at real environments. In this study, we propose a distributed data processing platform that can be applied to battalion level simulation models to provide visualized data for command decisions during training. 500,000 data points of strategic game were analyzed. Considering the winning factors in the data, variance processing was conducted to analyze the data for the top 10% teams. With the increase in the number of nodes, the model becomes scalable.