• Title/Summary/Keyword: Big Data Structure

Search Result 383, Processing Time 0.028 seconds

An Analysis of Big Data Structure Based on the Ecological Perspective (생태계 관점에서의 빅데이터 활성화를 위한 구조 연구)

  • Cho, Jiyeon;Kim, Taisiya;Park, Keon Chul;Lee, Bong Gyou
    • Journal of Information Technology Services
    • /
    • v.11 no.4
    • /
    • pp.277-294
    • /
    • 2012
  • The purpose of this research is to analyze big data structure and various objects in big data industry based on ecological perspective. Big data is rapidly emerging as a highly valuable resource to secure competitiveness of enterprise and government. Accordingly, the main issues in big data are to find ways of creating economic value and solving various problems. However big data is not systematically organized, and hard to utilize as it constantly expands to related industry such as telecommunications, finance and manufacturing. Under this circumstance, it is crucial to understand range of big data industry and to which stakeholders are related. The ecological approach is useful to understand comprehensive industry structure. Therefore this study aims at confirming big data structure and finding issues from interaction among objects. Results of this study show main framework of big data ecosystem including relationship among object elements composing of the ecosystem. This study has significance as an initial study on big data ecosystem. The results of the study can be useful guidelines to the government for making systemized big data ecosystem and the entrepreneur who is considering launching big data business.

Big data-based piping material analysis framework in offshore structure for contract design

  • Oh, Min-Jae;Roh, Myung-Il;Park, Sung-Woo;Chun, Do-Hyun;Myung, Sehyun
    • Ocean Systems Engineering
    • /
    • v.9 no.1
    • /
    • pp.79-95
    • /
    • 2019
  • The material analysis of an offshore structure is generally conducted in the contract design phase for the price quotation of a new offshore project. This analysis is conducted manually by an engineer, which is time-consuming and can lead to inaccurate results, because the data size from previous projects is too large, and there are so many materials to consider. In this study, the piping materials in an offshore structure are analyzed for contract design using a big data framework. The big data technologies used include HDFS (Hadoop Distributed File System) for data saving, Hive and HBase for the database to handle the saved data, Spark and Kylin for data processing, and Zeppelin for user interface and visualization. The analyzed results show that the proposed big data framework can reduce the efforts put toward contract design in the estimation of the piping material cost.

Estimation of Material Requirement of Piping Materials in an Offshore Structure using Big Data Analysis (빅데이터 분석을 이용한 해양 구조물 배관 자재의 소요량 예측)

  • Oh, Min-Jae;Roh, Myung-Il;Park, Sung-Woo;Kim, Seong-Hoon
    • Journal of the Society of Naval Architects of Korea
    • /
    • v.55 no.3
    • /
    • pp.243-251
    • /
    • 2018
  • In the shipyard, a lot of data is generated, stored, and managed during design, construction, and operation phases to build ships and offshore structures. However, it is difficult to handle such big data efficiently using existing data-handling technologies. As the big data technology is developed, the ship and offshore industries start to focus on the existing big data to find valuable information from it. In this paper, the material requirement estimation method of offshore structure piping materials using big data analysis is proposed. A big data platform for the data analysis in the shipyard is introduced and it is applied to the analysis of material requirement estimation to solve the problems in piping design by a designer. The regression model is developed from the big data of piping materials and verified using the existing data. This analysis can help a piping designer to estimate the exact amount of material requirement and schedule the purchase time.

Big Data Key Challenges

  • Alotaibi, Sultan
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.4
    • /
    • pp.340-350
    • /
    • 2022
  • The big data term refers to the great volume of data and complicated data structure with difficulties in collecting, storing, processing, and analyzing these data. Big data analytics refers to the operation of disclosing hidden patterns through big data. This information and data set cloud to be useful and provide advanced services. However, analyzing and processing this information could cause revealing and disclosing some sensitive and personal information when the information is contained in applications that are correlated to users such as location-based services, but concerns are diminished if the applications are correlated to general information such as scientific results. In this work, a survey has been done over security and privacy challenges and approaches in big data. The challenges included here are in each of the following areas: privacy, access control, encryption, and authentication in big data. Likewise, the approaches presented here are privacy-preserving approaches in big data, access control approaches in big data, encryption approaches in big data, and authentication approaches in big data.

Research on big data curriculum in university suitable for the era of the 4th industrial revolution (4차 산업혁명 시대에 적합한 빅데이터 대학 교육과정 연구)

  • Choi, Hun;Kim, Gimun
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.24 no.11
    • /
    • pp.1562-1565
    • /
    • 2020
  • With the development of digital technology, the industrial structure is becoming digitalize. The government selected big data as the key technology of the 4th industrial revolution. Among them, big data is widely used to create new values and services by utilizing vast amounts of information. In order to cultivate professional manpower for the use of big data, various education programs are provided at universities. We intend to develop a curriculum for systematic training of talented people who can acquire knowledge about the three stages of collection, analysis, and application of big data. To this end, subjects are classified into basic competency, technical competency, analysis competency, and business competency based on the big data competency model proposed by the Korea Internet & Security Agency.

Knowledge Creation Structure of Big Data Research Domain (빅데이터 연구영역의 지식창출 구조)

  • Namn, Su-Hyeon
    • Journal of Digital Convergence
    • /
    • v.13 no.9
    • /
    • pp.129-136
    • /
    • 2015
  • We investigate the underlying structure of big data research domain, which is diversified and complicated using bottom-up approach. For that purpose, we derive a set of articles by searching "big data" through the Korea Citation Index System provided by National Research Foundation of Korea. With some preprocessing on the author-provided keywords, we analyze bibliometric data such as author-provided keywords, publication year, author, and journal characteristics. From the analysis, we both identify major sub-domains of big data research area and discover the hidden issues which made big data complex. Major keywords identified include SOCIAL NETWORK ANALYSIS, HADOOP, MAPREDUCE, PERSONAL INFORMATION POLICY/PROTECTION/PRIVATE INFORMATION, CLOUD COMPUTING, VISUALIZATION, and DATA MINING. We finally suggest missing research themes to make big data a sustainable management innovation and convergence medium.

Is Big Data Analysis to Be a Methodological Innovation? : The cases of social science (빅데이터 분석은 사회과학 연구에서 방법론적 혁신인가?)

  • SangKhee Lee
    • The Journal of the Convergence on Culture Technology
    • /
    • v.9 no.3
    • /
    • pp.655-662
    • /
    • 2023
  • Big data research plays a role of supplementing existing social science research methods. If the survey and experimental methods are somewhat inaccurate because they mainly rely on recall memories, big data are more accurate because they are real-time records. Social science research so far, which mainly conducts sample research for reasons such as time and cost, but big data research analyzes almost total data. However, it is not easy to repeat and reproduce social research because the social atmosphere can change and the subjects of research are not the same. While social science research has a strong triangular structure of 'theory-method-data', big data analysis shows a weak theory, which is a serious problem. Because, without the theory as a scientific explanation logic, even if the research results are obtained, they cannot be properly interpreted or fully utilized. Therefore, in order for big data research to become a methodological innovation, I proposed big thinking along with researchers' efforts to create new theories(black boxes).

A Method for Selective Storing and Visualization of Public Big Data Using XML Structure (XML구조를 이용한 공공 빅데이터의 선별 저장 및 시각화 방법)

  • Back, BongHyun;Ha, Il-Kyu
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.21 no.12
    • /
    • pp.2305-2311
    • /
    • 2017
  • In recent years, there have been tries to open public data from various government agencies along with publicization of public information for the public interest. In other words, various kinds of electronic data generated and collected by the public institutions as a result of their work are opened in the public portal sites. However, users who use it are limited in their use of big data due to lack of understanding of data format, lack of data processing knowledge, difficulty in accessing and managing data, and lack of visualization data to understand collected and stored data. Therefore, in this study, we propose a big data collection, storing and visualization platform that can collect big data provided by various public sites using data set URL and API regardless of data format, re-process collected data using XML structure.

A Strategy Study on Sensitive Information Filtering for Personal Information Protect in Big Data Analyze

  • Koo, Gun-Seo
    • Journal of the Korea Society of Computer and Information
    • /
    • v.22 no.12
    • /
    • pp.101-108
    • /
    • 2017
  • The study proposed a system that filters the data that is entered when analyzing big data such as SNS and BLOG. Personal information includes impersonal personal information, but there is also personal information that distinguishes it from personal information, such as religious institution, personal feelings, thoughts, or beliefs. Define these personally identifiable information as sensitive information. In order to prevent this, Article 23 of the Privacy Act has clauses on the collection and utilization of the information. The proposed system structure is divided into two stages, including Big Data Processing Processes and Sensitive Information Filtering Processes, and Big Data processing is analyzed and applied in Big Data collection in four stages. Big Data Processing Processes include data collection and storage, vocabulary analysis and parsing and semantics. Sensitive Information Filtering Processes includes sensitive information questionnaires, establishing sensitive information DB, qualifying information, filtering sensitive information, and reliability analysis. As a result, the number of Big Data performed in the experiment was carried out at 84.13%, until 7553 of 8978 was produced to create the Ontology Generation. There is considerable significan ce to the point that Performing a sensitive information cut phase was carried out by 98%.

Renewable energy trends and relationship structure by SNS big data analysis (SNS 빅데이터 분석을 통한 재생에너지 동향 및 관계구조)

  • Jong-Min Kim
    • Convergence Security Journal
    • /
    • v.22 no.1
    • /
    • pp.55-60
    • /
    • 2022
  • This study is to analyze trends and relational structures in the energy sector related to renewable energy. For this reason, in this study, we focused on big data including SNS data. SNS utilizes the Instagram platform to collect renewable energy hash tags and use them as a word embedding method for big data analysis and social network analysis, and based on the results derived from this research, it will be used for the development of the renewable energy industry. It is expected that it can be utilized.