• Title/Summary/Keyword: deduplication

Search Result 69, Processing Time 0.361 seconds

A Safe and Efficient Secure Data Deduplication for Cloud Storage Environment (클라우드 스토리지 환경을 위한 안전하고 효율적인 암호데이터 중복제거 기술)

  • Kim, Won-Bin;Lee, Im-Yeong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • /
    • pp.714-717
    • /
    • 2015
  • 기존의 암호데이터 중복제거 기술은 데이터의 중복 여부를 판단하기 위해 다양한 방식으로 데이터를 전송하고 이를 기존에 저장된 데이터와 비교하여 중복여부를 판단하게 된다. 이러한 데이터 중복제거 기술의 중복제거 효율성을 높이기 위해 최근 블록 단위의 중복제거 기술이 사용되고 있다. 하지만 블록 단위 중복제거 기술의 적용 과정에서 다양한 보안 위협이 발생하는데, 이 중 포이즌 어택은 무결성 및 데이터 저장 시 저장되는 데이터에 대한 검증이 이루어지지 않는 시스템에서 발생하는 위협 중 하나이다. 이러한 위협을 해결하기 위해 암호화 기술을 적용한 여러 기술들이 연구되어 제안되었지만 과도하게 많은 통신 횟수와 연산이 발생되어 효율성이 떨어지는 문제가 존재한다. 따라서 본 논문에서는 클라우드 스토리지에 저장되는 데이터의 기밀성과 무결성을 보장하며, 연산량과 통신량에서 보다 효율적인 암호데이터 중복제거 기술을 제안한다.

Hybrid Data Deduplication Method for reducing wear-level of SSD (SSD의 마모도 감소를 위한 복합적 데이터 중복 제거 기법)

  • Lee, Seung-Kyu;Yang, Yu-Seok;Kim, Deok-Hwan
    • Proceedings of the Korean Information Science Society Conference
    • /
    • /
    • pp.543-546
    • /
    • 2011
  • SSD는 일반적으로 사용되는 HDD와는 달리 기계적 동작이 없는 반도체 메모리를 사용하여 데이터를 저장하는 장치이다. 플래시 기반의 SSD는 읽기 성능이 뛰어난 반면 덮어쓰기 연산이 안되는 단점이 있다. 즉 마모도가 존재하여 SSD의 수명에 영향을 준다. 하지만 HDD보다 뛰어난 성능 때문에 노트북이나 중요한 데이터 등을 다루는 시스템 등에서 많이 사용하고 있다. 본 논문에서는 이러한 SSD를 서버 스토리지로 사용할 때 기존의 데이터 중복 제거 기법의 장점만을 조합한 복합적 데이터 중복 제거 기술을 제안하고 이 기법이 SSD의 마모도 측면에서 훨씬 효율적임을 검증하였다.

Backup System with Effective File Deduplication Mechanism (효율적인 중복제거 메커니즘을 적용한 백업 시스템)

  • Jeong, Ho-Min;Kim, Byung-Ki;Yoo, Jae-Bong;Kim, Jin;Ko, Young-Woong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • /
    • pp.1233-1236
    • /
    • 2007
  • 인터넷과 PC 사용의 증대로 개인 사용자, 소규모 그룹의 중요한 파일 백업의 필요성이 증가하고 있다. 그러나 상용 백업 소프트웨어의 규모나 비용이 개인 또는 소규모 그룹에 맞지 않는 문제점을 안고 있다. 본 논문에서는 오픈소스의 범용 시스템을 기반으로 한 백업 시스템을 네트워크를 통해 연결된 백업 서버에 자동으로 백업 및 복구를 해주는 시스템을 제안하고 구현하였다.

  • PDF

Indexing method with deduplication for efficient RDF data retrieving (효율적인 RDF 데이터 검색을 위한 중복 제거 색인 방법)

  • Jang, Hyeonggyu;Bang, Sungho;Oh, Sangyoon
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • /
    • pp.61-62
    • /
    • 2020
  • RDF의 활용이 증가하면서 RDF데이터를 저장하는 방법 또한 많은 연구가 이루어졌다. 그래프 형태인 RDF 데이터를 테이블로 바꿀 때, 동일한 데이터가 중복 저장되어 검색 시 불필요한 연산을 하는 문제점이 발생한다. 본 논문에서는 중복저장 및 불필요한 검색을 줄이기 위해 색인을 주어(S), 목적어(O) 색인과 이들의 중복 값을 별도의 색인을 만들고, 검색 시 중복 값을 확인하여 필요한 색인만 검색하는 기법을 제안한다. 실험에서 본 기법을 사용하여 불필요한 검색을 줄여서 전체적인 검색 시간이 줄어드는 것을 확인하였다.

  • PDF

Technical Trends and Patent Analysis for Reaming Bits (확공비트의 기술동향 및 특허분석)

  • Min, Kyung-Nam;Jung, Chan-Muk;Kim, Gyui-Woong
    • The Journal of Engineering Geology
    • /
    • v.23 no.2
    • /
    • pp.127-136
    • /
    • 2013
  • We analyzed the patent trends for whole drilling bits to reveal the main R&D directions, focusing on patents applied for and registered in Korea, the USA, Japan, and Europe. The technology was classified into two groups as a primary classification step (reaming systems and fixed systems), and into seven groups as a secondary classification step (sliding, odex, horizontal pivot, vertical pivot, concentric, eccentric, and etc.). A total of 33,614 patents were retrieved and 870 patents were selected for final effective analysis by data deduplication and filtering. A portfolio analysis using the correlation between the number of patents and the applicants for each patents revealed a sliding system as the key technology with greatest growth potential. From an analysis of the barriers to patents being granted, we emphasize the need to avoid similar topics existing patents or patent applications and to develop differential technology.

Document Analysis based Main Requisite Extraction System (문서 분석 기반 주요 요소 추출 시스템)

  • Lee, Jongwon;Yeo, Ilyeon;Jung, Hoekyung
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.23 no.4
    • /
    • pp.401-406
    • /
    • 2019
  • In this paper, we propose a system for analyzing documents in XML format and in reports. The system extracts the paper or reports of keywords, shows them to the user, and then extracts the paragraphs containing the keywords by inputting the keywords that the user wants to search within the document. The system checks the frequency of keywords entered by the user, calculates weights, and removes paragraphs containing only keywords with the lowest weight. Also, we divide the refined paragraphs into 10 regions, calculate the importance of the paragraphs per region, compare the importance of each region, and inform the user of the main region having the highest importance. With these features, the proposed system can provide the main paragraphs with higher compression ratio than analyzing the papers or reports using the existing document analysis system. This will reduce the time required to understand the document.

A Scheme on High-Performance Caching and High-Capacity File Transmission for Cloud Storage Optimization (클라우드 스토리지 최적화를 위한 고속 캐싱 및 대용량 파일 전송 기법)

  • Kim, Tae-Hun;Kim, Jung-Han;Eom, Young-Ik
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.37 no.8C
    • /
    • pp.670-679
    • /
    • 2012
  • The recent dissemination of cloud computing makes the amount of data storage to be increased and the cost of storing the data grow rapidly. Accordingly, data and service requests from users also increases the load on the cloud storage. There have been many works that tries to provide low-cost and high-performance schemes on distributed file systems. However, most of them have some weaknesses on performing parallel and random data accesses as well as data accesses of frequent small workloads. Recently, improving the performance of distributed file system based on caching technology is getting much attention. In this paper, we propose a CHPC(Cloud storage High-Performance Caching) framework, providing parallel caching, distributed caching, and proxy caching in distributed file systems. This study compares the proposed framework with existing cloud systems in regard to the reduction of the server's disk I/O, prevention of the server-side bottleneck, deduplication of the page caches in each client, and improvement of overall IOPS. As a results, we show some optimization possibilities on the cloud storage systems based on some evaluations and comparisons with other conventional methods.

Data Deduplication Method using PRAM Cache in SSD Storage System (SSD 스토리지 시스템에서 PRAM 캐시를 이용한 데이터 중복제거 기법)

  • Kim, Ju-Kyeong;Lee, Seung-Kyu;Kim, Deok-Hwan
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.50 no.4
    • /
    • pp.117-123
    • /
    • 2013
  • In the recent cloud storage environment, the amount of SSD (Solid-State Drive) replacing with the traditional hard disk drive is increasing. Management of SSD for its space efficiency has become important since SSD provides fast IO performance due to no mechanical movement whereas it has wearable characteristics and does not provide in place update. In order to manage space efficiency of SSD, data de-duplication technique is frequently used. However, this technique occurs much overhead because it consists of data chunking, hasing and hash matching operations. In this paper, we propose new data de-duplication method using PRAM cache. The proposed method uses hierarchical hash tables and LRU(Least Recently Used) for data replacement in PRAM. First hash table in DRAM is used to store hash values of data cached in the PRAM and second hash table in PRAM is used to store hash values of data in SSD storage. The method also enhance data reliability against power failure by maintaining backup of first hash table into PRAM. Experimental results show that average writing frequency and operation time of the proposed method are 44.2% and 38.8% less than those of existing data de-depulication method, respectively, when three workloads are used.

JMP+RAND: Mitigating Memory Sharing-Based Side-Channel Attack by Embedding Random Values in Binaries (JMP+RAND: 바이너리 난수 삽입을 통한 메모리 공유 기반 부채널 공격 방어 기법)

  • Kim, Taehun;Shin, Youngjoo
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.9 no.5
    • /
    • pp.101-106
    • /
    • 2020
  • Since computer became available, much effort has been made to achieve information security. Even though memory protection defense mechanisms were studied the most among of them, the problems of existing memory protection defense mechanisms were found due to improved performance of computer and new defense mechanisms were needed due to the advent of the side-channel attacks. In this paper, we propose JMP+RAND that embedding random values of 5 to 8 bytes per page to defend against memory sharing based side-channel attacks and bridging the gap of existing memory protection defense mechanism. Unlike the defense mechanism of the existing side-channel attacks, JMP+RAND uses static binary rewriting and continuous jmp instruction and random values to defend against the side-channel attacks in advance. We numerically calculated the time it takes for a memory sharing-based side-channel attack to binary adopted JMP+RAND technique and verified that the attacks are impossible in a realistic time. Modern architectures have very low overhead for JMP+RAND because of the very fast and accurate branching of jmp instruction using branch prediction. Since random value can be embedded only in specific programs using JMP+RAND, it is expected to be highly efficient when used with memory deduplication technique, especially in a cloud computing environment.