• Title/Summary/Keyword: hash table

Search Result 137, Processing Time 0.021 seconds

Improvement of DHP Association Rules Algorithm for Perfect Hashing (완전해싱을 위한 DHP 연관 규칙 탐사 알고리즘의 개선 방안)

  • 이형봉
    • Journal of KIISE:Databases
    • /
    • v.31 no.2
    • /
    • pp.91-98
    • /
    • 2004
  • DHP mining association rules algorithm maintains previously independent direct hash table to reduce the sire of hash tree containing the frequency number of each candidate large itemset. It performs pruning by using the direct hash table when the hash tree is constructed. The mort large the size of direct hash table increases, the higher the effort of pruning becomes. Especially, the effect of pruning in phase 2 which generate 2-large itemsets is so high that it dominates the overall performance of DHP algorithm. So, following the speedy trends of producing VLM(Very Large Memory) systems, extreme increment of direct hash table size is being tried and one of those trials is perfect hash table in phase 2. In case of using perfect hash table in phase 2, we found that some rearrangement of DHP algorithm got about 20% performance improvement compared to simply |H$_2$| reconfigured DHP algorithm. In this paper, we examine the feasibility of perfect hash table in phase 2 and propose PHP algorithm, a rearranged DHP algorithm, which uses the characteristics of perfect hash table sufficiently, then make an analysis on the results in experimental environment.

Concurrent Hash Table Optimized for NUMA System (NUMA 시스템에 최적화된 병렬 해시 테이블)

  • Choi, JaeYong;Jung, NaiHoon
    • Journal of Korea Game Society
    • /
    • v.20 no.5
    • /
    • pp.89-98
    • /
    • 2020
  • In MMO game servers, NUMA (Non-Uniform Memory Access) architecture is generally used to achieve high performance. Furthermore, such servers normally use hash tables as internal data structure which have constant time complexity for insert, delete, and search operations. In this study, we proposed a concurrent hash table optimized for NUMA system to make MMO game servers improve their performance. We tested our hash table on 4 socket NUMA system, and the hash table shows at most 100% speedup over another high-performance hash table.

Approaches for Improving Bloom Filter-Based Set Membership Query

  • Lee, HyunYong;Lee, Byung-Tak
    • Journal of Information Processing Systems
    • /
    • v.15 no.3
    • /
    • pp.550-569
    • /
    • 2019
  • We propose approaches for improving Bloom filter in terms of false positive probability and membership query speed. To reduce the false positive probability, we propose special type of additional Bloom filters that are used to handle false positives caused by the original Bloom filter. Implementing the proposed approach for a routing table lookup, we show that our approach reduces the routing table lookup time by up to 28% compared to the original Bloom filter by handling most false positives within the fast memory. We also introduce an approach for improving the membership query speed. Taking the hash table-like approach while storing only values, the proposed approach shows much faster membership query speed than the original Bloom filter (e.g., 34 times faster with 10 subsets). Even compared to a hash table, our approach reduces the routing table lookup time by up to 58%.

An Improved Signature Hashing Algorithm for High Performance Network Intrusion Prevention System (고성능 네트워크 침입방지시스템을 위한 개선된 시그니처 해싱 알고리즘)

  • Ko, Joong-Sik;Kwak, Hu-Keun;Wang, Jeong-Seok;Kwon, Hui-Ung;Chung, Kyu-Sik
    • The KIPS Transactions:PartC
    • /
    • v.16C no.4
    • /
    • pp.449-460
    • /
    • 2009
  • The signature hashing algorithm[9] provides the fast pattern matching speed for network IPS(Intrusion Prevention System) using the hash table. It selects 2 bytes from all signature rules and links to the hash table by the hash value. It has an advantage of performance improvement because it reduces the number of inspecting rules in the pattern matching. However it has a disadvantage of performance drop if the number of rules with the same hash value increases when the number of rules are large and the corelation among rules is strong. In this paper, we propose a method to make all rules distributed evenly to the hash table independent of the number of rules and corelation among rules for overcoming the disadvantage of the signature hashing algorithm. In the proposed method, it checks whether or not there is an already assigned rule linked to the same hash value before a new rule is linked to a hash value in the hash table. If there is no assigned rule, the new rule is linked to the hash value. Otherwise, the proposed method recalculate a hash value to put it in other position. We implemented the proposed method in a PC with a Linux module and performed experiments using Iperf as a network performance measurement tool. The signature hashing method shows performance drop if the number of rules with the same hash value increases when the number of rules are large and the corelation among rules is strong, but the proposed method shows no performance drop independent of the number of rules and corelation among rules.

Data Cube Generation Method Using Hash Table in Spatial Data Warehouse (공간 데이터 웨어하우스에서 해쉬 테이블을 이용한 데이터큐브의 생성 기법)

  • Li, Yan;Kim, Hyung-Sun;You, Byeong-Seob;Lee, Jae-Dong;Bae, Hae-Young
    • Journal of Korea Multimedia Society
    • /
    • v.9 no.11
    • /
    • pp.1381-1394
    • /
    • 2006
  • Generation methods of data cube have been studied for many years in data warehouse which supports decision making using stored data. There are two previous studies, one is multi-way array algorithm and the other is H-cubing algorithm which is based on the hyper-tree. The multi-way array algorithm stores all aggregation data in arrays, so if the base data is increased, the size of memory is also grow. The H-cubing algorithm which is based on the hyper-tree stores all tuples in one tree so the construction cost is increased. In this paper, we present an efficient data cube generation method based on hash table using weight mapping table and record hash table. Because the proposed method uses a hash table, the generation cost of data cube is decreased and the memory usage is also decreased. In the performance study, we shows that the proposed method provides faster search operation time and make data cube generation operate more efficiently.

  • PDF

Fast URL Lookup Using URL Prefix Hash Tree (URL Prefix 해시 트리를 이용한 URL 목록 검색 속도 향상)

  • Park, Chang-Wook;Hwang, Sun-Young
    • Journal of KIISE:Information Networking
    • /
    • v.35 no.1
    • /
    • pp.67-75
    • /
    • 2008
  • In this paper, we propose an efficient URL lookup algorithm for URL list-based web contents filtering systems. Converting a URL list into URL prefix form and building a hash tree representation of them, the proposed algorithm performs tree searches for URL lookups. It eliminates redundant searches of hash table method. Experimental results show that proposed algorithm is $62%{\sim}210%$ faster, depending on the number of segment, than conventional hash table method.

An Efficient Hashing Mechanism of the DHP Algorithm for Mining Association Rules (DHP 연관 규칙 탐사 알고리즘을 위한 효율적인 해싱 메카니즘)

  • Lee, Hyung-Bong
    • The KIPS Transactions:PartD
    • /
    • v.13D no.5 s.108
    • /
    • pp.651-660
    • /
    • 2006
  • Algorithms for mining association rules based on the Apriori algorithm use the hash tree data structure for storing and counting supports of the candidate frequent itemsets and the most part of the execution time is consumed for searching in the hash tree. The DHP(Direct Hashing and Pruning) algorithm makes efforts to reduce the number of the candidate frequent itemsets to save searching time in the hash tree. For this purpose, the DHP algorithm does preparative simple counting supports of the candidate frequent itemsets. At this time, the DHP algorithm uses the direct hash table to reduce the overhead of the preparative counting supports. This paper proposes and evaluates an efficient hashing mechanism for the direct hash table $H_2$ which is for pruning in phase 2 and the hash tree $C_k$, which is for counting supports of the candidate frequent itemsets in all phases. The results showed that the performance improvement due to the proposed hashing mechanism was 82.2% on the maximum and 18.5% on the average compared to the conventional method using a simple mod operation.

Efficient Processing of Multidimensional Vessel USN Stream Data using Clustering Hash Table (클러스터링 해쉬 테이블을 이용한 다차원 선박 USN 스트림 데이터의 효율적인 처리)

  • Song, Byoung-Ho;Oh, Il-Whan;Lee, Seong-Ro
    • Journal of the Institute of Electronics Engineers of Korea SP
    • /
    • v.47 no.6
    • /
    • pp.137-145
    • /
    • 2010
  • Digital vessel have to accurate and efficient mange the digital data from various sensors in the digital vessel. But, In sensor network, it is difficult to transmit and analyze the entire stream data depending on limited networks, power and processor. Therefore it is suitable to use alternative stream data processing after classifying the continuous stream data. In this paper, We propose efficient processing method that arrange some sensors (temperature, humidity, lighting, voice) and process query based on sliding window for efficient input stream and pre-clustering using multiple Support Vector Machine(SVM) algorithm and manage hash table to summarized information. Processing performance improve as store and search and memory using hash table and usage reduced so maintain hash table in memory. We obtained to efficient result that accuracy rate and processing performance of proposal method using 35,912 data sets.

A Clustered Flash Translation Layer for Mobile Storage Systems (휴대용 저장장치 시스템을 위한 Clustered Flash Translation Layer)

  • Park, Kwang-Hee;Kim, Deok-Hwan
    • Journal of the Institute of Electronics Engineers of Korea SD
    • /
    • v.45 no.3
    • /
    • pp.94-100
    • /
    • 2008
  • It is necessary to develop the flash memory system software FTL(Flash Translation Layer) which is used in mobile storage like Compact Flash memory. In this paper, we design the FTL using clustered hash table and two phase software caching method to translate logical address into physical address fastly. The experimental results show that the address translation performance of CFTL is 13.3% higher than that of NFTL and 8% higher than that of AFTL, and the memory usage of CFTL is 75% smaller than that of AFTL.

Distributed Hash Table based Service Discovery in Mobile Ad Hoc Network (모바일 애드 혹 네트워크에서 분산 해쉬 테이블 기반의 서비스 탐색 기법)

  • Jung, Jae-Hoon;Lee, Seung-Hak;Kim, Nam-Gi;Yoon, Hyn-Soo
    • Journal of KIISE:Information Networking
    • /
    • v.35 no.1
    • /
    • pp.91-97
    • /
    • 2008
  • In order to get a desired service in such environments, we need a service discovery method for discovering a device providing that service. In this paper, we propose a service discovery protocol which is based on DHTs (Distributed Hash Tables) to solve these problems. Our protocol is scalable since it does not require a central lookup server and does not rely on multicast or flooding. Simulation results show that our protocol is scalable and outperforms existing service discovery protocols.