Distributed Indexing Methods for Moving Objects based on Spark Stream

Lee, Yunsou;Song, Seokil

  • Received : 2015.03.09
  • Accepted : 2015.03.25
  • Published : 2015.03.28


Generally, existing parallel main-memory spatial index structures to avoid the trade-off between query freshness and CPU cost uses light-weight locking techniques. However, still, the lock based methods have some limits such as thrashing which is a well-known problem in lock based methods. In this paper, we propose a distributed index structure for moving objects exploiting the parallelism in multiple machines. The proposed index is a lock free multi-version concurrency technique based on the D-Stream model of Spark Stream. The proposed method exploits the multiversion nature of D-Stream of Spark Streaming.


Moving Objects;Spark;Steaming;Index


  1. K. Kim, S. K. Cha, and K. Kwon, "Optimizing Multidimensional Index Trees for Main Memory Access," SIGMOD Rec., vol. 30, no. 2, 2001, pp. 139-150.
  2. L. Biveinis, S. Saltenis, and C. S. Jensen, "Main-memory Operation Buffering for Efficient R-tree Update," Proc. VLDB, 2007, pp. 591-602.
  3. J. Dittrich, L. Blunschi, and M. A. V. Salles, "Indexing Moving Objects using Short-lived Throwaway Indexes," Proc. SSTD, 2009, pp. 189-207.
  4. D. Sidlauskas, S. Saltenis, and C. S. Jensen, "Parallel Mainmemory Indexing for Moving-object Query and Update Workloads," Proc. ACM SIGMOD, 2012, pp. 37-48.
  5. M. Zaharia, et al, "Discretized Streams: An Efficient and Fault-tolerant Model for Stream Processing on Large Clusters." Proc. USENIX on Hot Topics in Cloud Computing, 2012, p. 10.
  6. A. Akdogan, C. Shahabi, and U. Demiryurek, "ToSS-it: A Cloud-Based Throwaway Spatial Index Structure for Dynamic Location Data", Proc. MDM, 2014, pp. 249-258.
  7. W. Lu, Y. Shen, S. Chen, and B. C. Ooi, "Efficient Processing of k Nearest Neighbor Joins using MapReduce," Proc. VLDB, 2012, pp. 1016-1027.
  8. C. Zhang, F. Li, and J. Jestes, "Efficient parallel kNN Joins for Large Data in MapReduce," Proc. EDBT, 2012, pp. 38-39.
  9. Z. Deng, X. Wu, L. Wang, X. Chen, R. R. Zomaya, and A. Dan Chen, "Parallel Processing of Dynamic Continuous Queries over Streaming Data Flows," IEEE Transactions on Parallel and Distributed Systems, vol. 26, issue 3, 2015 , pp. 834-846.
  10. C. S. Jensen, "GPU-Based Computing of Repeated Range Queries over Moving Objects," Proc. Euromicro International Conference on Parallel, Distributed and Network-Based Processing, 2014, pp. 640-647.
  11. F. Lettich, S. Orlando, C. Silvestri, and C. S. Jensen, "Manycore Processing of Repeated Range Queries over Massive Moving," CoRR, 2014.


Supported by : National Research Foundation of Korea (NRF)