Privacy Enhanced Data Security Mechanism in a Large-Scale Distributed Computing System for HTC and MTC

Rho, Seungwoo;Park, Sangbae;Hwang, Soonwook

  • Received : 2016.04.06
  • Accepted : 2016.05.30
  • Published : 2016.06.28


We developed a pilot-job based large-scale distributed computing system to support HTC and MTC, called HTCaaS (High-Throughput Computing as a Service), which helps scientists solve large-scale scientific problems in areas such as pharmaceutical domains, high-energy physics, nuclear physics and bio science. Since most of these problems involve critical data that affect the national economy and activate basic industries, data privacy is a very important issue. In this paper, we implement a privacy enhanced data security mechanism to support HTC and MTC in a large-scale distributed computing system and show how this technique affects performance in our system. With this mechanism, users can securely store data in our system.


HTC;MTC;Privacy;Security;Pilot Job;HTCaaS;Distributed Computing


  1. I. Raicu, I. Foster, and Y. Zhao, "Many-Task Computing for Grids and Supercomputers," Proc. Workshop on Many-Task Computing on Grids and Supercomputers, 2008, pp. 1-11.
  2. Jik-Soo Kim, Seungwoo Rho, Seoyoung Kim, Sangwan Kim, Seokkyoo Kim, and Soonwook Hwang, "HTCaaS: Leveraging Distributed Supercomputing Infrastructures for Large-Scale Scientific Computing," Proc. Workshop on Many-Task Computing on Clouds, Grids, and Supercomputers, 2013.
  3. R. Henderson and D. Tweten, “Portable Batch System: External reference specification,” Technical report, NASA Ames Research Center, 1996.
  4. A. Prenneis, Jr, "LoadLeveler: Workload Management for Parallel and Distributed Computing Environments," Proc. of Supercomputing Europe, 1996.
  5. HTCondor, http://research.cs, 2016.
  6. A. Luckow, M. Santcroos, O. Weider, A. Merzky, S. Maddineni, and S. Jha, "Towards a common model for pilot-jobs," Proc. of The International ACM Symposium on High-Performance Parallel and Distributed Computing, 2012.
  7. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, and M.Wilde, "Falkon: a Fast and Light-weight tasK executiON framework," Proc. ACM/IEEE conference on Supercomputing, 2007, pp. 1-12.
  8. A. Luckow, L. Lacinski, and S. Jha, "SAGA BigJob: An Extensible and Interoperable Pilot-Job Abstraction for Distributed Applications and Systems," Proc. IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, 2010, pp. 135-144.
  9. E. Walker, J. P. Gardner, V. Litvin, and E. L. Turner, "Creating Personal Adaptive Clusters for Managing Scientific Jobs in a Distributed Computing Environment," Proc. Challenges of Large Applications in Distributed Environments, 2006, pp. 95-103.
  10. A. Tsaregorodtsev, M. Bargiotti, N. Brook, A. C. Ramo, G. Castellani, P. Charpentier, C. Cioffi, J. Closier, R. G. Diaz, G. Kuznetsov, Y. Y. Li, R. Nandakumar, S. Paterson, R. Santinelli, A. C. Smith, M. S. Miguelez, and S. G. Jimenez, “DIRAC: a community grid solution,” Journal of Physics: Conference Series, 2008.
  11. KEGG,, 2016.
  12. Lang PT, Brozell SR, Mukherjee S, Pettersen EF, Meng EC, Thomas V, "DOCK 6: combining techniques to model RNA-small molecule complexes," RNA, 2009.
  13. Open Grid Forum Job Submission Description Language, documents/GFD.56.pdf, 2016.
  14. Partnership & Leadership for the nationwide Supercomputing Infrastructure,, 2016.
  15. O. Trott and A. J. Olson, “AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization and multithreading,” Journal of Computational Chemistry, vol. 31, 2010, pp. 455-461.

Cited by

  1. E-commerce big data computing platform system based on distributed computing logistics information pp.1573-7543, 2018,


Grant : 자연보전 정책대응기술

Supported by : 한국과학기술정보연구원