隨著Big Data時代的來臨，Volume、Variety、Velocity，Big Data三大訴求使雲端運算面臨新的挑戰。因應Big Data Analytics的使用需求，以分散式運算叢集處理巨量資料是一大趨勢。本論文以探討現今雲端運算平台提供的分散式運算叢集之運算效能為出發點，發現用於天文、大氣、分子化學、物理頻譜…等科學運算的高效能運算工具″訊息傳遞介面(MPI)運算叢集″，因雲端運算平台提供較少相關的整合服務，導致MPI運算叢集無法有效率地調度雲端運算資源並發揮其高運算效能的特色。本論文欲設計可有效率調度雲端運算平台運算資源的MPI分散式運算叢集架構。我們打破傳統MPI分散式運算叢集環境建置的限制，以Socket Interface-TCP/IP Server & Client建立Communication System，作為此架構的傳遞訊息管道，以及設計Group Manager簡化系統運算資源管理機制。以分離的方式彈性地掌控作業系統運算資源，使MPI運算叢集可良好地運作於運算資源虛擬化且彈性化的雲端運算平台。為了能使MPI運算叢集能更容易的在雲端運算平台上彈性地調度運算資源，我們參考Chiu 和Huang提出的Kernel Distributed Computing Management(KDCM)，借助KDCM統一管理及分配運算資源的能力，提供MPI運算叢集調度運算資源的途徑。以使MPI運算叢集運行於Linux核心驅動程式，並裝載於KDCM為目標，我們命名設計的MPI分散式運算叢集架構為MPI Kernel Cluster(MPIKC)。MPIKC與KDCM緊密的配合，達到有效率的調度運算資源，且發揮MPI運算叢集高運算效能的特色，並且提供作業系統即雲端運算環境的雲端運算平台。最後，我們實測驗證MPIKC on KDCM的運作正確性，並使用MPIKC進行高運算量的分散式運算，在Sum of Absolute Difference演算法及K-means clustering演算法的驗證結果，得到每多使用一個運算單位即提升0.5~1倍運算效能。證實MPIKC為可良好的配合雲端運算平台，且發揮高運算效能之優點的分散式運算叢集。
關鍵字：雲端運算、分散式運算叢集、Big Data、MPI、核心驅動程式

With the age of Big Data coming, the three defining characteristics of Big Data--Volume, variety and Velocity, make Cloud Computing facing new challenges. In response to the demand of Big Data analytics,Using distributed computing cluster to process vast amounts of data is a megatrend .In this paper ,as the outset,we discuss the performance of distributed computing clusters provided by the current cloud computing platforms. Found that for the Message-Passing Interface (MPI) cluster, which is used in Scientific Computing, such as astronomical, atmosphere, physical spectrum...etc., cloud computing platforms provide less relevant integration services. Which made MPI Cluster make inefficient use of cloud computing resources and unable to exert its high computing performance. Following, we propose The MPI Cluster
Architecture makes efficient use of cloud computing resources. To break the limitation of constructing traditional MPI Clusters computing environment, The MPI Cluster uses Socket Interface-TCP/IP Server & Client to build Communication System as the message-passing channel, and simplifies Operating System computing resources management mechanism by Group Manager. As the separated way described above,The MPI Cluster can resiliently grasp Operating System computing resources ,and work well on the cloud computing platform,that computing resources are virtualization and flexible. In order to make The MPI Cluster flexible dispatch computing resources on the cloud computing platform more easily, we adopt Kernel Distributed Computing Management (KDCM), proposed by Chiu and Huang. By its ability to unify manage and allocate computing resources, provides The MPI Cluster a path to flexible dispatch computing resources. As the goal of The MPI Cluster: Running in Linux kernel driver, and loading on KDCM, We name it MPI Kernel Cluster (MPIKC). As MPIKC and KDCM fit tightly ,they can efficient dispatch computing resources,exert its high computing performance,and provide Operating System is cloud computing environment on cloud computing platform. At the end, We verify the correctness of running MPIKC on KDCM, and use MPIKC to do distributed computing with high load, in the result of Sum of Absolute Difference and K-means clustering, each computing unit used, enhances a 0.5~1 times performance.We proved that MPIKC fits well with cloud computing platform, and exerts its high performance as the advantage of distributed computing cluster.

Keywords：Cloud Computing, Distributed Computing Cluster, Big Data, MPI, Kernel Driver

論文審定書 i
論文公開授權書 ii
誌 謝 iii
摘 要 iv
Abstract v
目　錄 vii
圖次 x
表次 xiii
第1章 簡介 1
1.1 研究動機 1
1.2 研究目的 2
1.3 論文架構 3
第2章 背景知識與相關研究 4
2.1 現有雲端運算平台介紹 4
2.1.1 Google Cloud Platform 4
2.1.2 OpenStack 5
2.1.3 Amazon Web Services 7
2.1.4 雲端平台概況分析 8
2.2 Kernel Distributed Computing Managment 10
2.2.1 Management Server 10
2.2.2 Computing Node 13
2.2.3 User Client 14
2.2.4 KDCM分析 14
2.3 分散式運算叢集 15
2.3.1 Apache Hadoop 15
2.3.2 Apache Spark 22
2.3.3 Message-Passing Interface 25
2.3.4 分散式運算叢集比較 32
2.4 MPI運算叢集環境 34
2.4.1 Process Manager 34
2.4.2 Process Manager Interface 35
2.4.3 MPI運算叢集環境總結 38
第3章 MPI雲端運算平台架構 39
3.1 MPIKC 39
3.1.1 MPIKC運作流程 41
3.1.2 MPIKC封包格式 43
3.2 MPI Group Manager 45
3.2.1 MPI Launcher 45
3.2.2 Cache Buffer 48
3.2.3 Communication System 49
3.3 MPI Library 51
3.3.1 Groups 51
3.3.2 Datatypes 55
3.3.3 Point to Point Communication 58
3.3.4 Collective Communication 62
3.4 MPIKC on KDCM 68
3.5 MPI雲端運算平台總結 71
第4章 實現成果與分析 72
4.1 MPIKC運算效能測試 72
4.1.1 基本MPI分配運算任務及切割資料方式 72
4.1.2 MPIKC測試環境 73
4.1.3 定積分逼近圓周率 74
4.1.4 Advance Encryption Standard (AES) 78
4.1.5 Sub of Absolute Difference (SAD) 80
4.1.6 K-means clustering 83
4.1.7 MPIKC運算效能測試總結 85
4.2 MPIKC on KDCM 驗證 86
第5章 結論 91
參考文獻 92

[1] Peter Mell, Timothy Grance, “The NIST definition of Cloud Computing”, NIST Special Publication 800-145, Sep 2011, 7 pages.
[2] Paul Zikopoulos, Chris Eaton,”Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data” McGraw-Hill Osborne Media, 2011.
[3] Andrew McAfee and Erik Bynjolfsson,”Big Data: The Management Revolution”, Harvard Business Review, Oct 2012, pp. 61-67.
[4] Changqing Ji,Yu Li,Wenming Qiu,Uchechukwu Awada,Keqiu Li,”Big Data Processing in Cloud Computing Environment”,12th International Symposium on Pervasive Systems, Algorithms and Networks ,Dec 2012,pp. 17-23.
[5] Google, Google Cloud Platform , Available：https://cloud.google.com/products/
[6] OpenStack , Available：http://www.openstack.org/software/
[7] Amazon Web Services , Available：https://aws.amazon.com/tw/
[8] Apache Hadoop 2.7.2 , Available：https://hadoop.apache.org/docs/r2.7.2/
[9] Wikipedia,Apache Hadoop , Available：https://en.wikipedia.org/wiki/Apache_Hadoop
[10] Apache Spark 1.6.2 , Available：http://spark.apache.org/docs/latest/index.html
[11] Wikipedia, Apache Spark , Available：https://en.wikipedia.org/wiki/Apache_Spark
[12] Matei Zaharia,Mosharaf Chowdhury,Tathagata Das,Ankur Dave,Justin Ma,Murphy McCauley,Michael J.Franklin,Scott Shenker,Ion Stoica, “Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing”, NSDI’12 Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation, 2012, pp. 2-2.
[13] HPC is dying ,and MPI is killing it , Avaialbe：http://www.dursi.ca/hpc-is-dying-and-mpi-is-killing-it/
[14] David F. Bacon, Susan L. Graham, Oliver J. Sharp,”Compiler Transformations for High-Performance Computing”, ACM Computing Surveys (CSUR) Volume 26 Issue 4, Dec 1994, pp. 345-420.
[15] Message-Passing Interface Standard Version3.1, Available：http://www.mpi-forum.org/docs/
[16] HIGH PERFORMANCE COMPUTING TRAINING, Available：https://computing.llnl.gov/?set=training&page=index#training_materials
[17] Jorge L.Reyes-Ortiz, Luca Oneto, Davide Anguita,”Big Data Analytics in the Cloud: Spark on Hadoop vs MPI/OpenP on Beowulf”, Procedia Computer ScienceVolume 53, 2015, pp.121-130.
[18] William Gropp, Ewing Lusk, Nathan Doss, Anthony Skjellum, “A high-performance, portable implementation of the MPI message passing interface standard”, Parallel Computing Volume 22 Issue 6, Sep 1996, pp. 789-828.
[19] Pavan Balaji,Darius Buntinas,David Goodell,William Gropp,Jayesh Krishna,Ewing Lusk,Rajeev Thakur, “PMI: A Scalable Parallel Process-Management Interface for Extreme-Scale Systems”, EuroMPI 2010, pp. 31-41.
[20] Nicholas T. Karonis, Brian Toonen, Ian Foster, “MPICH-G2: A Grid-enabled implementation of the Message Passing Interface”, Journal of Parallel and Distributed Computing Volume 63 Issue 5, May 2003, pp. 551-563.
[21] MPICH wiki, Available: https://wiki.mpich.org/mpich/index.php/Main_Page
[22] 邱日清,黃晟育,郭寶仁, “以核心程式設計方式實現分散式運算管理架構”, in Porc. 2015 全國計算機會議(NCS2015),Dec 2015 .
[23] Philip Russom, “Big Data Analytics”, TDWI BEST PRACTICES REPORT, Fourth Quarter 2011, pp. 1-35.