隨著網路的發展，人們對網路的依賴越來越深，許多桌上型應用程式慢慢轉移到網路環境上的應用程式，其中包括文書處理、行事曆、相簿管理、甚至是應用程式開發都可以在網路上面進行。Google 正是一個提供網路服務的網路公司，快速的搜尋服務以及之後發展的電子郵件信箱服務都被大眾所稱讚，他們靠的是快速的反應時間，大量的資料儲存空間來吸引許多使用者，並且提供其它廠商刊登廣告來賺取收費；另外一個知名的社群網站 Facebook，也是眾所皆知大型網站之一，能夠即時的處理龐大的社交訊息，處理不同使用者之間關連性的推薦，如此龐大的資料收集及運算靠的就是當今最火紅的技術「雲端計算」。
雲端計算使用分散式儲存空間與分散式運算技術來達到快速處理反應，由於這是個新的技術，目前看到的應用有限，例如字串搜尋分析、Log 檔案分析等等；有鑑於此，本論文將雲端計算應用在不同的領域方面，如封包儲存統計分析與封包還原，將大量的封包儲存在分散式檔案系統當中，藉由不同的需求來對這些封包做處理，以達到類似 IPS (Intrusion Prevention System) 的效果。

With the growing of Internet, people use network frequently. Many PC applications have moved to the network-based environment, such as text processing, calendar, photo management, and even user can develop applications on the network. Google is a company providing web services. Its popular services are search engine and gmail which attracts people by short response time and lots amount of data storage. It also charges businesses to place their own advertisements. Another hot social network is Facebook which is also a popular website. It processes huge instant messages and social relationships between users. The magic power of doing this depends on the new technique, Could Computing.
Cloud computing has ability to keep high-performance processing and short response time, and its kernel components are distributed data storage and distributed data processing. Because of the new concept, there are fewer application, such as pattern searching and log file analysis, related to the cloud computing. Therefore, we use the technique to perform the packet analysis and packet dissection. The packet data are placed by distributed file system, and further process according to different requirements, which acts as IPS (Intrusion Protection System).

致謝 II
中文摘要 III
Abstract IV
目次 V
圖目錄 VIII
表目錄 IX
1. 序論 1
1.1. 研究動機與目的 1
1.2. 論文架構 2
2. Hadoop概論與介紹 3
2.1. 雲端計算簡介 3
2.2. Hadoop簡介 4
2.3. Hadoop安裝與測試 7
2.3.1. Local(Standalone) Mode 8
2.3.2. Pseudo-Distributed Mode 8
2.3.3. Fully-Distributed Mode 10
2.4. 設定檔 11
2.5. Hadoop Daemons 14
3. HDFS 17
3.1. HDFS的設計 17
3.2. HDFS的概念 20
3.2.1. 區塊 20
3.2.2. Namenode and Datanodes 21
3.3. DataFlow 23
3.3.1. 從HDFS讀取資料 23
3.3.2. 從HDFS寫入資料 25
3.4. HDFS檔案型態 27
4. MapReduce 29
4.1. MapReduce介紹 29
4.2. Data Flow 30
4.3. MapReduce運作 35
4.4. MapReduce失敗處理 39
4.5. 排程 41
4.5.1. 工作排程 41
4.5.2. 公平演算法 41
4.6. Shuffle and Sort 42
4.7. 型態與格式 44
4.7.1. MapReduce型態 44
4.7.2. MapReduce格式 46
4.8. MapReduce結論 50
5. HBase 51
5.1. HBase簡介 51
5.2. Data Model 52
5.3. HBase versus RDBMS 55
6. 設計與實作 57
6.1. 軟體介紹 57
6.2. 環境架構 58
6.3. 封包擷取策略 60
6.4. 封包流量分析 61
6.5. 封包還原 62
6.5.1. 封包還原概念 62
6.5.2. HTTP 62
6.5.3. Problem and Solvent 64
6.5.4. Algorithm 64
6.5.5. Table Schema 65
6.6. 實驗結果與分析 66
7. 結論 71
參考文獻 72

[1] Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung, “The Google File System”, 19th ACM Symposium on Operating Systems Principles, Lake George, NY, October, 2003.
[2] Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber, “Bigtable: A Distributed Storage System for Structure Data”, OSDI, 2006
[3] Dhruba Borthakur. “The Hadoop Distributed File System: Architecture and Design”, 2007
[4] Tom White. “Hadoop: The Definitive Guide”, O’Reilly Media, Inc., 2009
[5] Cloud Computing Wikipedia, http://en.wikipedia.org/wiki/Cloud_computing
[6] R. Fielding, UC Irvine, J. Gettys, Compaq/W3C, J. Mogul, Compaq, H. Frystyk, W3C/MIT, L. Masinter, Xerox, P. Leach, Microsoft, T. Berners-Lee, W3C/MIT, “Hypertext Transfer Protocol – HTTP/1.1”, RFC 2616, June 1999
[7] Hypertext Transfer Protocol Wikipedia, http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol
[8] MIME Wikipedia, http://en.wikipedia.org/wiki/MIME
[9] W. Richard Stevens, Bill Fenner, Andrew M. Rudoff, UNIXR Network Programming Volume 1, Third Edition: The Sockets Networking API, Addison Wesley, November 2003.
[10] Apache Hadoop Project, http://hadoop.apache.org/
[11] Apache HBase Project, http://hbase.apache.org/
[12] NCHC Cloud Computing Research Group, http://trac.nchc.org.tw/cloud/
[13] Jpcap, a Java library for capturing and sending network packets, http://netresearch.ics.uci.edu/kfujii/jpcap/doc/
[14] Open Flash Chart project, http://teethgrinder.co.uk/open-flash-chart-2/
[15] Hadoop Wiki, http://wiki.apache.org/hadoop/
[16] Hadoop API, http://hadoop.apache.org/common/docs/r0.20.2/api/index.html
[17] Hbase API, http://hbase.apache.org/docs/current/api/index.html
[18] Java SDK Ed. 6 API, http://java.sun.com/javase/6/docs/api/