論文使用權限 Thesis access permission:校內校外完全公開 unrestricted
開放時間 Available:
校內 Campus: 已公開 available
校外 Off-campus: 已公開 available
論文名稱 Title |
雲端化智慧遠端監控管理平台之研發 The development of an intelligent, cloud-based remote monitoring management system |
||
系所名稱 Department |
|||
畢業學年期 Year, semester |
語文別 Language |
||
學位類別 Degree |
頁數 Number of pages |
83 |
|
研究生 Author |
|||
指導教授 Advisor |
|||
召集委員 Convenor |
|||
口試委員 Advisory Committee |
|||
口試日期 Date of Exam |
2012-09-14 |
繳交日期 Date of Submission |
2012-10-25 |
關鍵字 Keywords |
數據收集、分散式系統、線性迴歸、資料庫 K-means, Hadoop, Data collection, MapReduce, Distributed programming, Database, Linear regression |
||
統計 Statistics |
本論文已被瀏覽 5687 次,被下載 1871 次 The thesis/dissertation has been browsed 5687 times, has been downloaded 1871 times. |
中文摘要 |
在本論文中主要說明兩點:(1)如何應用MapReduce 在遠端傳 感器的溫度收集。(2)如何導入人工智慧使系統具有節能減碳的功 能。在過去的監控系統都是使用一台電腦收集所有傳感器的數值,隨 著資訊的爆炸,傳感器數量也越來越多,而使系統效能降低;如今, 使用雲端運算的技術,將傳感器平均分配到每一個map,當成每個 map 必須執行的任務,而且所有的map 任務都可以在多台電腦同時 運作與執行,以達到平行處理的目的,這種方法可以減少延遲時間, 提高數據收集的效率,特別是在傳感器數量龐大的情況下。本論文也 導入了線性迴歸、K-means 等方法來預測遠端傳感器的溫度。藉由預 測遠端傳感器的溫度值和系統可能發生異常狀況的時間點以達到節能 減碳的效果。 |
Abstract |
In this thesis, a data collection application based on MapReduce programming is described. This application aims to collect tempera- ture data stream continuously from a specied set of sensors. Instead of collecting the temperature information of all the sensors by one machine, the sensors are divided into several subsets each of which is handled as a Map task. In each Map task, the temperature data stream of the assigned sensors is collected continuously and stored in a predened database. All the Map tasks can run simultaneously on several machines. This method can reduce the delay time and improve the eciency of the data collection service, especially in the case of having a huge number of sensors monitored remotely by a data center through Internet. We can use the value of remote sensors to predict the next value of remote sensors by some methods such as linear regres- sion and K-means. And, we can use it to predict the system alarm. Experimental results show that the proposed method is eective in temperature data collection,and eective in carbon reduction. |
目次 Table of Contents |
Abstract i 摘要ii 目錄iii 第一章簡介 1 1.1 研究背景 1 1.2 研究目的 3 1.3 問題定義 4 1.4 論文架構 4 第二章文獻探討 6 2.1 Hadoop 6 2.1.1 Hadoop 文件讀取 9 2.1.2 Hadoop 文件寫入 10 2.1.3 MapReduce 11 2.1.4 Hadoop 執行MapReduce 13 2.2 資料探勘 15 2.3 類神經網路 16 2.4 群聚分析 17 2.5 相似度 18 2.6 線性迴歸 20 2.6.1 最小平方法 20 2.6.2 相關係數 21 2.6.3 信賴區間 22 2.7 高級加密標準 22 2.8 Modbus 23 第三章研究方法 26 3.1 研究動機 26 3.2 HAESModbus Model 29 3.2.1 HAESModbus 的輸入設計 30 3.2.2 HAESModbus 的MAP 設計 35 3.3 使用KLR 模組進行資料分析 40 3.3.1 機率密度函數的前處理 40 3.3.2 K-Means 的應用 42 3.3.3 線性回歸的分析 43 3.3.4 KLR 效能分析 43 3.4 系統異常預測 44 第四章實驗結果與分析 50 4.1 雲端運算遠端監控模組 50 4.2 線性迴歸溫度預測模組 56 4.2.1 Threshold 56 4.2.2 誤差總和效能比較 57 4.2.3 指定係數的關係比較 59 4.2.4 準確率和效能比較 61 4.3 異常警報模組 64 第五章結論和未來研究方向 66 5.1 結論 66 5.2 未來研究方向 67 參考文獻 68 圖目錄 圖1.1 遠端監控架構圖. . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 圖1.2 雲端運算環境之虛擬化技術. . . . . . . . . . . . . . . . 3 圖2.1 Hadoop 組成元件. . . . . . . . . . . . . . . . . . . . . . . . . . . 6 圖2.2 單一reduce task . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 圖2.3 多個reduce tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 圖2.4 沒有reduce task . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 圖2.5 Hadoop 資料讀取mode . . . . . . . . . . . . . . . . . . . . . 9 圖2.6 Hadoop 網路距離. . . . . . . . . . . . . . . . . . . . . . . . 10 圖2.7 Hadoop 資料寫入mode . . . . . . . . . . . . . . . . . . 11 圖2.8 Hadoop 如何去執行一個job . . . . . . . . . . . . . . . . 14 圖2.9 資料探勘相關領域技術. . . . . . . . . . . . . . . . . . . . 16 圖2.10 多層類神經網路示意圖. . . . . . . . . . . . . .. . . . 17 圖2.11 線性迴歸示意圖. . . . . . . . . . . . . . . . . . . . . . . . . . . 21 圖2.12 Modbus 在ISO 七層的位置. . . . . . . . . . . . . . . . . 23 圖2.13 串列網路的模組. . . . . . . . . . . . . . . . . . . . . . . . . . . 24 圖2.14 PDU 格式. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 圖3.1 雲端化智慧遠端監控管理平台之研發概念圖 . . . 27 圖3.2 雲端運算環境進行遠端監控. . . . . . . . . . . . . . . . . 28 圖3.3 資料探勘擷取特徵架構圖. . . . . . . . . . . . . . . . . . . 28 圖3.4 雲端化智慧遠端監控管理平台之研發架構圖. . 29 圖3.5 HAESModbus 模組. . . . . . . . . . . . . . . . . . . . . .. . . 30 圖3.6 TCP/IP Modbus 格式. . . . . . . . . . . . . . . . . . . . . . . 31 圖3.7 雲端資安模組示意圖. . . . . . . . . . . . . . . . . . . .. . . . 32 圖3.8 傳感器列表的創建. . . . . . . . . . . . . . . . . . . . . . . . . 33 圖3.9 分散式資料儲存. . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 圖3.10 各地控制單元的溫度收集架構圖. . . . . . . . . . . . 37 圖3.11 傳感器溫度的表示方法. . . . . . . . . . . . . . . . . . . . . 39 圖3.12 溫度預測模組的架構圖. . . . . . . . . . . . . . .. . . . . . 41 圖3.13 溫度預測模組架構圖. . . . . . . . . . . . . . . . . . . . . . 41 圖3.14 溫度警報預測架構圖. . . . . . . . . . . . . . . . . . . . . . . 45 圖3.15 系統預測架構圖. . . . . . . . . . . . . . . . . . . . . . . . . . . 45 圖3.16 警報示意圖(1) . . . . . . . . . . . . . . . . . . . . . . . . . 46 圖3.17 警報示意圖(2) . . . . . . . . . . . . . . . . . . . .. . . . . . 47 圖3.18 警報示意圖(3) . . . . . . . . . . . . . . . . . . . . . . . . . 48 圖3.19 警報示意圖(4) . . . . . . . . . . . . . . . . . . .. . . . . . 49 圖4.1 800 sensors (records/sec) . . . . . . . . . . . . . . . . . 53 圖4.2 4000 sensors (records/sec) . . . . . . . . . . . . . . . . 54 圖4.3 8000 sensors (records/sec) . . . . . . . . . .. . . 55 圖4.4 雲端效能比較. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 圖4.5 處理前的資料和處理後的資料比較. . . . . . . . . . 57 圖4.6 總誤差直方圖. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 圖4.7 效能比較(總誤差)曲線圖. . . . . . . . . . . . . . . . . . . . . 59 圖4.8 指定係數關係直方圖. . . . . . . . . . . . . . . . . . . . .. . . 60 圖4.9 指定係數關係曲線圖. . . . . . . . . . . . . . . . . . . . . . . 61 圖4.10 準確率比較直方圖. . . . . . . . . . . . . . . . . . . . . . . . 62 圖4.11 效能比較(準確率)曲線圖. . . . . . . . . . . . . . . .. . . . 63 圖4.12 異常預測準確率直方圖. . . . . . . . . . . . . . . . .. . . 65 圖5.1 技術與知識架構. . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 表目錄 表2.1 常用Function Code . . . . . . . . . . . . . . . . . . . . . . 25 表3.1 傳感器的基本資料結構. . . . . . . . . . . . . . . . . . . . . . 34 表3.2 示範資料庫中儲存的數據. . . . . . . . . . . . . . . . . . . 39 表4.1 傳感器資料結構. . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 表4.2 雲端監控中心接收資訊(1) . . . . . . . . . . . . . . . 52 表4.3 雲端監控中心接收資訊(2) . . . . . . . . . . . . . . . . 52 表4.4 800 個感測器效能分析. . . . . . . . . . . . . . . . . . . . . 53 表4.5 4000 個感測器效能分析. . . . . . . . . . . . . . . . . . . . . 54 表4.6 8000 個感測器效能分析. . . . . . . . . . . . . . . . . . . . 54 表4.7 三組Threshold . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 表4.8 總誤差比較. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 表4.9 效能比較(總誤差) . . . . . . . . . . . . . . . . . . . . . . . . . . 59 表4.10 指定係數的關係. . . . . . . . . . . . . . . . . . . . . . . . . . . 60 表4.11 準確率比較. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 表4.12 效能比較(準確率) . . . . . . . . . . . . . . . . . . . . . . . . . 63 表4.13 異常預測準確率. . . . . . . . . . . . . . . . . . . . . . . . . . . 64 表5.1 遠端監控成本比較表. . . . . . . . . . . . . . . . . . . . . . . . 67 |
參考文獻 References |
[1] Announcing the advanced encryption standard (aes), 2001. [2] Alberto Abell, Catalunya Barcelonatech, Jaume Ferrarons, Catalunya Barcelonatech, Oscar Romero, and Catalunya Barcelonatech. Building cubes with mapreduce. [3] S. S. R. Abidi and J. Ong. A data mining strategy for inductive data clustering: a synergy between self-organising neural networks and k- means clustering techniques. In Proc. TENCON 2000, volume 2, pages 568-573, 2000. [4] R. Agrawal and R. Srikant. Mining sequential patterns. In Proc. Eleventh Int Data Engineering Conf, pages 3-14, 1995. [5] E.-W. Bai and K. M. Nagpal. Least squares type algorithms for iden- tication in the presence of modeling uncertainty. In Proc. 33rd IEEE Conf. Decision and Control, volume 4, pages 3602-3607, 1994. [6] James Bezdek, Robert Gunderson, Robert Ehrlich, and Tom Meloy. On the extension of fuzzy k-means algorithms for detection of linear clusters. In Proc. IEEE Conf. Decision and Control including the 17th Symp. Adaptive Processes, volume 17, pages 1438-1443, 1978. [7] Gautam Biswas, Jerry B. Weinberg, and Douglas H. Fisher. Iterate: A conceptual clustering algorithm for data mining, 1998. [8] J. Borges and M. Levene. Evaluating variable-length markov chain mod- els for analysis of user web navigation sessions. 19(4):441-452, 2007. [9] D. Carstoiu, A. Cernian, and A. Olteanu. Hadoop hbase-0.20.2 perfor- mance evaluation. In Proc. 4th Int New Trends in Information Science and Service Science (NISS) Conf, pages 84-87, 2010. [10] Fay Chang, Jerey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deb- orah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber. Bigtable: A distributed storage system for structured data, 2006. [11] Shih-Tsun Chu, Chao-Chun Yeh, and Chun-Lung Huang. A cloud-based trajectory index scheme. In Proc. IEEE Int. Conf. e-Business Engineer- ing ICEBE '09, pages 602-607, 2009. [12] O. Cordon and F. Herrera. A proposal for improving the accuracy of linguistic modeling. 8(3):335-344, 2000. [13] D. M. Cox, D. J. Trevor, E. A. Rohlng, and A. Kaldor. Measurements of magnetic moments of metal atom clusters (abstract). J Appl Phys, 57(8), 1985. [14] B. De Moor and J. Vandewalle. A unifying theorem for linear and total linear least squares. 35(5):563-566, 1990. [15] S. L. Elliott. A neural network learning algorithm applying linear re- gression that determines and uses target values for hidden neurons. In Proc. Int Neural Networks IJCNN. Joint Conf, volume 3, pages 829-838, 1992. [16] Hichem Frigui and R. Krishnapuram. Competitive fuzzy clustering. In Proc. NAFIPS Fuzzy Information Processing Society 1996 Biennial Conf. of the North American, pages 225-228, 1996. [17] LiMin Fu. Knowledge discovery by inductive neural networks. 11(6):992-998, 1999. [18] Sanjay Ghemawat, Howard Gobio, and Shun-Tak Leung. The google le system, 2003. [19] V. Heikkinen, T. Tokola, J. Parkkinen, I. Korpela, and T. Jaaskelainen. Simulated multispectral imagery for tree species classication using sup- port vector machines. 48(3):1355-1364, 2010. [20] H. Ishibuchi, R. Fujioka, and H. Tanaka. Neural networks that learn from fuzzy if-then rules. 1(2):85-97, 1993. [21] E. Kapetanios and M. C. Norrie. Data mining and modeling in scientic databases. In Proc. Conf. Ninth Int Scientic and Statistical Database Management, pages 24-27, 1997. [22] Aram Karalic. Linear regression in regression tree leaves, 1992. [23] Kenji Koga and Harutoshi Takeo. In situ grazing incidence x-ray dirac- tion system for clusters deposited on a low-temperature substrate. Rev Sci Instrum, 67(12):4092-4097, 1996. [24] R. Krishnapuram and C.-P. Freg. Fuzzy algorithms to nd linear and planar clusters and their applications. In Proc. CVPR '91. IEEE Com- puter Society Conf Computer Vision and Pattern Recognition, pages 426-431, 1991. [25] Xuhui Liu, Jizhong Han, Yunqin Zhong, Chengde Han, and Xubin He. Implementing webgis on hadoop: A case study of improving small le i/o performance on hdfs. In Proc. IEEE Int. Conf. Cluster Computing and Workshops CLUSTER '09, pages 1-8, 2009. [26] R. L. Racicot. Approximate condence intervals for reliability of a series system. (4):265-269, 1976. [27] Rakesh Agrawal Ramakrishnan Srikant. Mining sequential patterns: Generalizations and performance improvements, 1995. [28] C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis. Evaluating mapreduce for multi-core and multiprocessor systems. In Proc. IEEE 13th Int. Symp. High Performance Computer Architecture HPCA 2007, pages 13-24, 2007. [29] Doug Cutting Rohit Khare. Nutch: A exible and scalable open-source web search engine, 2004. [30] R. Setiono. Extracting m-of-n rules from trained neural networks. 11(2):512-519, 2000. [31] Dipesh Shrestha. Text mining with lucene and hadoop: Document clus- tering with feature extraction by. [32] Konstantin Shvachko, Hairong Kuang, Sanjay Radia, and Robert Chansler. The hadoop distributed le system. [33] Konstantin Shvachko, Hairong Kuang, Sanjay Radia, and Robert Chansler. The hadoop distributed le system. [34] I. A. Taha and J. Ghosh. Symbolic interpretation of articial neural networks. 11(3):448-463, 1999. [35] Greg Madey Timothy W. Schoenharl. Evaluation of measurement tech- niques for the validation of agent-based simulations against streaming data. [36] I. Ungi, A. Thury, and M. Csanady. Investigation of the correlation between heart rate and heart rate variability. In Proc. Computers in Cardiology 1995, pages 189-191, 1995. [37] M. A. Wajeed and T. Adilakshmi. Dierent similarity measures for text classication using knn. In Proc. 2nd Int Computer and Communication Technology (ICCCT) Conf, pages 41-45, 2011. [38] C. Wartena and R. Brussee. Topic detection by clustering keywords. In Proc. 19th Int. Workshop Database and Expert Systems Application DEXA '08, pages 54-58, 2008. [39] B. Weber and R. Scholl. A new kind of light-generation mechanism: Incandescent radiation from clusters. J Appl Phys, 74(1):607-613, 1993. [40] J. Wilpon and L. Rabiner. A modied k-means clustering algorithm for use in isolated work recognition. 33(3):587-594, 1985 |
電子全文 Fulltext |
本電子全文僅授權使用者為學術研究之目的,進行個人非營利性質之檢索、閱讀、列印。請遵守中華民國著作權法之相關規定,切勿任意重製、散佈、改作、轉貼、播送,以免觸法。 論文使用權限 Thesis access permission:校內校外完全公開 unrestricted 開放時間 Available: 校內 Campus: 已公開 available 校外 Off-campus: 已公開 available |
紙本論文 Printed copies |
紙本論文的公開資訊在102學年度以後相對較為完整。如果需要查詢101學年度以前的紙本論文公開資訊,請聯繫圖資處紙本論文服務櫃台。如有不便之處敬請見諒。 開放時間 available 已公開 available |
QR Code |