為了保證儲存服務品質和節省能源，我們提出快速資料遷移/傳送機制和能源節省演算法的節點管理。快速資料遷移/傳送機制包含了三種機制，第一使用群體廣播改善網路頻寬並解決I/O和網路瓶頸；第二使用網路編碼技增加網路吞吐量且保持高容錯性；第三利用使用者/連結管理防止重要的封包遺失和搭配CPU和I/O約束排程使得資料平均地儲存於系統中。此快速資料遷移/傳送機制分別改善了56%的上傳頻寬和85%的反應時間。能源節省演算法先使用卡曼過濾器和然後結合樣本分析去預估系統工作量以動態調整節點個數以達到節省能源。根據實驗結果證明能源節省的節點管理演算法達到92.97%的準確率，並分別改善了52.25%的能源消耗並達到低於3.82%的錯誤率。

In order to achieve the storage QoS and power saving, we proposed a fast data migration/transmission scheme and a power saving algorithm for Dataenode management. The fast data migration/ transmission scheme consists of three mechanisms. First, it uses multicast to improve the network bandwidth and solve the I/O and bandwidth bottlenecks. Then, a network coding is used to increase the network throughput and retain high fault tolerance. Third, it uses a user/Dataenode connection management to prevent missing the important message and collocates with CPU & I/O bound scheduling to make data evenly stored in the system. Experimental results show the proposed fast data migration/transmission improves 56% and 85% efficiency in the upload bandwidth and the response time. The proposed power saving algorithm applies the Kalman filter first and then add with the pattern analysis to predict the system workload to adjust the number of Dataenodes dynamically in order to save power. Experimental results show that the proposed power saving algorithm for Dataenode management achieves more than 92.97% accuracy in the workload prediction and improves 52.25% in power consumption with 3.82% error rate.

List of Figures vii
List of Tables x
1. Introduction 1
2. Related Work 11
2.1 Network Coding 11
2.2 Cloud Storage System 12
2.3 Scheduler of Hadoop 13
2.4 Power Saving 14
3. Proposed Architecture 18
3.1 System Overview 18
3.1.1 CPU & I/O Bound Scheduling 20
3.1.2 Monitor 22
3.2 Fast Data Migration/Transmission Scheme 24
3.2.1 User/Dataenode Connection Management 24
3.2.2 Multicast & Network Coding 27
3.3 Power Saving Algorithm for Dataenode Management 32
4. Performance Evaluation 45
4.1 Performance of the Fast Data Migration/Transmission Scheme 45
4.2 Performance of the Proposed Power Saving Algorithm for Dataenode Management 49
5. Conclusions 70
References 71

[1] Facebook. A social network. http://www.facebook.com
[2] Youtube. An online video. http://www.youtube.com
[3] Hadoop. A cloud platform. http://hadoop.apache.org
[4] Nutch. A open source web-search software. http://nutch.apache.org/
[5] S. Y. R. Li, R. W. Yeung, and N. Cai, “Linear Network Coding,” IEEE
Transactions on Information Theory, vol. 49, pp. 371, 2003.
[6] I. S. Reed and G. Solomon, “Polynomial codes over certain finite field,”
Journal of the Society for Industrial and Applied Mathematics, Vol. 8, No. 2,
Jun. 1960.
[7] R. Ahlswede, N. Cai, S. R. Li, and R. W. Yeung, “Network Information Flow,”
IEEE Transactions on Information Theory, vol. 46, no. 4, pp.1204-1216, July
2000.
[8] A. R. Bharambe and C. Herley, “Analyzing and Improving BitTorrent
Performance,” Microsoft Research, Cambridge, U.K., Tech. Rep.
MSRTR-2005-03, 2005.
[9] J. Byers, J. Considine, M. Mitzenmacher, and A. Rege, “A Digital Fountain
Approach to Reliable Distribution of Bulk Data” in Proc. SIGCOMM, pp.56-67,
Sep. 1998.
[10] C. Gkantsidis and P. Rodriguez, “Network Coding for Large Scale Content
Distribution,” in Proc. of IEEE INFOCOM 2005, March 2005.
[11] C. Gkantsidis, J. Miller, and P. Rodriguez, “Anatomy of a P2P Content
Distribution System with Network Coding,” in Proc. of the 5th International
Workshop on Peer-to-Peer Systems (IPTPS 2006), 2006.
[12] R. Sandberg, D. Goldberg, S. Kleiman, D. Walsh, and B. Lyon, “Design and
Implementation of the Sun Network Filesystem,” 1985
[13] D. R. Brownbridge, L. F. Marshall, and B. Randell, “The Newcastle Connection
or UNIXes of the World Unite!” Software Practice and Experience,
12(12):1147-1162, 1982
[14] Amazon Simple Storage Service. A storage for the Internet.
http://aws.amazon.com/s3/
[15] J. Tate, F. Lucchese and R. Moore, “Introduction to Storage Area
Networks,”International Technical Support Organization, Four Edition, July
2006
[16] R. Lidl and H. Niederreiter, “Introduction to Finite Fields and Their
Applications.”Cambridge, United Kingdom: Cambridge University Press, 1994
[17] L. N. Trefethen, and D. Bau, “Numberical Linear Algebra,”Philadelphia, PA:
Society for Industrial and Applied Mathematics, 1997
[18] R.E. Kalman, “A new approach to linear filtering and prediction problems,”
Journal of Basic Engineering, 1960, pp.35-45.
[19] Storage QoS. Descripition of storage Qos.
http://www.ssrc.ucsc.edu/proj/storageqos.html
[20] A. Silberschatz, P. B Galvin, and G. Gagne, “Operating System
Principles,”Wiley Asia Student Edition, Seventh Edition
[21] L. Zheng, “A Task Migration Constrained Energy-Efficient Scheduling
Algorithm for Multiprocessor Real-time Systems,” Wireless Communications,
Networking and Mobile Computing, 2007. WiCom 2007. International
Conference on, Oct. 2007
[22] J. H. Abawajy, and S. P. Dandamudi, “Scheduling Parallel Jobs with CPU and
I/O Resource Requirements in Cluster Computing Systems,” Proceeding of the
11th IEEE/ACM International Symposium on Modeling, Analysis and
Simulation of Computer Telecommunications Systems, MASCOTS 2003
[23] “Hadoop Fair Scheduler Design Document,” October 18, 2010
[24] C. Tian, H. Zhou, Y. He, and L. Zha, “A Dynamic MapReduce Scheduler for
Heterogeneous Workloads,” the 8th International Conference on Grid and
Cooperative Computing, 2009
[25] C. J. Hughes, and S.V Adve, “A Formal Approach to Frequent Energy
Adaptations for Multimedia Applications,” In International Symposium on
Computer Architecture(ISCA), Munich, Germany, 2008, pp. 138-149.
[26] J. Choi, and H. Cha, “Memory-Aware Dynamic Voltage Scaling for Multimedia
Applications,” IET Computers and Digital Techniques, vol. 153, no.2, 2006, pp.
130-136.
[27] Y. Gu, and S. Chakraborty, “Control Theory-Based DVS for Interactive 3D
Games,” In Proceedings of Design Automation Conference (DAC), Anaheim,
California, USA, 2008, pp. 740-745.
[28] V. Almeida, M. Arlitt, and J. Rolia, “Analyzing a Web-Based System's
Performance Measures at Multiple Time Scales,” SIGMETRICS Performance
Evaluation Review, vol.30, no.2, 2002, pp. 3-9.
[29] S. Bennett, “Nicholas Minorsky and the Automatic Steering of Ships,” IEEE
Control Systems Magazin, vol. 4, no.4, 1984.
[30] B. Li, J. Li, T. WO, Q. Li, and L. Zhong, “EnaCloud: An Energy-Saving
Application Live Placement Approach for Cloud Computing Environment, ”
2009 IEEE International Conference on Cloud Computing, 21-25 Sept. 2009
[31] T. V. T. Duy, Y. Sato, and Y. Inoguchi, “Performance Evaluation of a Green
Scheduling Algorithm for Energy Savings in Cloud Computing,” 2010 IEEE
International Symposium on Parallel & Distributed Processing, Workshops and
Phd Forum (IPDPSW), 19-23 April 2010
[32] A. G. Dimakis, P. B. Godfrey, Y. Wu, M. Wainwright, and K. Ramchandran,
"Network Coding for Distributed Storage Systems," IEEE Transactions on
Information Theory, September, 2010