在現今的軟體發展中，通常每個組織都會為本身的軟體做一套測試系統，主要是因為每個軟體本身都會存在著一些缺陷，這些缺陷嚴重者可能造成公司產生金錢以及資訊安全上的損失，因此透過一套有程序的測試系統可以提防預先在軟體上找出漏洞或錯誤來進行修正，但是有些錯誤可能經過一段時間又會重新產生，因此，這時候透過自動化測試系統，我們可以有效益、有效率的去自動測試並找出系統的漏洞，減少時間以及人力成本的付出。
自動化測試的出現，解決的以往測試方式的缺點，在本論文中會提及兩種自動化測試，分別為具體-符號測試 ( concolic testing )以及隨機測試 ( random testing) ，在2009年的文獻中顯示了目前有許多的跡象可以去說明具體-符號測試是可以勝任隨機測試的，但是，卻鮮少有證明去證明之間的效益，因此希望藉由此論文去論證具體-符號測試與隨機測試之間的效益。

The development of software today, the company has their own test system usually. Because there has a few bugs in the every software. And it will make the damage of company’s property or security of information. We can find the bugs in the software by the test systems. But the few bugs will appear repeatedly even if you have been fixed it. In this time, it will be effective if we use the automatic test systems. They can solve the waste of time and cost.
Appearance of the automatic test system has been solved the defect of the test method in the past. In this paper will mention two kind of automatic test systems, one of them is concolic testing, and another is random testing. In the 2009, there had the few of evidence to discuss that the concolic testing was more effective than the random testing, but there wasn’t have the enough demonstration. So I hope to prove that the effectiveness comparison between concolic and random testing by this paper.

1. 緒論 .................................................................................................................. 1
1.1 研究背景 ...............................................................................................1
1.2 研究動機 ...............................................................................................2
1.3 問題描述 ...............................................................................................2
1.4 論文架構 ...............................................................................................2
2. 文獻探討 .......................................................................................................... 4
2.1 自動化測試系統 (Automated Testing System ) ..................................4
2.1.1 自動化測試的優點 ..................................................................... 4
2.2 具體-符號測試 ( Concolic Testing ) ....................................................4
2.2.1 具體-符號測試介紹 .................................................................... 4
2.2.2 具體-符號測試工具 .................................................................... 8
2.2.3 小結 ............................................................................................. 8
2.3 隨機測試 ( Random Testing ) ..............................................................8
2.3.1 隨機測試介紹 ............................................................................. 8
2.3.2 隨機測試的優點 ......................................................................... 9
2.3.3 隨機測試的可靠性 ..................................................................... 9
2.3.4 小結 ............................................................................................ 11
ii
3. 研究架構與實作 ............................................................................................ 12
3.1 系統架構 .............................................................................................12
3.1.1 系統架構介紹 ........................................................................... 13
3.2 實驗工具 .............................................................................................14
3.2.1 CREST 實驗工具介紹 .............................................................. 14
3.2.2 CREST 插裝、編譯與執行 ...................................................... 16
3.2.3 具體-符號推演 ( Concolic Execution ) ................................... 16
3.2.4 Quick Check 實驗工具介紹 ..................................................... 17
3.2.5 Quick Check 插裝、編譯與執行 ............................................. 19
3.2.6 Erlang ......................................................................................... 23
4. 實驗結果與討論 ............................................................................................ 24
4.1 實驗目的 .............................................................................................24
4.2 實驗假設與限制 .................................................................................24
4.3 實驗樣本 .............................................................................................26
4.4 實驗流程 .............................................................................................27
4.4.1 實驗準則 ................................................................................... 27
4.5 實驗結果與討論 .................................................................................28
4.5.1 實驗一：具體-符號測試之成效 .............................................. 28
iii
4.5.2 實驗一：隨機測試之成效 ....................................................... 29
4.5.3 實驗一：具體-符號測試與隨機測試結果比較 ...................... 31
4.5.4 實驗二：具體-符號與隨機對於突變程式結構之效益比較 .. 36
4.5.5 實驗二：具體-符號與隨機對於突變程式之效益比較結果 .. 36
5. 結論與未來研究 ............................................................................................ 52
5.1 實驗討論 .............................................................................................52
5.2 未來工作 .............................................................................................53
6. 參考文獻 ........................................................................................................ 54

1. 鄭炳強, 軟體工程：從實務出發2007: 智勝文化.
2. Harman.M., L.K.a.M.P.a., Automated Test Data Generaction for Coverage - Haven't We Solved This Problem yet. 2009: p. 1.
3. Hamlet, R., Random Testing, in Encyclopedia of Software Engineering2002, John Wiley & Sons, Inc.
4. Most Influential ICFP Paper Award. Available from: http://www.sigplan.org/award-icfp.htm.
5. Why Automated Testing. Available from: http://smartbear.com/products/qa-tools/automated-testing/manager-overview/.
6. Cadar, C., D. Dunbar, and D. Engler, KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs, in Proceedings of the 8th USENIX conference on Operating systems design and implementation2008, USENIX Association: San Diego, California. p. 209-224.
7. Sen, K., Concolic testing, in Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering2007, ACM: Atlanta, Georgia, USA. p. 571-572.
8. Sen, K., Scalable automated methods for dynamic program analysis, 2006, Illinois.
9. Cadar, C., et al., EXE: Automatically Generating Inputs of Death. ACM Trans. Inf. Syst. Secur., 2008. 12(2): p. 1-38.
10. Tillmann, N. and J.D. Halleux, Pex: white box test generation for .NET, in Proceedings of the 2nd international conference on Tests and proofs2008, Springer-Verlag: Prato, Italy. p. 134-153.
11. Kim, Y., M. Kim, and N. Dang, Scalable Distributed Concolic Testing: A Case Study on a Flash Storage Platform
Theoretical Aspects of Computing – ICTAC 2010, A. Cavalcanti, et al., Editors. 2010, Springer Berlin / Heidelberg. p. 199-213.
12. 方超群, 可協助除錯的自動評估系統, in 資訊管理系研究所2010, 國立中山大學: 高雄.
55
13. Godefroid, P., N. Klarlund, and K. Sen, DART: directed automated random testing, in Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation2005, ACM: Chicago, IL, USA. p. 213-223.
14. Williams, N., B. Marre, and P. Mouy. On-the-fly generation of k-path tests for C functions. in Automated Software Engineering, 2004. Proceedings. 19th International Conference on. 2004.
15. Williams, N., et al., PathCrawler: Automatic Generation of Path Tests by Combining Static and Dynamic Analysis
Dependable Computing - EDCC 5, M. Dal Cin, M. Kaaniche, and A. Pataricza, Editors. 2005, Springer Berlin / Heidelberg. p. 281-292.
16. Sen, K. and G. Agha, CUTE and jCUTE: Concolic Unit Testing and Explicit Path Model-Checking Tools
Computer Aided Verification, T. Ball and R. Jones, Editors. 2006, Springer Berlin / Heidelberg. p. 419-423.
17. Sen, K., D. Marinov, and G. Agha, CUTE: a concolic unit testing engine for C. SIGSOFT Softw. Eng. Notes, 2005. 30(5): p. 263-272.
18. Cadar, C. and D. Engler, Execution Generated Test Cases: How to Make Systems Code Crash Itself
Model Checking Software, P. Godefroid, Editor 2005, Springer Berlin / Heidelberg. p. 902-902.
19. Duran, J.W. and S.C. Ntafos, An Evaluation of Random Testing. Software Engineering, IEEE Transactions on, 1984. SE-10(4): p. 438-444.
20. Dalley, J.L. The art of software testing. in Aerospace and Electronics Conference, 1991. NAECON 1991., Proceedings of the IEEE 1991 National. 1991.
21. R. A. Thayer, M.L., and E. C. Nelson, Software Reliability.Amsterdam. 1978.
22. Rault, E.G.a.J.-C., A programming technique for soft ware
reliability. 1973: p. 44-50.
23. Duran, J.W. and J.J. Wiorkowski, Quantifying Software Validity by Sampling. Reliability, IEEE Transactions on, 1980. R-29(2): p. 141-144.
56
24. 許立文, 以導引式隨機測試方法探索軟體未規範實作功能, in 訊管理系研究所2007, 國立中山大學: 高雄.
25. Claessen, K. and J. Hughes, QuickCheck: a lightweight tool for random testing of Haskell programs. SIGPLAN Not., 2011. 46(4): p. 53-64.
26. Burnim, J. and K. Sen, Heuristics for Scalable Dynamic Test Generation, in Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering2008, IEEE Computer Society. p. 443-446.
27. Necula, G., et al., CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs
Compiler Construction, R. Horspool, Editor 2002, Springer Berlin / Heidelberg. p. 209-265.
28. Dutertre, B. and L. de Moura, A Fast Linear-Arithmetic Solver for DPLL(T)
Computer Aided Verification, T. Ball and R. Jones, Editors. 2006, Springer Berlin / Heidelberg. p. 81-94.
29. Brummayer, R. and A. Biere, Boolector: An Efficient SMT Solver for Bit-Vectors and Arrays
Tools and Algorithms for the Construction and Analysis of Systems, S. Kowalewski and A. Philippou, Editors. 2009, Springer Berlin / Heidelberg. p. 174-177.
30. de Moura, L. and N. Bjorner, Z3: An Efficient SMT Solver
Tools and Algorithms for the Construction and Analysis of Systems, C. Ramakrishnan and J. Rehof, Editors. 2008, Springer Berlin / Heidelberg. p. 337-340.
31. Erlang. Available from: http://zh.wikipedia.org/zh/Erlang.