近來在組合測試的研究上，可分為二參數組合涵蓋(pair-wise coverage)及多參數組合涵蓋(multi-wise coverage)兩類，學者們紛紛提出各種的組合策略以求自動產生測試資料的目的，但為了能衡量產生的測試組套，往往都須滿足一定的涵蓋標準，但這些組合策略卻忽略了使用上的彈性。
從實務面來看軟體測試，會發現軟體測試往往都受到成本的制約，因此如何在有限成本下獲得最大的測試利益是實務上必須考量的，但在現存組合策略所產生的測試組套，並沒有辦法作有彈性的使用，換句話說，我們必須全盤的對每筆測試資料進行測試，因此，利用現存組合策略所產生的測試資料，在實務上的使用有很大的限制存在。
因此，本研究針對實務上的需求，提出一個以CSP 為基底的彈性組合策略，除了可兼顧成本狀況下作最有效的測試，也允許使用者在測試中隨時加入限制式，動態調整我們產出的測試資料，而實驗中也顯示，藉由我們提出的方法，能夠避免掉一些使用者認為效益較小或是不必要的測試，以達最大測試利益目的，更進一步達到縮減測試資料量的目的。

In recent year the research field of the combinatorial testing, which can roughlybe divided into two kinds including pair-wise coverage and multi-wise coverage. a lot of scholar try to use various strategies to generate test data automatically. In order to weight the generated test set, the generated test data must satisfy certain criterion. But these combinatorial strategy neglected the flexibility of using on the practice.
Considering software testing from the practice, which often be restricted by the cost. For this reason, how to obtaint the greatest testing benefits under the limited cost must be considered on the parctice. But in the extant combinatorial strategy, there is no flexible use. In other words, we must testing test set totally. Therefore, there is very great restriction to exist on using the test data generated by the extant combinatorial strategy on the practice.
So, this paper proposes a flexible combinatorial strategy based on CSP, which allow users to do the most valid testing under the limited cost, which also allow users join the constraints that needs at any time during the testing process, revise the test data that we produced dynamically. The experimental result indicate that our method perform well, it can avoid including the test data whether some users think the interests less or unnecessarier, in order to achive the greatest testing beneifts.Further, we can achive the goal of reducing the quantity of testing data.

第一章 緒論 1
第一節 研究背景 1
第二節 研究動機 2
第三節 研究目標 3
第四節 論文架構 4
第二章 文獻探討 5
第一節 功能測試(functional testing) 5
2-1-1 等類分割(equivalencw class partition) 5
2-1-2 邊界值分析(boundary value analysis) 5
2-1-3 因果圖(cause-effect graph) 6
2-1-4 決策表(decision table) 6
第二節 組合測試(combinatorial testing) 6
2-2-1 組合策略的涵蓋標準(coverage criteria) 7
2-2-2 二參數組合涵蓋測試(two-way testing) 7
2-2-2-1 非固定(non-deterministic)型組合策略 7
2-2-2-2 固定型(deterministic)組合策略 10
2-2-3 多參數組合涵蓋測試(multi-way testing) 14
第三節 限制滿足問題(CSP) 18
2-3-1 CSP 的限制式種類 20
2-3-2 CSP 求解演算法 21
2-3-3 CSP 與 組合測試(Combinatorial testing) 28
第四節 結構測試 30
第三章 研究方法與架構 33
第一節 組合測試的問題與策略 33
3-1-1 具相同意義的測試組合 33
3-1-2 不具意義的測試組合 34
3-1-3 輸入參數中存在矛盾關係 36
3-1-4 組合測試組套於實務上使用的彈性 41
第二節 彈性化組合策略 42
3-2-1 產生測試組套所應涵蓋的組合 45
3-2-2 初始化並加入必要限制式 46
3-2-3 決定變數初始化的順序 47
3-2-4 根據變數初始化順序給予變數值 48
3-2-5 Forward Checking(FC) 50
3-2-6 最佳化處理(Optimization) 51
3-2-7 插裝非必要限制式(Plug in constraint) 52
第四章 實驗結果與分析 53
第一節 標竿問題 54
第二節 實驗一 56
4-2-1 實驗數據 56
4-2-2 實驗結果討論 58
第三節 實驗二 59
4-3-1 實驗數據 59
4-3-2 實驗結果討論 60
4-3-2 實驗一與實驗二小結 61
第四節 實驗三 61
4-4-1 實驗數據 62
4-4-2 實驗結果討論 63
第五節 實驗四 64
4-5-1 實驗數據 68
4-5-2 實驗結果討論 69
第六節 實驗五 70
4-6-1 情境一 70
4-6-2 實驗數據 71
4-6-3 實驗結果討論 71
4-6-4 情境二 72
4-6-5 實驗數據 72
4-6-6 實驗結果討論 73
第七節 組合策略的比較 73
第五章 結論與未來方向 75
參考文獻 77

1. A.S. Heydayat, N.J.A.S., John Stufken, Orthogonal arrays : theory and applications. 1999, New York: Springer.
2. Chen, H.Y., et al, In black and white: an integrated approach to class-level testing of object-oriented programs. ACM Transactions on Software
Engineering and Methodology, 1998. 7(3): p. 250-295.
3. Cohen, D.M., et al, The Automatic Efficient Test Generator (AETG) system. in 1994. Proceedings., 5th International Symposium on Software Reliability
Engineering, 1994.
4. Cohen, M.B., et al, Constructing test suites for interaction testing. in Proceedings of the 25th international conference on Software engineering,
2003.
5. Dalal, S.R.a.C.L.M., Factor-Covering Designs for Testing Software. Technometrics, 1998. 40(3): p. 234-243.
6. Grindal, M., Handling Combinatorial Explosion in Software Testing. PhD Thesis at University of Skövde, 2007.
7. Grindal, M., J. Offutt, and S.F. Andler, Combination testing strategies: a survey. Software Testing, Verification and Reliability, 2005. 15(3): p.167-199.
8. Grindal, M.O., J. Mellin, J., Managing Conflicts When Using Combination Strategies to Test Software. Software Engineering Conference, 2007. ASWEC
2007. 18th Australian, 2007: p. 255-264.
9. Kuhn, D.R.W., D.R. Gallo, A.M., Jr., Software fault interactions and implications for software testing. Software Engineering, IEEE Transactions on, 2004. 30(6): p. 418-421.
10. Kumar, V., Algorithms for Constraint-Satisfaction Problems: A Survey. AI Magazine, 1992. 13(1): p. 32-44.
11. Mandl, R., Orthogonal Latin squares: an application of experiment design to compiler testing. Communications of the ACM, 1985. 28(10): p. 1054-1058.
12. Mats Grindal, J.O.a.J.M., Handling Constraints in the Input Space when Using Combination Strategies for Software Testing. Technical Report HS-IKI-TR-06-001 School of Humanities and Informatics University of Sk¨ovde, 2006.
13. Michael Jampel, E.C.F., Michael Maher, Over-constrained systems. 1996, New York: Springer.
14. Myers，G•J, The Art of Software Testing. 2004: The Art of Software Testing.
15. Piwowarski, P., M. Ohba, and J. Caruso, Coverage measurement experience during function test. Proceedings of the 15th international conference on
Software Engineering, 1993: p. 287-301.
16. Sagarna, R.a.J.A.L., On The Performance of Estimation Of Distribution Algorithms Applied to Software Testing. Applied Artificial Intelligence, 2005.
19(5): p. 457-489.
17. Sagarna, R.a.J.A.L., Scatter Search in software testing, comparison and collaboration with Estimation of Distribution Algorithms. European Journal
of Operational Research, 2006. 169(2): p. 392-412.
18. Sally C. Brailsford, C.N.P.a.B.M.S., Constraint Satisfaction Problems: Algorithms and Applications. European Journal of Operational Research, 1999. 199: p. 557-581.
19. Shiba, T.T., T. Kikuno, T., Using artificial life techniques to generate test cases for combinatorial testing. Computer Software and Applications Conference, 2004, 2004: p. 72-77.
20. Sthamer, H.H., The Automatic Generation of Software Test Data Using Genetic Algorithms. 1995: University of Glamorgan.
21. Tai, K.C.a.Y.L., A test generation strategy for pairwise testing. Software Engineering, IEEE Transactions on, 2002. 28(1): p. 109-111.
22. Tracey, N., J. Clark, and K. Mander, Automated program flaw finding using simulated annealing. ACM SIGSOFT Software Engineering Notes, 1998.
23(2): p. 73-81.
23. Tsang, E., Foundations of Constraint Satisfaction. 1993: Academic Pr.
24. Tsang, E.a.A.K., Mapping Constraint Satisfaction Problems to Algorithms and Heuristics. Technical Report CSM254, Department of Computer Science,
University of Essex, Colchester, 1993.
25. Williams, A.W.a.R.L.P., A Measure for Component Interaction Test Coverage. Proceedings of the ACS/IEEE International Conference on Computer Systems and Applications, 2001.
26. Williams, A.W.a.R.L.P., A practical strategy for testing pair-wise coverage of networkinterfaces. Software Reliability Engineering, 1996. Proceedings.,
Seventh International Symposium on, 1996.
27. Yu Lei, R.K., D. Richard Kuhn,Vadim Okun and James Lawrence, IPOG/IPOG-D: efficient test generation for multi-way combinatorial testing.
Wiley InterScience, 2007.
28. 王修齊, 針對自動化產生測試資料的混合式基因演算法. 中山大學資訊管理所碩士論文, 2006.
29. 蔡宗翰, 利用限制條件法於組合策略產生黑箱測試資料之研究. 中山大學資訊管理所碩士論文, 2006.
30. 鄭炳強, 軟體工程：從實務出發. 2007: 智勝文化事業有限公司.
31. 錢一一譯, F.P.B., 人月神話：軟體專案管理之道，譯自：The Mythical Man-Month. 2004: 經濟新潮社.