本論文主要的參考書本為蒙地卡羅統計方法 ( Monte Carlo Statistical Methods, second edition )，作者為 Robert 及 Casella (2004)，參考章節為第一章∼第五章 (不含第四章變異數控制部分)。本論文內容主要目的是將此書前五章的內容進行： 1.中文化 2.修正錯誤 3.加入公式推導過程 4.將例題所用的演算法寫成程式等四大工作項目，以及應用模擬退火演算法處理捨入資料對參數估計的問題。其中利用軟體 Mathematica (7版) 編寫各例題所需要的程式碼，並附上實際操作的結果，可作為提供日後有興趣研讀此書或需要處理相關問題的人士參考的工具書。

　　This paper is refer to the chapter 1 to chapter 5 (except chapter 4 ) of the book, Monte Carlo Statistical Methods(second edition), the author is Robert and Casella(2004). The goal is to translate the chapter 1 to chapter 5 contents of this book into Chinese, modify the mistakes, add the details of the examples, translate the algorithm of the examples into Mathematica(7th) codes, and use the Simulated Annealing method to deal with the estimation of parameters by rounding data, and discuss the results. This paper provides Mathematica(7th) codes of almost every example, and show the actual results, so it can be regarded as a toolbook for those people who are interested in reading this book or may solve some problems related to those examples.

目錄
誌謝　i
摘要　ii
Abstract　iii
1 介紹(Introduction) 1
　1.1 統計模型(Statistical Models) 1
　1.2 概似估計法(Likelihood Methods) 4
　1.3 貝氏方法(Bayesian Method) 10
　1.4 決定性數值法(Deterministic Numerical Method) 16
　　1.4.1 最佳化(Optimization) 16
　　1.4.2 積分(Integration) 18
　　1.4.3 比較(Comparison) 18
2 隨機變數的生成(Random Variable Generation) 20
　2.1 簡介(Introduction)　20
　　2.1.1 一致分佈的模擬(Uniform Simulation) 20
　　2.1.2 逆變換(The Inverse Transform) 21
　　2.1.3 其他例題(Alternatives) 23
　2.2 一般變換方法(General Transformation Methods ) 23
　2.3 接受拒絕法(Accept-Reject Method ) 28
　　2.3.1 模擬的基礎定理(The Fundamental Theorem of Simulation ) 29
　　2.3.2 接受拒絕演算法(The Accept-Reject Algorithm ) 32
　2.4 包絡接受拒絕法(Envelope Accept-Reject Method) 34
　　2.4.1 夾擠原理(The Squeeze Principle) 34
　　2.4.2 對數凹密度函數(Log-Concave Densities ) 　36
3 蒙地卡羅積分(Monte Carlo Integration) 43
　3.1 簡介(Introduction) 43
　3.2 典型蒙地卡羅積分(Classical Monte Carlo Integration) 47
　3.3 重要抽樣(Importance Sampling) 52
　　3.3.1 原理(Principles)　52
　　3.3.2 變異數有限之估計量(Finite Variance Estimators) 57
　　3.3.3 重要抽樣與接受拒絕法比較(ImportSampling vsAR method) 64
　3.4 拉普拉斯近似(Laplace Approximations) 68
4 蒙地卡羅最佳化(Monte Carlo Optimization) 70
　4.1 介紹(introduction) 70
　4.2 隨機探索(Stochastic Exploration) 71
　　4.2.1 基本解(A Basic Solution) 71
　　4.2.2 梯度法(Gradient Methods) 73
　　4.2.3 模擬退火演算法(Simulated Annealing) 75
　　4.2.4 事前回饋(Prior Feedback) 80
　4.3 隨機逼近算法(Stochastic Approximation)　82
　　4.3.1 遺失資料模型及去邊際化(MissData Models and Demarginal) 82
　　4.3.2 EM演算法( The EM Algorithm) 84
　　4.3.3 蒙地卡羅EM(Monte Carlo EM) 91
　　4.3.4 EM 標準差( EM Standard Errors) 95
5 最佳化問題之模擬研究：捨入資料對最大概似估計量的影響　100
　5.1 概似函數之參數估計　100
　　5.1.1 常態分佈的參數估計　101
　　5.1.2 柯西分佈的參數估計　102
　　5.1.3 指數分佈的參數估計　103
　　5.1.4 Gamma分佈的參數估計　103
　5.2 模擬結果　104
參考文獻　106
附錄　111
勘誤表　111
表目錄 114

表 1-1 太空梭起飛時的溫度及O-ring零件的狀態. . . . . . . . . . . . . . . .114
表 2-1 普魯士軍隊遭受馬匹踢死的數據. . . . . . . . . . . . . . . . . . . . . .114
表 3-1 常態分佈表. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .114
表 3-2 列聯表. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .114
表 3-3 卡方檢定的截點. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115
表 3-4 事後期望值之估計值. . . . . . . . . . . . . . . . . . . . . . . . . . . .115
表 3-5 積分值的近似結果. . . . . . . . . . . . . . . . . . . . . . . . . . . . .115
表 4-1 模擬退火演算法估計最小值結果. . . . . . . . . . . . . . . . . . . . . .115
表 4-2 模擬退火演算法之接受率. . . . . . . . . . . . . . . . . . . . . . . . .115
表 4-3 參數估計結果. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116
表 4-4 入學考試成績. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116
表 4-5 期望值之最大概似估計量. . . . . . . . . . . . . . . . . . . . . . . . .116
圖目錄 117

圖 1-1 柯西分佈的概似函數圖. . . . . . . . . . . . . . . . . . . . . . . .117
圖 1-2 牛頓-拉福生法. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117
圖 2-1-1 數對 (Yn, Yn+100) 的散佈圖. . . . . . . . . . . . . . . . . . . .118
圖 2-1-2 數列 Yn 的直方圖. . . . . . . . . . . . . . . . . . . . . . . . .118
圖 2-2 演算法，接受(U, V )的機率. . . . . . . . . . . . . . . . . . . .118
圖 2-3 貝它分佈隨機變數的生成. . . . . . . . . . . . . . . . . . . . . . . . .118
圖 2-4 來自集合{(x, u) : 0 < u < f(x)}的一致分佈樣本. . . . . . . . . . . .119
圖 2-5 h(x) = log f(x)的上下界線. . . . . . . . . . . . . . . . . . . . . . .119
圖 2-6 北方針尾鴨資料. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .120
圖 2-7 ARS演算法生成的5000筆樣本. . . . . . . . . . . . . . . . . . . . . .120
圖 2-8 區域 [x2, x3] 所對應的機率、ARS演算法所得樣本. . . . . . . . . . . .121
圖 3-1 蒙地卡羅積分法近似結果. . . . . . . . . . . . . . . . . . . . . . . . .121
圖 3-2-1 虛無假設下之分佈的直方圖及近似卡方分佈的密度函數. . . . . . . 121
圖 3-2-2 移動經驗百分位數. . . . . . . . . . . . . . . . . . . . . . . . . . . .122
圖 3-3 經驗累積分佈函數. . . . . . . . . . . . . . . . . . . . . . . . . . . . .122
圖 3-4-1 重要抽樣法估計風險R1 . . . . . . . . . . . . . . . . . . . . . . . . .122
圖 3-4-2 重要抽樣法估計風險R2 . . . . . . . . . . . . . . . . . . . . . . . . .123
圖 3-5 重要抽樣法估計Ef [h1(X)]結果. . . . . . . . . . . . . . . . . . . . . .123
圖 3-6 雙重伽瑪分佈估計Ef [h1(X)]結果. . . . . . . . . . . . . . . . . . . .124
圖 3-7 重要抽樣法估計Ef [h2(X)]結果. . . . . . . . . . . . . . . . . . . . . .124
圖 3-8 重要抽樣法估計Ef [h3(X)]結果. . . . . . . . . . . . . . . . . . . . . .124
圖 3-9 估計E[h5]的結果. . . . . . . . . . . . . . . . . . . . . . . . . . . . .125
圖 3-10 估計E[h3(X)] = E[x / (1 + x)]的收斂結果. . . . . . . . . . . . . . . 125
圖 4-1-1 h(x)的函數圖. . . . . . . . . . . . . . . . . . . . . . . . . . . . .125
圖 4-1-2 U[0, 1]樣本計算h(x)的結果. . . . . . . . . . . . . . . . . . . . . .126
圖 4-2 h(x, y)的函數圖. . . . . . . . . . . . . . . . . . . . . . . . . . . . .126
圖 4-3 梯度法所得θj的收斂路徑. . . . . . . . . . . . . . . . . . . . . . . . .127
圖 4-4 模擬退火演算法所得( x(t), h( x(t) ) )的軌跡. . . . . . . . . . . . . . 127
圖 4-5 模擬退火演算法所得( xt , yt )的軌跡. . . . . . . . . . . . . . . . . . . .128
圖 4-6-1 三種概似函數的圖形. . . . . . . . . . . . . . . . . . . . . . . . . . .128
圖 4-6-2 EM估計結果. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .129
圖 4-7 混合模型的對數概似函數. . . . . . . . . . . . . . . . . . . . . . . . .129
圖 4-8 遺傳連鎖資料之參數的EM估計結果. . . . . . . . . . . . . . . . . . .129
圖 5-1-1 常態分佈的模擬結果： 的bias . . . . . . . . . . . . . . . . . . . . .130
圖 5-1-2 常態分佈的模擬結果： 的bias . . . . . . . . . . . . . . . . . . . . .131
圖 5-1-3 常態分佈的模擬結果： 的variance . . . . . . . . . . . . . . . . . .132
圖 5-1-4 常態分佈的模擬結果： 的variance . . . . . . . . . . . . . . . . . .133
圖 5-2-1 柯西分佈的模擬結果： 的bias . . . . . . . . . . . . . . . . . . . . .134
圖 5-2-2 柯西分佈的模擬結果： 的bias . . . . . . . . . . . . . . . . . . . . .135
圖 5-2-3 柯西分佈的模擬結果： 的variance . . . . . . . . . . . . . . . . . . .130
圖 5-2-4 柯西分佈的模擬結果： 的variance . . . . . . . . . . . . . . . . . .137
圖 5-3-1 指數分佈的模擬結果： 的bias . . . . . . . . . . . . . . . . . . . . .138
圖 5-3-2 指數分佈的模擬結果： 的variance . . . . . . . . . . . . . . . . . .139
圖 5-4-1 Gamma分佈的模擬結果： 的bias . . . . . . . . . . . . . . . . . .140
圖 5-4-2 Gamma分佈的模擬結果： 的bias . . . . . . . . . . . . . . . . . . .141
圖 5-4-3 Gamma分佈的模擬結果： 的variance . . . . . . . . . . . . . . . .142
圖 5-4-4 Gamma分佈的模擬結果： 的variance . . . . . . . . . . . . . . . .143

[1] Aarts, E. and Kors, T. (1989). Simulated Annealing and Boltzman Machines: A Stochastic Approach to Combinatorial Optimisation and Neural Computing.
John Wiley, New York.
[2] Agresti, A. (1992). A survey of exact inference for contingency tables (with discussion). Statist. Science, 7:131-177.
[3] Ahrens. J. and Dieter, U. (1974). Computer methods for sampling from gamma, beta, Poisson and binomial distributions. Computing, 12:223-246.
[4] Aldous, D. (1987). On the Markov chain simulation method for uniform combinatorial distributions and simulated annealing. Pr. Eng. Inform. Sciences, 1:33-46.
[5] Atkinson, A. (1979). The computer generation of Poisson random variables. Appl. Statist., 28:29-35.
[6] Ball, F., Cai, Y., Kadane, J., and O’Hagan, A. (1999). Baysian inference for ion channel gating mechanisms directly from single channel recordings, using Markov chain Monte Carlo. Proc. Royal Society London A, 455:2879-2932.
[7] Barnett, G., Kohn, R., and Sheather, S. (1996). Bayesian estimation of an autoregressive model using Markov chian Monte Carlo. J. Econometrics, 74:237-254.
[8] Berger, J. (1985). Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, New York, second edition.
[9] Berger, J. (1990). Robust Bayesian analysis: sensitivity to the prior. J. Statist. Plann. Inference, 25:303- 328.
[10] Berger, J. (1994). An overview of robust Bayesian analysis(with discussion). TEST, 3:5-124.
[11] Berger, J. and Bernardo, J. (1992). On the development of the reference prior method. In Berger, J., Bernardo, J., Dawid, A., and Smith, A., editors, Bayesian Statistic 4, pages 35-49. Oxford University Press, London.
[12] Berger, J. and Pericchi, L. (1998). Accurate and stable Bayesian model selection: the median intrinsic Bayes factor. Sankhya B, 60:1-18.
[13] Berger, J., Philippe, A., and Robert, C. (1998). Estimation of quadratic functions: reference priors for non-centrality parameters. Statistica Sinica, 8(2):359-
375.
[14] Bernardo, J. (1979). Reference posterior distributions for Bayesian inference (with discussion). J. Royal Statist. Soc. Series B, 41:113- 147.
[15] Bernardo, J. and Gir`on, F. (1986). A Bayesian approach to cluster analysis. In Second Catalan International Symposium on Statistics, Barcelona, Spain.
[16] Bernardo, J. and Giron, F. (1988). A Bayesian analysis of simple mixture problems. In Bernardo, J., DeGroot, M., Lindley, D., and Smith, A., editors, Bayesian Statistics 3, pages 67-78. Oxford University Press, Oxford.
[17] Bernardo, J. and Smith, A. (1994). Bayesian Theory. John Wiley, New York.
[18] Berthelsen, K. and Moller, J. (2003). Likelihood and non-parametric Bayesian MCMC infcrence for spatial point processes based on perfect simulation and path sampling. Scandinavian J. Statist., 30:549-564.
[19] Billio, M., Monfort, A., and Robert, C. (1998). The simulated likelihood ratio method. Technical Report 9821, CREST, INSEE, Paris.
[20] Box. G. and Muller, M. (1958). A note on the generation of random normal variates. Ann. Mathemat. Statist., 29:610-611.
[21] Boyles, R. (1983). On the convergence of the EM algorithm. J. Royal Statist. Soc. Series B, 45:47-50.
[22] Breslow, N. and Clayton, D. (1993). Approximate inference in generalized linear mixed models. J. American Statist. Assoc., 88:9-25.
[23] Carlin, B. and Louis, T. (2001). Bayes and Empirical Bayes Methods for Data Analysis. Chapman and Hall, New York, second edition.
[24] Casella, G., and Berger, R. (2001). Statistical Inference. Wadsworth, Belmont, CA.
[25] Castledine, B. (1981). A Bayesian analysis of multiple-recapture sampling for a closed population. Biometrika, 67:197-210.
[26] Chen, M. and Shao, Q. (1997). On Monte Carlo methods for estimating ratios of normalizing constants. Ann. Statist., 25:1563-1594.
[27] Cheng, R. (1977). The generation of gamma ariables with non-integral shape parameter. Applied Statistics (Ser. C), 26:71-75.
[28] Cheng, R. and Fest, G. (1979). Some simple gamma variate generators. Appl. Statist., 28:290-295.
[29] Cipra, B. (1987). An introduction to the Ising model.
American Mathematical Monthly, 94:937-959.
[30] Dempster, A., Laird, N., and Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Royal Statist. Soc. Series B, 39:1-38.
[31] D’Epifanio. (1996). Notes on a recursive procedure for point estimate. TEST, 5:203-225.
[32] Devroye, L. (1981). The computer generation of Poisson random variables. Computing, 26:197-207.
[33] Devroye, L. (1985). Non-Uniform Random Variate Genemtion. Springer-Verlag, New York.
[34] Eberly, L. E. and Casella, G. (2003). Estimating Bayesian credible intervals. J. Statist. Plann. Inference, 112:115-132.
[35] Evans, M. and Swartz, T. (1995). Methods for approximating integrals in Statistics with special emphasis on Bayesian integration problems. Statist. Science, 10:254-272.
[36] Geweke, J. (1991). Efficient simulation from the multivariate normal and student t-distributions subject to linear constraints. Computer Sciences and Statistics: Proc. 23d Symp. Interface.
[37] Geyer, C. (1996). Estimation and Optimization of functions. In Gilks, W., Richardson, S., and Spiegelhalter, D., editors, Markov chain Monte Carlo in Practice, pages 241-258. Chapman and Hall,
New York.
[38] Haario, H. and Sacksman, E. (1991). Simulated annealing in general state space. Adv. Appl. Probab,. 23:866-893.
[39] H`ajek, B. (1988). Cooling schedules for optimal annealing. Math. Operation. Research, 13:311-329.
[40] Hesterberg, T. (1998). Weighted average importance sampling and defensive mixture distributions. Technometrics, 37:185-194.
[41] Hwang. C. (1980). Laplace’s method revisited: Weak convergence of probability measures. Ann. Probab., 8:1177-1182.
[42] Johnson, D. and Hoeting, J. (2003). Autoregressive models for capturerecapture data: A Bayesian approach. Biometrics, 59(2):341-350.
[43] Laird, N., and Stram, D. (1987). Maximum likelihood computations with repeated measures: Application of the EM algorithm. J. American Statist. Assoc., 82:97-105.
[44] Lavielle, M. and Moulines, E. (1997). On a stochastic approximation version of the EM algorithm. Statist. Compute., 7:229-236.
[45] Liu. J. (1996a). Metropolized independent sampling with comparisions to rejection sampling and imporlance sampling. Statistics and Computing, 6:113-119.
[46] Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., and Teller, E. (1953). Equations of state calculations by fast computing machines. J. Chem. Phys, 21:1087-1092.
[47] Murray, G. (1977). Comments on ”Maximum likelihood from incomplete data via the EM algorithm”. J. Royal Statist. Soc. Series B, 39:27-28.
[48] Robbins, H. and Monro, S. (1951). A stochastic approximation method. Ann. Mathemat. Statist., 22:400-407.
[49] Robert, Christian P. and Casella, George. (2004).
Monte Carlo Statistical Methods. Springer-Verlag, New York.
[50] Robert, C. (1991). Generalized inverse normal distributions. Statist. Prob. Lett., 11:37-41.
[51] Robert, C. (1993). Prior feedback: A Bayesian approach to maximum likelihood etimation. Comput Statist., 8:279-294.
[52] Rubinstein, R. (1981). Simulation and the Monte Carlo Method. John Wiley, New York.
[53] Scherrer, J. (1997). Monte Carlo estimation of transion probabilities in capturerecapture data. Technical report, Biometrics Unit, Cornell Univ., Ithaca, New York. Masters Thesis.
[54] Seber, G. (1983). Capture-recapture methods. In Kotz, S. and Jonhson, N., editors, Encyclopedia of Statistical Science. John Wiley, New York.
[55] Wu, C. (1983). On the convergence properties of the EM algorithm. Ann Statist., 11:95-103.
[56] 李根良(2010)，「四捨五入型資料參數估計之研究」。國立中山大學應用數學系碩士論文。