由於以方塊為基礎處理的技術是影像或視訊壓縮最常見的方式，因此產生了方塊效應 (blocking artifact)。方塊效應產生在方塊的邊界，其不連續性影響著視覺，是現在影像或視訊壓縮的常見問題。並且在低位元率時，方塊效應的影響更趨嚴重。因此在本篇論文中，我們提出了一個經由稀疏表示式有效的去除方塊效應之演算法。首先，利用BM3D (block-matching and 3D filtering) 演算法將影像或視訊模糊化並分解成低頻部分以及高頻部分。接著高頻部分再經由以多樣性元件分析 (morphological component analysis 或MCA) 為基礎的字典學習(dictionary learning) 及稀疏編碼 (sparse coding) 技術拆解成“方塊成份”以及“非方塊成份。”最後將“方塊成份”從具有方塊效應的影像或視訊中移除。最後的實驗結果顯示我們的演算法會優於H.264/AVC 之去除方塊效應濾波器。

Blocking artifact, characterized by visually noticeable changes in pixel values along block boundaries, is a common problem in block-based image/video compression, especially at low bitrate coding. Various post-processing techniques have been proposed to reduce blocking artifacts, but they usually introduce excessive blurring or ringing effects. This paper proposes a self-learning-based image/ video deblocking framework via properly formulating deblocking as an MCA (morphological component analysis)-based image decomposition problem via sparse representation. The proposed method first decomposes an image/video frame into the low-frequency and high-frequency parts by applying BM3D (block-matching and 3D filtering) algorithm. The high-frequency part is then decomposed into a “blocking component” and a “non-blocking component” by performing dictionary learning and sparse coding based on MCA. As a result, the blocking component can be removed from the image/video frame successfully while preserving most original image/video details. Experimental results demonstrate the efficacy of the proposed algorithm.

中文摘要 ........................................................................................................................... i
Abstract ............................................................................................................................ ii
Contents ........................................................................................................................... iii
List of Figures ................................................................................................................... v
List of Tables ................................................................................................................... vii
Chapter 1 Introduction ................................................................................................... 1
1.1 Overview of Image/Video Coding .................................................................. 1
1.2 What are Blocking Artifacts? .......................................................................... 3
1.3 Motivation ....................................................................................................... 5
1.4 Contributions................................................................................................... 6
1.5 Organization of the Thesis .............................................................................. 7
Chapter 2 Background Review ...................................................................................... 8
2.1 H.264/AVC Compression Standard and In-loop Deblocking Filter ............... 9
2.1.1 Overview of In-loop Deblocking Filter for H.264/AVC .................................. 10
2.1.2 Boundary Strength (BS) and Filtering Determination ....................................... 12
2.2 Post-processing-based Image/Video Deblocking Techniques ...................... 15
2.2.1 Image Enhancement............................................................................................... 16
2.2.2 Image Restoration .................................................................................................. 17
2.3 MCA-based Image Decomposition............................................................... 20
2.4 Sparse Coding and Dictionary Learning ....................................................... 22
Chapter 3 Proposed Image/Video Deblocking Framework via Sparse Representation ......................................................................... 24
3.1 Preprocessing and Problem Formulation ...................................................... 28
3.2 Dictionary Learning and Partition ................................................................ 29
3.3 Removal of Blocking Artifacts ..................................................................... 31
3.4 Major Differences between Existing Deblocking Method and Proposed Method ......................................................................................... 32
Chapter 4 Experimental Results .................................................................................. 34
4.1 Objective Measurement ................................................................................ 35
4.2 Experimental Results .................................................................................... 36
Chapter 5 Conclusions and Future Works ................................................................. 56
5.1 Conclusion .................................................................................................... 56
5.2 Future Work .................................................................................................. 57
Reference ...................................................................................................................... 58

[1] T. Sikora, “Trends and perspectives in image and video coding,” Proc. IEEE, vol. 93, no. 1, pp. 6–17, Jan. 2005.
[2] T. Wiegand and G. J. Sullivan, “The H.264/AVC video coding standard,” IEEE Sig. Process. Mag., vol. 24, no. 2, pp. 148–153, 2007.
[3] ITU-T Recommendation H.264 & ISO/IEC 14496-10 (MPEG-4) AVC. Advance video coding for generic audiovisual services (version 1: 2003, version 2: 2004, version 3: 2005).
[4] M. Y. Shen and C. C. J. Kuo, “Review of postprocessing techniques for compression artifacts removal,” J. Vis. Commun. Image Rep., vol. 9, no. 1, pp. 2–14, Mar. 1998.
[5] M. Yuen and H. R. Wu, “A survey of hybrid MC/DPCM/DCT video coding distortions,” Sig. Process., vol. 70, no. 3, pp. 247–278, 1998.
[6] T. C. Hsung, D. P. Lun, and W. Siu, “A deblocking technique for block-transform compressed images using wavelet transform modulus maxima,” IEEE Trans. Image Process., vol. 7, no. 10, Oct. 1998.
[7] A. S. Al-Fahoum and A. M. Reza, “Combined edge crispiness and statistical differencing for deblocking JPEG compressed images,” IEEE Trans. Image Process., vol. 10, no. 9, pp. 1288–1298, 2001.
[8] S. Liu and A. C. Bovik, “Efficient DCT-domain blind measurement and reduction of blocking artifacts,” IEEE Trans. Cir. Sys. Video Tech., vol. 12, no. 12, pp. 1139–1149, Dec. 2002.
[9] S. C. Tai, Y. Y. Chen, and S. F. Shen, “Deblocking filter for low bit rate MPEG-4 video,” IEEE Trans. Cir. Sys. Video Tech., vol. 5, no. 6, pp. 733-741, June 2005.
[10] A. Z. Averbuch, A. Schlar, and D. L. Donoho, “Deblocking of block-transform compressed images using weighted sums of symmetrically aligned pixels,” IEEE Trans. Image Process., vol. 14, no. 2, pp. 200-212, Feb. 2005.
[11] A. Foi, V. Katkovnik, and K. Egiazarian, “Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color Images”, IEEE Trans. Image Process., vol. 16, no. 5, pp. 1395-1411, May 2007.
[12] G. Zhai, W. Zhang, X. Yang, W. Lin, and Y. Xu, “Efficient image deblocking based on postfiltering in shifted windows,” IEEE Trans. Cir. Sys. Video Tech., vol. 18, no. 1, pp.122-126, Jan. 2008.
[13] G. Zhai, W. Zhang, X. K. Yang, W. Lin, and Y. Xu, “Efficient deblocking with coefficient regularization, shape adaptive filtering and quantization constraint,” IEEE Trans. on Multimedia, vol. 10, no. 8, pp. 735-745, Aug. 2008.
[14] C. H. Yeh, T. F. Ku, S. J. Fan Jiang, M. J. Chen, and J. A. Jhu, “Post-processing deblocking filter algorithm for various video decoders,” IET Image Process., in press.
[15] P. List, A. Joch, J. Lainema, G. Bjontegaard and M. Karczewicz, “Adaptive deblocking filter,” IEEE Trans. Cir. Sys. Video Tech., vol. 13, no. 7, pp. 614-619, July 2003.
[16] S. H. Shin, K. H. Park, Y. J. Chai, and T. Y. Kim, “Variable block-based deblocking filter for H.264/AVC on low-end and low-bit rates terminals,” Signal Process.: Image Commun., vol. 25, no. 4, pp. 255-267, Apr. 2010.
[17] S. W. Wu and A. Gersho, “Improved decoder for transform coding with application to the JPEG baseline system,” IEEE Trans. Commun., vol. 40, no. 2, pp. 251–254, Feb. 1992.
[18] T. Ozcelik, J. C. Brailean, and A. K. Katsaggelos, “Image and video compression algorithms based on recovery techniques using mean field annealing,” Proc. IEEE, vol. 83, no. 2, pp. 304–315, Feb 1995.
[19] J. Luo, C. W. Chen, K. J. Parker, and T. S. Huang, “Artifact reduction in low bit rate DCT-based image compression,” IEEE Trans. Image Process., vol. 5, no. 9, pp. 1363-1368, Sept. 1996.
[20] A. Zakhor, “Iterative procedure for reduction of blocking effects in transform image coding,” IEEE Trans. Cir. Sys. Video Tech., vol. 2, no. 1, pp. 91-95, Mar. 1992.
[21] Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, “Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images,” IEEE Trans. Cir. Sys. Video Tech., vol. 3, no. 8, pp. 421–432, Dec. 1993.
[22] H. Paek, R. C. Kim, and S. U. Lee, “On the POCS-based postprocessing technique to reduce the blocking artifacts in transform coded images,” IEEE Trans. Cir. Sys. Video Tech., vol. 8, no. 3, pp. 358-367, 1998.
[23] Y. Jeong, I. Kim, and H. Kang, “A practical projection-based postprocessing of block-coded images with fast convergence rate,” IEEE Trans. Cir. Sys. Video Tech., vol. 10, no. 4, pp. 617–623, Jun. 2000.
[24] C. Jung, L. Jiao, H. Qi, and T. Sun, “Image deblocking via sparse representation,” Signal Process.: Image Commun., vol. 27, no. 6, pp. 663-677, July 2012.
[25] J. M. Fadili, J. L. Starck, J. Bobin, and Y. Moudden, “Image decomposition and separation using sparse representations: an overview,” Proc. IEEE, vol. 98, no. 6, pp. 983–994, Jun. 2010.
[26] J. M. Fadili, J. L. Starck, M. Elad, and D. L. Donoho, “MCALab: reproducible research in signal and image decomposition and inpainting,” IEEE Computing in Science & Engineering, vol. 12, no. 1, pp. 44–63, 2010.
[27] B. A. Olshausen and D. J. Field, “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature, vol. 381, no. 13, pp. 607–609, June 1996.
[28] S. Mallat and Z. Zhang, “Matching pursuits with time-frequency dictionaries,” IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
[29] J. Mairal, F. Bach, J. Ponce, and G. Sapiro, “Online learning for matrix factorization and sparse coding,” J. Mach. Learn. Res., vol. 11, pp. 19–60, 2010.
[30] M. Aharon, M. Elad, and A. M. Bruckstein, “The K-SVD: an algorithm for designing of overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006.
[31] L. W. Kang, C. W. Lin, and Y. H. Fu, “Automatic single-image-based rain streaks removal via image decomposition,” IEEE Trans. Image Process., vol. 21, no. 4, pp. 1742-1755, Apr. 2012.
[32] L. W. Kang, C. W. Lin, C. T. Lin, and Y. C. Lin, “Self-learning-based rain streak removal for image/video,” in Proc. IEEE Int. Sym. Cir. Sys., Seoul, Korea, May 2012.
[33] Y. H. Fu, L. W. Kang, C. W. Lin, and C. T. Hsu, “Single-frame-based rain removal via image decomposition,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., Prague, Czech Republic, May 2011, pp. 1453–1456.
[34] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3D transform-domain collaborative filtering,” IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080-2095, Aug. 2007.
[35] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., San Diego, CA, USA, June 2005, vol. 1, pp. 886–893.
[36] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.