在視訊資料庫中，每段視訊都包含物體間的時間關係與空間關係。時間關係能藉由畫面先後順序來確定。而空間關係則可藉由同一個畫面間，物體間的相對關係來確定。除此之外，最明顯的資訊之一，便是隨著時間而變動位置和移動速度的物體。最近幾年來，有許多用來縮短搜尋時間的視訊索引方法被提出來。而這些方法都包含著以上所提到的資訊。舉例來說，3D C-string 方法利用了物體的投影，來表示視訊中物體之間空間和時間的關係。除此之外，3D C-string 方法還能追蹤到物體在視訊中的移動和大小的變化。然而，這3D C-string 方法產生了三個問題。首先，它不能索引某些種類的視訊，如物體不僅一次的出現後又消失的視訊。再者，3D C-string 推導出空間關係的過程又太過複雜。最後，由於 3D C-string 只記錄了物體間的相對位置，它無法推導出物體本身的絕對位置。在本論文裡，我們提出了三種新的空間關係。而利用這三個關係，我們可以表達出物體消失與出現的情形。除此之外，我們可以藉由連續的空間關係變化字串，推導出時間關係出來。以此技術為基礎，我們提出三個視訊資料庫的索引方法。第一個提出的方法是Temporal UID Matrix (TUID)。我們使用了在 UID 方法裡的13個識別碼，加上剛新增的三個識別碼，來代表空間關係。然後，我們會將連續的空間關係變化字串，儲存在 TUID 的陣列(Matrix)裡。藉由這種方法，我們可以有效率的支援空間關係、時間關係、與及空間-時間關係的查詢方式。然而，由於 TUID 方法並沒有記錄物體的資訊，它沒有辦法支援物體資訊的查詢方式。所以，我們提出了第二種方法— 2D Video String，來追蹤視訊中物體的移動、位置和大小的變化。即使 2D Video String 方法能夠支援所有的查詢方式，但和 TUID 方法相較之下，它的查詢效率仍是較低的。最後藉著利用這兩種方法的優點，我們提出了另一種索引方法—the Hybrid。我們在 TUID 的陣列對角處記錄了物體的資訊。在我們的模擬研究中，除了 2D Video String 方法裡的時間關係查詢方式，我們顯示了我們所提出的方法能夠爲視訊資料庫提供比Lee等人提出的 3D C-string 方法較短的搜尋時間。

In the video database, each video contains temporal
and spatial relationships between content objects. The temporal relationships can be specified between frame sequences and the spatial relationships can be specified by the relationships between objects in a single frame.
Moreover, the information related to locations and motions of objects is included in video database. Many video indexing strategies have been proposed, which include the above information to speed up the query processing time. For example, the 3D C-string strategy, it uses the projections of objects to represent spatial and temporal relations between objects in a video. Moreover, the 3D C-string strategy can keep track of the motions and size changes of the objects in a video. However, there are three problems caused by the 3D C-string strategy. The first one is that it cannot index some kinds of videos in which an object appears and then disappears for more than one time. The second one is that the representation of the 3D C-string is too complex for deriving spatial relationships. The last one is that the 3D C-string cannot derive the absolute locations of objects, since it records the relative locations of objects. In this thesis, in order to solve the problems of the 3D C-string strategy, we propose three new spatial relationships. By making use of the three spatial relationships, we can express the condition that objects disappear and appear. Moreover, based on the sequence of spatial relationships, we can derive the temporal relationships. Based on this technique, we propose three index processing strategies for video database. The first strategy is the Temporal UID Matrix (TUID) strategy. We use those 13 unique numbers used in the UID strategy and our 3 new added unique numbers to represent spatial relationships. Then, we store the sequence of spatial relationships in the TUID matrix. In this way, we can efficiently support query types of spatial, temporal, and spatio-temporal relationships. However, since the TUID strategy does not record the information of objects, it cannot support the query type by the information of objects. Therefore, we propose the second strategy, the 2D Video String strategy, to keep track of the motions, locations, and size changes associated with the video objects. Although the 2D Video String strategy can support all types of queries, it is less efficient than the TUID strategy. By making use of the advantages of both strategies, we propose another video indexing strategy, the Hybrid strategy. We record the information of objects in the diagonal part of the TUID matrix. From our simulation study, we show that our proposed strategies can provide a shorter search time for video data than Lee et al.'s 3D C-string strategy, except the 2D Video String strategy for the temporal query.

TABLE OF CONTENTS Page
ABSTRACT . . . . . . . i
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Video Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Query Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 The CVQL Video Query Language . . . . . . . . . . . . . . . . . . . 4
1.4 Iconic Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Problems in Indexing Video Data . . . . . . . . . . . . . . . . . . . . 7
1.6 Video Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.7 Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.8 Organization of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . 17
2. A Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1 Successive Modular Operations . . . . . . . . . . . . . . . . . . . . . 18
2.2 3D-List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3 2D C-trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4 9DLT Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5 STCI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.6 Binary String Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.7 3D C-strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3. Video Indexing Strategies . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.1 Three New Spatial Relationships . . . . . . . . . . . . . . . . . . . . 37
3.2 The Temporal Unique-ID-Based (TUID) Strategy . . . . . . . . . . . 40
3.2.1 The Unique-ID-Based Matrix (UID Matrix) . . . . . . . . . . 40
3.2.2 Deriving Temporal Relationships . . . . . . . . . . . . . . . . 44
3.2.3 The Temporal Unique-ID-Based Matrix (TUID Matrix) . . . . 46
3.2.4 Query Processing of the Temporal UID Strategy . . . . . . . . 51
3.3 The 2D Video String Strategy . . . . . . . . . . . . . . . . . . . . . . 57
3.3.1 The Index Data Structure . . . . . . . . . . . . . . . . . . . . 58
3.3.2 Query Processing of the 2D Video String Strategy . . . . . . . 60
3.4 The Hybrid Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.4.1 The Index Data Structure . . . . . . . . . . . . . . . . . . . . 63
3.4.2 Query Processing of the Hybrid Strategy . . . . . . . . . . . . 65
4. Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.1 The Performance Model . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.2 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

[1] Z. Aghbari, K. Kaneko, and A. Makinouchi, “Content-Trajectory Approach for Searching Video Databases,” IEEE Trans. on Multimedia, Vol. 5, No. 4, pp. 516-531, Dec.2003.
[2] J. F. Allen, “Maintaining Knowledge about Temporal Intervals,” Communications of the ACM, Vol. 26, No. 11, pp. 832-843, Nov. 1983.
[3] F. Arman, R. Depommier, A. Hsu, and M. Y. Chiu, “Content-Based Browsing of Video Sequences,” Proc. of the 2nd ACM Int. Conf. on Multimedia, pp. 97-103, 1994.
[4] Y. K. Chan and C. C. Chang, “An Efficient Data Structure for Storing Similar Binary Images,” Proc. of the 5th Int. Conf. on Foundations of Data Organization, pp. 268-275, Nov. 1998.
[5] Y. K. Chan and C. C. Chang, “Spatial Similarity Retrieval in Video Databases,” Journal of Visual Communication and Image Representation, Vol. 12, No. 2, pp. 107-122, June 2001.
[6] C. C. Chang, “Spatial Match Retrieval of Symbolic Pictures,” Journal of Information Science and Eng., pp. 405-422, 1991.
[7] C. C. Chang and C. F. Lee, “Relative Coordinates Oriented Symbolic String for Spatial Relationship Retrieval,” Pattern Recognition, Vol. 28, No. 4, pp. 563-570, April 1995.
[8] C. W. Chang and S. Y. Lee, “Video Content Representation, Indexing, and Matching in Video Information Systems,” Journal of Visual Communication and Image Representation, Vol. 8, No. 2, pp. 107-120, June 1997.
[9] S. K. Chang, Q. Y. Shi, and C. W. Yan, “Iconic Indexing by 2D Strings,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 9, No. 3, pp. 413-429, May 1987.
[10] Y. I. Chang, H. Y. Ann, and W. H. Yeh, “A Unique-Id-Based Matrix Strategy for Efficient Iconic Indexing of Symbolic pictures,” Pattern Recognition, Vol. 33, No. 8, pp. 1263-1276, Aug. 2000.
[11] L. Chen, V. Oria, and M. T. Ozsu, “A Multi-Level Index Structure for Video Databases,” Multimedia Information Systems 2002, pp. 28-37, 2002.
[12] T. S. Chua and L. Q. Ruan, “A Video Retrieval and Sequencing System,” ACM Tran. on Information Systems, Vol. 13, No. 4, pp. 373-407, Oct. 1995.
[13] M. E. Donderler, O. Ulusoy, and U. Gudukbay, “A Rule-Based Approach to Represent Spatio-Temporal Relations in Video Data,” Int. Conf. on Advances in Information Systems, pp. 409-418, Oct. 2000.
[14] B. Erol and F. Kossentini, “Shape-based Retrieval of Video Objects,” IEEE Transactions on Multimedia, Vol. 7, No. 1, pp. 179-181, Feb. 2005.
[15] M. S. Hacid, C. Decleir, and J. Kouloumdjian, “A Database Approach for Modeling and Querying Video Data,” IEEE Trans. on Knowledge and Data Engineering, Vol. 12, No. 5, pp. 729-750, Oct. 2000.
[16] M. Hadjieleftheriou, G. Kollios, V. J. Tsotras, and D. Gunopulos, “Efficient Indexing of Spatio-temporal Objects,” Proc. of the Int. Conf. Extending Database Technology (EDBT), pp. 251-268, 2002.
[17] A. Hampapur, T. E. Weymouth, and R. Jain, “Digital Video Segmentation,” Proc. of the 2nd ACM Int. Conf. on Multimedia, pp. 357-364, 1994.
[18] A. Hanjalic and L. Q. Xu, “Affective Video Content Representation and Modeling,” IEEE Transactions on Multimedia, Vol. 7, No. 1, pp. 143-154, Feb. 2005.
[19] F. J. Hsu and S. Y. Lee, “Similarity Retrieval by 2D C-Trees Matching in Image Databases,” Journal of Visual Communication and Image Representation, Vol. 9, No. 1, pp. 87-100, March 1998.
[20] F. J. Hsu, S. Y. Lee, and B. S. Lin, “Video Data Indexing by 2D C-Trees,” Journal of Visual Languages and Computing, Vol. 9, No. 4, pp. 375-397, Sept. 1998.
[21] F. J. Hsu, S. Y. Lee, and B. S. Lin, “2D C-Tree Spatial Representation for Iconic Images,” Journal of Visual Languages and Computing, Vol. 10, No. 2, pp. 147-164, April 1999.
[22] P. W. Huang and Y. R. Jean, “Using 2D C+-Strings as Spatial Knowledge Representation for Image Database Systems,” Pattern Recognition, Vol. 27, No. 9, pp. 1249-1257, Sept. 1994.
[23] V. Kobla, D. DeMenthon, and D. Doermann, “Identifying Sports Video Using Replaying, Text, and Camera Motion Features,” Proc. of the SPIE Conf. on Storage and Retrieval for Media Databases, pp. 332-342, 2000.
[24] G. Kollios, V. J. Tsotras, D. Gunopulos, A. Delis, and M. Hadjieleftheriou, “Indexing Animated Object Using Spatiotemporal Access Methods,” IEEE Trans. on Knowledge and Data Engineering, Vol. 13, No. 5, pp. 758-777, Oct. 2001.
[25] M. Koprulu, N. K. Cicekli, and A. Yazici, “Spatio-Temporal Querying in Video Databases,” Information Sciences, Vol. 160, No. 1-4, pp. 131-152, March 2004.
[26] T. C. T. Kuo and A. L. P. Chen, “Content-Based Query Processing for Video Databases,” IEEE Tran. on Multimedia, Vol. 2, No. 1, pp. 1-13, 2000.
[27] A. J. T. Lee, H. P. Chiu, and P. Yu, “3D C-String: A New Spatio-Temporal Knowledge Representation for Video Database Systems,” Pattern Recognition, Vol. 35, No. 11, pp. 2521-2537, 2002.
[28] C. Lee and T. M. Tseng, “Temporal Grid File: A File Structrue for Interval Data,” Data and Knowledge Engineering, Vol. 26, No. 1, pp. 71-97, May 1998.
[29] S. Y. Lee and F. J. Hsu, “2D C-String: A Spatial Knowledge Representation for Image Database Systems,” Pattern Recognition, Vol. 23, No. 10, pp. 1077-1088, Oct. 1990.
[30] S. Y. Lee and F. J. Hsu, “Spatial Reasoning and Similarity Retrieval of Images Using 2D C-String Knowledge Representation,” Pattern Recognition, Vol. 25, No. 3, pp. 305-318, March 1992.
[31] S. Y. Lee, M. K. Shan, and W. P. Yang, “Similarity Retrieval of Iconic Image Database,” Pattern Recognition, Vol. 22, No. 6, pp. 675-682, 1989.
[32] S. Y. Lee, M. C. Yang, and J. W. Chen, “2D B-String: A Spatial Knowledge Representation for Image Database Systems,” Proc. of the 2nd Int. Computer Sci. Conf., pp. 609-615, 1992.
[33] S. Y. Lee, M. C. Yang, and J. W. Chen, “Signature File as A Spatial Filter for Iconic Image Database,” Journal of Visual Languages and Computing, Vol. 3, No. 4, pp. 373-397, Dec. 1992.
[34] C. H. Lin and A. L. P. Chen, “Motion Event Derivation and Query Language for Video Databases,” Storage and Retrieval for Media Databases 2001, Proc. Of SPIE, pp. 208-218, 2001.
[35] C. C. Liu and A. L. P. Chen, “3D-List: A Data Structure for Ecient Video Query Processing,” IEEE Trans. on Knowledge and Data Engineering, Vol. 14, No. 1, pp. 106-122, Feb. 2002.
[36] C. C. Lo, S. Wang, and L. W. Huang, “Video Retrieval Using Successive Modular Operations on Temporal Similarity,” Computer Standards and Interfaces, Vol. 26, No. 4, pp. 317-328, Aug. 2004.
[37] D. Papadias, N. Mamoulis, and V. Delis, “Approximate Spatio-Temporal Retrieval,” ACM Trans. on Information Systems, Vol. 19, No. 1, pp. 53-96, Jan. 2001.
[38] S. Park and K. H. Hyun, “Trie for Similarity Matching in Large Video Databases,” Information Systems, Vol. 29, No. 8, pp. 641-652, Dec. 2004.
[39] M. K. Shan and S. Y. Lee, “A Framework for Temporal Similarity Measures of Content-Based Scene Retrieval,” Pattern Recognition Letters, Vol. 22, No. 5,
pp. 517-532, April 2001.
[40] K. Shearer, H. Bunke, and S. Venkatesh, “Video Indexing and Similarity Retrieval by Largest Common Subgraph Detection Using Decision Trees,” Pattern Recognition, Vol. 34, No. 5, pp. 1075-1091, May 2001.
[41] K. Shearer, D. Kieronska, and S. Venkatesh, “Resequencing of Video using Spatial Indexing,” Journal of Visual Languages and Computing, Vol. 8, No. 2, pp. 193-214, April 1997.
[42] S. W. Smoliar and H. Zhang, “Content-Based Video Indexing and Retrieval,” IEEE Multimedia, Vol. 1, No. 2, pp. 62-72, 1994.
[43] S. L. Tanimoto, “An Iconic/Symbolic Data Structuring Scheme,” Pattern Recognition and Articial Intelligence, pp. 452-471, 1976.
[44] D. Vasilis and H. Thanasis, “Binary String Relations: A Foundation for Spatio -temporal Knowledge Representation,” Proc. of the 8th Int. Conf. on Information
and Knowledge Management, pp. 27-37, 1999.
[45] M. Vazirgiannis, Y. Theodoridis, and T. Sellis, “Spatio-temporal Composition and Indexing for Large Multimedia Applications,” Multimedia Systems, Vol. 6, No. 4, pp. 284-298, July 1998.
[46] T. T. Y. Wai and A. L. P. Chen, “Retrieving Video Data via Motion Tracks of Content Symbols,” Proc. of ACM Int. Conf. on Information and Knowledge Management (CIKM), 1997.