Mesh based segmentation and update for object based video

Abstract

There is no normative segmentation method in the emerging MPEG-4 standard. In this paper, we propose mesh-based segmentation method for object-based video which fuses mesh-based motion and color information using region approaches instead of pixel-based ones suitable for MPEG-4. A 2D Delaunay triangulated dynamic content-based mesh is initially designed on the first frame of the sequence with an optimal number of nodes. Nodes motion values and motion values of their neighbors are used to form a test region of triangles for the determination of the boundary of the video object. The formed test region is then remeshed and the whole mesh is updated. Color information of the neighboring triangles on one step layers provides us with a refinement of the boundary of the video object. The inside of the refined boundary is optimally meshed and a search mechanism is used through layers. The mesh based system can handle multiple video objects and the related results are given. The proposed 2D mesh based segmentation, occlusion detection and representation method suitable for progressive transmission can be applied in object-based video coding, storage and manipulation.