Recent advancements in 3D television allow for the capture of scene depth from multiple cameras and the interactive selection of view point and direction within a certain range, the so-called Free Viewpoint Video (FVV). State-of-the-art video codecs such as H.264/AVC exploit the large amount of inter-view statistical dependencies by combined temporal and inter-view prediction, i.e. prediction from temporally neighboring images as well as from images in adjacent views. This is known as Multi-view Video Coding (MVC). We propose herein an alternative object oriented video coding scheme for multi-view video with associated multiple depth data (N-video plus N-depth). A structure that we call a Multi-view Video Plane (MVP) is introduced. Object planes associated with a certain view are approximated as multilinear components of an image that are projected onto other views in a tensor-like fashion. The order of the tensor equals the number of multiple views. The coefficients of the tensor subspace projections as well as the updates of the multi-linear components (object-planes) are quantized and transmitted in the MPEG stream. Motion-compensated prediction is carried out in order to transmit the residual object planes (P-frames) using conventional MPEG algorithms.