Applied optics. Computer Science, Medicine. View 5 excerpts, cites methods. On the performance of objective quality metrics for lightfields. Signal Process. Image Commun. Highly Influential.
View 4 excerpts, references background and methods. View 1 excerpt, references background. Fast depth estimation using non-iterative local optimization for super multi-view images. Mathematics, Computer Science. View 2 excerpts, references background. Multi-camera Scene Reconstruction via Graph Cuts. This volume demonstrates the power of the Markov random field MRF in vision, treating the MRF both as a tool for modeling image data and, utilizing recently developed algorithms, as a means of … Expand.
View 1 excerpt, references methods. View 2 excerpts, references background and methods. Related Papers. In real scenes, the ordering constraint is often violated in the case of big perspective changes or in the case of thin objects.
In such cases, ordering constraint can introduce errors in the estimated disparity maps. The uniqueness constraint [ 28 , 29 ] imposes one-to-one correspondence between the pixel in the center and in the side views. If a given pixel A in the central view is assigned to a corresponding pixel B in the side view, then no other pixel in the central view can be assigned to correspondence with pixel B in the side view. This way, a unique pixel to pixel correspondence is forced across all of the views.
There are many disparity estimation algorithms known that handle occlusion in an efficient way [ 29 , 30 , 31 ]. The main drawback of all of those algorithms are additional constraints terms imposed in optimization procedures which increase complexity and thus execution time of the disparity estimation. Another approach to occlusion handling is to modify the cost term Eq. Because of the occlusions, Tanimoto [ 16 ] proposed to pick just the most similar fragment from either the left or from the right view.
The intuition is that the occluded fragment will be less similar, thus the minimum of similarity metrics from the left and the right view is used. In this paper, we propose yet another way to define the cost function which takes into account an occlusion possible within the scene. In such a case, searching for a correspondence in this particular side view left or right is pointless, as the given fragment of the scene is not visible from that particular side view.
Considering the correspondence for an occluded fragment of an image can cause errors in estimated disparity. Therefore the correspondence search should be performed only in those side views in which a considered fragment of a center view is not occluded. The cost function should be constructed in such a way that it considers only similarity metrics from non-occluded views. If a given fragment is visible in both views, then the cost function should be an average of both similarity metrics, in order to reduce the influence of noise which is present in all views.
We propose to define the cost function in a way that it considers only similarity metrics of fragments from non-occluded views either left or right Eq. If a given pixel is occluded in both side views, Eq. Consider the example in Fig. Point B is closer to the cameras and point A is farther. The distance to the camera z is reciprocal to disparity. Therefore, a fragment of an image representing a closer object point B has bigger disparity than the fragment representing the farther object point A.
Rather, it is a fragment of some other, closer object B that occludes object A in the left view. Obviously, in general, disparity maps for the left and the right views may be unknown before estimating the disparity for the central view.
Commonly, disparity maps are estimated iteratively with the use of algorithms like belief propagation or graph cuts. This disparity map is further refined in the further iterations of the algorithm. For our occlusion detection, we propose to use disparity maps of the side views created based on the disparity map of the center view through Depth-Image-Based Rendering DIBR.
Our idea can be applied to any depth estimation algorithm, as it modifies only similarity metrics. DERS is a state-of-the-art disparity estimation technique, designed with 3D video application in mind. It uses three input videos and produces a single output disparity map. In many applications which use depth maps, such as free viewpoint television, depth maps are never presented directly to the viewer, but they are mainly used for the purpose of creating an additional view of the scene by means of view synthesis [ 7 , 35 ].
Therefore, we have evaluated our proposed method indirectly, by assessing the quality of synthesized views. Such methodology is widely recognized and accepted in the literature for assessing depth maps quality [ 1 , 6 , 30 , 35 ]. In order to compare the influence of the proposed idea, we have estimated disparity maps for two views: A and B Fig. Based on the views A and B and the estimated disparity maps for views A and B , a view V that is positioned in between views A and B was synthesized.
Exact view numbers for each of the test sequences used during experiments are provided in Table 1. The quality of the estimated disparity maps for views A and B is measured as a quality of synthesized view V. The quality of synthesized view V is expressed with the PSNR of luminance in comparison with the view V captured by a real camera positioned at the same spatial position see Fig.
Comparison of depth maps fragment estimated for Poznan Carpark [ 33 ]. Standard Middlebury dataset [ 36 ] used for evaluation of the proposed algorithm.
In the course of evaluation, disparity maps were estimated for every frame of the sequences mostly frames per view. This allowed evaluation of our algorithm on a wide range of different images. The disparity estimation was done with various precisions: per-pixel, half-pixel and quarter-pixel precision.
Also, a wide range of regularization terms used in Graph Cut algorithm has been evaluated. In the experiments, a range of 1—4 for the smoothing coefficient was explored. In Fig. Obviously, depth maps near the edge of a foreground object the lamp have been improved. The comparison of quality of the estimated disparity maps for the proposed method versus the original DERS can be found in Fig. As it can be noticed, the smoothing coefficient can have a significant impact on the quality of disparity maps estimated with DERS.
It can be expected that in a real-world-use scenario, this parameter will be automatically controlled to provide the best results. Therefore, in the summarized Table 2 , we have presented only the best performing cases.
Depending on the case, the proposed occlusion handling brings a gain of 0. On average, the proposal provides an improvement of 1. We have also evaluated our algorithm using a different methodology developed in [ 36 ] and used in widely recognized Middlebury test bench. In the Middlebury methodology, the quality of depth maps is evaluated directly by counting the number of pixels where the estimated disparity differs from ground truth disparity obtained by means of structured lighting.
For the evaluation, we have modified the DERS algorithm to directly output raw disparity maps in the format required by the Middlebury evaluation webpage [ 10 ]. The application of the proposed occlusion handing algorithm on Middlebury images results in an improvement of maximally 0. Please keep in mind that Middlebury datasets have very little occlusions and thus the attained gains cannot be significant. We have presented a novel approach to occlusion handling in disparity estimation, based on a modification of the similarity cost function.
The proposed approach has been tested in a three-view disparity estimation scenario. For occlusion detection, synthesized disparity maps of the left and the right view have been used. For well-known multi-view video test sequences, the experimental results show that the proposed approach provides improvement of virtual view quality of about 1. Moreover, direct quality evaluation of estimated disparity, based on the Middlebury dataset, reveals that the proposed approach reduces the number of bad pixels by 0.
Muller, K. IEEE 99 4 , — Article Google Scholar. Zhang, L. IEEE Trans. Annex, I. Tech, G. Kim, S. Hartley, R. Cambridge University Press, Cambridge Scharstein, D. Middlebury Stereo Evaluation—Version 2.
Webpage visited Okutomi, M. PAMI 15 4 , — Collins, R. Seitz, S. Stankowski, J. In: Image Processing and Communications Challenges 2. Advances in Intelligent and Soft Computing, vol. Springer, Berlin ISBN: Google Scholar.
Tanimoto, M. M, Archamps, France Wildeboer, M. Lee, S. Stankiewicz, O. Wegner, K. Birchfield, S. Pattern Anal. Sun, J. Felzenszwalb, P.
Veksler, O.
0コメント