Overall comparison of algorithms with
sub-pixel estimation using two views |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
table below is the current, off-line version of Figure 3 of our paper. It
compares the performance of different stereo algorithms on four test |
|
|
|
|
|
|
image
pairs, including all the algorithms reported to the Middlebury website. The
numbers represent the percentage of "bad" pixels, i.e., pixel
whose |
|
|
|
|
|
absolute
disparity error is greater than [0.50 | 0.75 | 1.00 | 1.50 |2.00], the error
threshold can be changed by clicking on the tabs at the end of |
|
|
|
|
|
the
table. For each image pair, we report percentages for (1) non-occluded
pixels, (2) all pixels and (3) pixels near depth. All the scoring and
ranking |
|
|
|
|
|
schemes are the
same as the Middlebury benchmark. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
All
algorithms are run with constant parameter settings across all four images.
The rank of each algorithm in each column is indicated with a small red
number. |
|
|
|
The
algorithms are listed roughly in decreasing order of overall performance (as
measured by average rank). |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
difference between this benchmark and the Middlebury one is that we provide
sub-pixel estimation for all the algorithms reported to the Middlebury
website |
|
|
|
by
using only a single view (the reference view) or two views. The lines with
the blue highlighting are the experimental results with our sub-pixel
estimations. |
|
|
|
This
sub-pixel benchmark proves that the sub-pixel estimation algorithm proposed
is very robust; it works for all the algorithms. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Algorithms |
Avg. |
|
|
Tsukuba |
|
|
|
|
|
Venus |
|
|
|
|
|
Teddy |
|
|
|
|
|
Cones |
|
|
|
|
Rank |
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
AdaptingBP |
2.25 |
1.11 |
3 |
1.37 |
2 |
5.79 |
3 |
0.1 |
1 |
0.21 |
1 |
1.44 |
1 |
4.22 |
4 |
7.06 |
3 |
11.79 |
4 |
2.48 |
1 |
7.92 |
3 |
7.32 |
1 |
DoubleBP |
3.42 |
0.88 |
1 |
1.29 |
1 |
4.76 |
1 |
0.13 |
4 |
0.45 |
6 |
1.87 |
5 |
3.53 |
2 |
8.3 |
5 |
9.63 |
1 |
2.9 |
3 |
8.78 |
10 |
7.79 |
2 |
DoubleBP(subpixel) |
4.58 |
1.24 |
5 |
1.76 |
6 |
5.98 |
4 |
0.12 |
3 |
0.46 |
7 |
1.74 |
4 |
3.45 |
1 |
8.38 |
6 |
10.01 |
2 |
2.93 |
5 |
8.73 |
9 |
7.91 |
3 |
AdaptingBP(subpixel) |
6.25 |
1.51 |
15 |
1.94 |
10 |
7.25 |
14 |
0.12 |
2 |
0.23 |
2 |
1.61 |
3 |
4.02 |
3 |
7.12 |
4 |
11.4 |
3 |
2.95 |
6 |
8.35 |
4 |
8.58 |
9 |
SymBP+occ |
9.42 |
0.97 |
2 |
1.75 |
5 |
5.09 |
2 |
0.16 |
6 |
0.33 |
3 |
2.19 |
7 |
6.47 |
12 |
10.72 |
7 |
17.03 |
11 |
4.79 |
25 |
10.72 |
17 |
10.88 |
16 |
Segm+visib |
9.75 |
1.3 |
7 |
1.57 |
3 |
6.92 |
12 |
0.79 |
19 |
1.06 |
14 |
6.76 |
20 |
5 |
5 |
6.54 |
1 |
12.33 |
5 |
3.72 |
11 |
8.62 |
7 |
10.23 |
13 |
Segm+visib(subpixel) |
11 |
1.53 |
16 |
1.84 |
7 |
7.16 |
13 |
0.65 |
14 |
0.9 |
13 |
6.5 |
18 |
5.28 |
7 |
6.9 |
2 |
13.47 |
7 |
3.76 |
13 |
8.66 |
8 |
10.28 |
14 |
SymBP+occ(subpixel) |
11.08 |
1.33 |
9 |
2.31 |
17 |
6.28 |
6 |
0.17 |
7 |
0.36 |
5 |
2.33 |
9 |
6.11 |
10 |
10.74 |
8 |
17.11 |
12 |
4.47 |
19 |
10.43 |
16 |
10.61 |
15 |
C-SemiGlob(subpixel) |
11.42 |
1.79 |
20 |
2.49 |
19 |
8.32 |
21 |
0.26 |
11 |
0.58 |
11 |
3.54 |
13 |
5.39 |
8 |
11.45 |
9 |
14.15 |
8 |
2.91 |
4 |
8.56 |
6 |
8.49 |
7 |
C-SemiGlob |
12.42 |
2.61 |
33 |
3.29 |
24 |
9.89 |
28 |
0.25 |
10 |
0.57 |
9 |
3.24 |
11 |
5.14 |
6 |
11.79 |
10 |
12.96 |
6 |
2.77 |
2 |
8.35 |
5 |
8.2 |
5 |
RegionTreeDP |
13.83 |
1.39 |
12 |
1.64 |
4 |
6.85 |
10 |
0.22 |
9 |
0.57 |
10 |
1.93 |
6 |
7.42 |
18 |
11.91 |
11 |
16.75 |
10 |
6.31 |
31 |
11.9 |
26 |
11.84 |
19 |
RegionTreeDP(subpixel) |
14.42 |
1.7 |
18 |
2.07 |
13 |
7.52 |
16 |
0.13 |
5 |
0.49 |
8 |
1.58 |
2 |
7.27 |
16 |
11.97 |
12 |
17.81 |
18 |
5.54 |
29 |
11.2 |
19 |
11.1 |
17 |
AdaptWeight |
14.42 |
1.38 |
11 |
1.85 |
8 |
6.9 |
11 |
0.71 |
15 |
1.19 |
16 |
6.13 |
16 |
7.88 |
21 |
13.29 |
17 |
18.64 |
24 |
3.97 |
15 |
9.79 |
13 |
8.26 |
6 |
AdaptWeight(subpixel) |
15.08 |
1.8 |
21 |
2.41 |
18 |
8.04 |
18 |
0.62 |
13 |
1.07 |
15 |
5.79 |
15 |
7.45 |
19 |
13.02 |
15 |
18.37 |
20 |
3.72 |
12 |
9.64 |
11 |
8.11 |
4 |
SegTreeDP(subpixel) |
16.92 |
2.67 |
35 |
3.34 |
25 |
10.99 |
33 |
0.18 |
8 |
0.33 |
4 |
2.2 |
8 |
9.16 |
28 |
14.99 |
23 |
18.57 |
23 |
3.06 |
7 |
7.46 |
1 |
8.54 |
8 |
SegTreeDP |
17.33 |
2.21 |
28 |
2.76 |
20 |
10.32 |
30 |
0.46 |
12 |
0.6 |
12 |
2.44 |
10 |
9.58 |
29 |
15.17 |
25 |
18.43 |
21 |
3.23 |
9 |
7.86 |
2 |
8.83 |
10 |
SemiGlob |
20 |
3.26 |
37 |
3.96 |
31 |
12.75 |
38 |
1 |
21 |
1.57 |
19 |
11.27 |
32 |
6.02 |
9 |
12.19 |
13 |
16.32 |
9 |
3.06 |
8 |
9.75 |
12 |
8.9 |
11 |
SemiGlob(subpixel) |
20.17 |
2.65 |
34 |
3.42 |
28 |
11.39 |
34 |
0.93 |
20 |
1.48 |
18 |
11.18 |
31 |
6.37 |
11 |
12.78 |
14 |
17.47 |
15 |
3.47 |
10 |
10.22 |
15 |
10.04 |
12 |
RealtimeBP |
20.92 |
1.49 |
14 |
3.4 |
27 |
7.87 |
17 |
0.77 |
18 |
1.9 |
24 |
9 |
27 |
8.72 |
25 |
13.24 |
16 |
17.17 |
13 |
4.61 |
21 |
11.64 |
23 |
12.44 |
26 |
RealtimeBP(subpixel) |
21.83 |
1.75 |
19 |
3.67 |
29 |
8.08 |
19 |
0.72 |
17 |
1.86 |
22 |
9.06 |
29 |
8.84 |
27 |
13.33 |
18 |
18 |
19 |
4.41 |
18 |
11.44 |
20 |
12.25 |
25 |
Layered(subpixel) |
22.33 |
1.92 |
24 |
2.25 |
16 |
9.52 |
26 |
0.71 |
16 |
1.23 |
17 |
6.35 |
17 |
7.8 |
20 |
13.58 |
19 |
17.61 |
17 |
6.53 |
33 |
14.61 |
31 |
14.72 |
32 |
GC+occ(subpixel) |
23.25 |
1.32 |
8 |
2.18 |
14 |
6.38 |
7 |
1.41 |
29 |
1.97 |
26 |
6.97 |
22 |
10.46 |
33 |
16.91 |
31 |
19.51 |
26 |
5.28 |
27 |
12.41 |
27 |
13.01 |
29 |
Layered |
23.33 |
1.57 |
17 |
1.87 |
9 |
8.28 |
20 |
1.34 |
28 |
1.85 |
21 |
6.85 |
21 |
8.64 |
24 |
14.31 |
21 |
18.54 |
22 |
6.59 |
34 |
14.71 |
32 |
14.39 |
31 |
GC+occ |
23.58 |
1.19 |
4 |
2.01 |
12 |
6.24 |
5 |
1.64 |
33 |
2.19 |
28 |
6.75 |
19 |
11.18 |
35 |
17.4 |
34 |
19.77 |
29 |
5.36 |
28 |
12.45 |
28 |
12.97 |
28 |
MultiCamGC(subpixel) |
23.58 |
1.45 |
13 |
2.2 |
15 |
6.79 |
9 |
1.89 |
37 |
2.25 |
29 |
3.35 |
12 |
11.14 |
34 |
17.04 |
32 |
21.33 |
33 |
4.74 |
24 |
11.68 |
24 |
12.04 |
21 |
MultiCamGC |
24.67 |
1.27 |
6 |
1.99 |
11 |
6.48 |
8 |
2.79 |
40 |
3.13 |
38 |
3.6 |
14 |
11.99 |
36 |
17.63 |
36 |
21.99 |
34 |
4.89 |
26 |
11.83 |
25 |
12.09 |
22 |
TensorVoting(subpixel) |
26 |
1.9 |
23 |
2.98 |
22 |
8.32 |
22 |
1.04 |
22 |
1.69 |
20 |
10.92 |
30 |
10.1 |
32 |
17.46 |
35 |
25.31 |
40 |
4.39 |
17 |
11.52 |
22 |
12.56 |
27 |
TensorVoting |
28.25 |
3.79 |
38 |
4.79 |
37 |
8.86 |
24 |
1.23 |
23 |
1.88 |
23 |
11.53 |
34 |
9.76 |
30 |
17.05 |
33 |
24.03 |
36 |
4.38 |
16 |
11.44 |
21 |
12.23 |
24 |
RealTimeGPU(subpixel) |
29 |
2.45 |
31 |
4.73 |
35 |
12.02 |
36 |
1.48 |
32 |
2.54 |
33 |
18.2 |
40 |
6.65 |
13 |
14.13 |
20 |
17.17 |
14 |
6.12 |
30 |
13.24 |
29 |
16.23 |
35 |
GenModel |
29.58 |
2.57 |
32 |
4.74 |
36 |
13.03 |
39 |
1.72 |
35 |
3.08 |
37 |
16.93 |
38 |
6.86 |
14 |
14.99 |
24 |
19.21 |
25 |
4.64 |
23 |
14.89 |
34 |
11.38 |
18 |
RealTimeGPU |
29.92 |
2.05 |
27 |
4.22 |
33 |
10.64 |
32 |
1.92 |
38 |
2.98 |
35 |
20.27 |
43 |
7.23 |
15 |
14.38 |
22 |
17.57 |
16 |
6.41 |
32 |
13.7 |
30 |
16.46 |
36 |
CostRelax |
30 |
4.76 |
41 |
6.08 |
41 |
20.32 |
43 |
1.41 |
30 |
2.48 |
32 |
18.53 |
41 |
8.18 |
22 |
15.94 |
27 |
23.8 |
35 |
3.91 |
14 |
10.17 |
14 |
11.84 |
20 |
ReliabilityDP(subpixel) |
30.58 |
1.84 |
22 |
3.95 |
30 |
8.43 |
23 |
1.24 |
24 |
2.37 |
31 |
8.92 |
26 |
8.74 |
26 |
16.16 |
29 |
19.52 |
27 |
12.53 |
46 |
19.59 |
44 |
19.52 |
39 |
GenModel(subpixel) |
31.08 |
2.74 |
36 |
4.99 |
38 |
13.36 |
42 |
1.7 |
34 |
2.96 |
34 |
16.41 |
37 |
7.4 |
17 |
15.46 |
26 |
20.23 |
31 |
4.61 |
22 |
14.8 |
33 |
12.15 |
23 |
CostRelax(subpixel) |
31.75 |
5.29 |
44 |
6.68 |
42 |
21.18 |
44 |
1.32 |
26 |
2.36 |
30 |
17.29 |
39 |
8.28 |
23 |
16.1 |
28 |
24.03 |
37 |
4.55 |
20 |
10.81 |
18 |
13.19 |
30 |
ReliabilityDP |
32.33 |
1.36 |
10 |
3.39 |
26 |
7.25 |
15 |
2.35 |
39 |
3.48 |
40 |
12.22 |
36 |
9.82 |
31 |
16.87 |
30 |
19.54 |
28 |
12.88 |
47 |
19.94 |
46 |
19.71 |
40 |
TreeDP(subpixel) |
32.42 |
2.26 |
30 |
3.11 |
23 |
10.42 |
31 |
1.24 |
25 |
1.96 |
25 |
7.71 |
23 |
14.47 |
40 |
22.62 |
40 |
25.53 |
41 |
9.25 |
38 |
17.58 |
36 |
18.26 |
37 |
TreeDP |
34.17 |
1.99 |
26 |
2.84 |
21 |
9.96 |
29 |
1.41 |
31 |
2.1 |
27 |
7.74 |
24 |
15.92 |
44 |
23.9 |
45 |
27.09 |
44 |
10.05 |
41 |
18.35 |
40 |
18.92 |
38 |
GC(subpixel) |
34.17 |
2.23 |
29 |
4.41 |
34 |
9.8 |
27 |
1.33 |
27 |
3 |
36 |
9.02 |
28 |
15.15 |
41 |
23.85 |
44 |
24.24 |
38 |
7.35 |
35 |
17.85 |
38 |
15.06 |
33 |
GC |
35.17 |
1.94 |
25 |
4.12 |
32 |
9.39 |
25 |
1.79 |
36 |
3.44 |
39 |
8.75 |
25 |
16.47 |
45 |
25.03 |
47 |
24.93 |
39 |
7.7 |
36 |
18.15 |
39 |
15.27 |
34 |
DP(subpixel) |
40.42 |
4.57 |
40 |
5.56 |
40 |
13.34 |
41 |
8.19 |
47 |
9.17 |
47 |
19.5 |
42 |
13.61 |
37 |
21.24 |
37 |
20.22 |
30 |
10.08 |
42 |
18.67 |
41 |
20.47 |
41 |
DP |
41.08 |
4.12 |
39 |
5.04 |
39 |
11.95 |
35 |
10.1 |
50 |
11.03 |
50 |
21.03 |
45 |
14 |
38 |
21.58 |
38 |
20.56 |
32 |
10.54 |
43 |
19.1 |
42 |
21.1 |
42 |
SSD+MF(subpixel) |
42.5 |
6.09 |
45 |
8.01 |
45 |
25.26 |
46 |
3.5 |
41 |
4.9 |
42 |
11.46 |
33 |
15.44 |
42 |
23.84 |
43 |
31.71 |
45 |
9.83 |
40 |
19.16 |
43 |
24.79 |
45 |
SSD+MF |
43.75 |
5.23 |
43 |
7.07 |
43 |
24.14 |
45 |
3.74 |
43 |
5.16 |
43 |
11.89 |
35 |
16.53 |
46 |
24.85 |
46 |
32.91 |
46 |
10.6 |
44 |
19.82 |
45 |
26.26 |
46 |
STICA(subpixel) |
44.08 |
7.85 |
48 |
9.86 |
50 |
28.2 |
48 |
5.83 |
45 |
7.22 |
45 |
35.15 |
49 |
14.4 |
39 |
22.25 |
39 |
34.96 |
47 |
9.07 |
37 |
17.13 |
35 |
26.67 |
47 |
STICA |
45.33 |
7.7 |
47 |
9.63 |
48 |
27.83 |
47 |
8.19 |
48 |
9.58 |
48 |
40.3 |
50 |
15.83 |
43 |
23.22 |
41 |
37.74 |
48 |
9.81 |
39 |
17.8 |
37 |
28.7 |
48 |
SO(subpixel) |
45.42 |
6.19 |
46 |
8.34 |
46 |
13.19 |
40 |
7.61 |
46 |
9.15 |
46 |
20.6 |
44 |
19 |
49 |
27.32 |
49 |
25.54 |
42 |
12.44 |
45 |
22.34 |
49 |
21.71 |
43 |
SO |
46 |
5.08 |
42 |
7.22 |
44 |
12.18 |
37 |
9.44 |
49 |
10.94 |
49 |
21.92 |
46 |
19.95 |
50 |
28.17 |
50 |
26.26 |
43 |
13.01 |
48 |
22.85 |
50 |
22.25 |
44 |
Infection(subpixel) |
46.83 |
8.15 |
50 |
9.73 |
49 |
28.96 |
50 |
3.68 |
42 |
4.79 |
41 |
30.94 |
47 |
16.94 |
47 |
23.31 |
42 |
42.78 |
49 |
13.84 |
49 |
20.46 |
47 |
37.17 |
49 |
Infection |
47.92 |
7.95 |
49 |
9.54 |
47 |
28.88 |
49 |
4.41 |
44 |
5.53 |
44 |
31.67 |
48 |
17.7 |
48 |
25.09 |
48 |
44.41 |
50 |
14.34 |
50 |
21.33 |
48 |
38.04 |
50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|