Overall comparison of algorithms with
sub-pixel estimation using two views |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
table below is the current, off-line version of Figure 3 of our paper. It
compares the performance of different stereo algorithms on four test |
|
|
|
|
|
|
image
pairs, including all the algorithms reported to the Middlebury website. The
numbers represent the percentage of "bad" pixels, i.e., pixel
whose |
|
|
|
|
|
absolute
disparity error is greater than [0.50 | 0.75 | 1.00 | 1.50 |2.00], the error
threshold can be changed by clicking on the tabs at the end of |
|
|
|
|
|
the
table. For each image pair, we report percentages for (1) non-occluded
pixels, (2) all pixels and (3) pixels near depth. All the scoring and
ranking |
|
|
|
|
|
schemes are the
same as the Middlebury benchmark. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
All
algorithms are run with constant parameter settings across all four images.
The rank of each algorithm in each column is indicated with a small red
number. |
|
|
|
The
algorithms are listed roughly in decreasing order of overall performance (as
measured by average rank). |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
difference between this benchmark and the Middlebury one is that we provide
sub-pixel estimation for all the algorithms reported to the Middlebury
website |
|
|
|
by
using only a single view (the reference view) or two views. The lines with
the blue highlighting are the experimental results with our sub-pixel
estimations. |
|
|
|
This
sub-pixel benchmark proves that the sub-pixel estimation algorithm proposed
is very robust; it works for all the algorithms. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Algorithms |
Avg. |
|
|
Tsukuba |
|
|
|
|
|
Venus |
|
|
|
|
|
Teddy |
|
|
|
|
|
Cones |
|
|
|
|
Rank |
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
AdaptingBP |
2.83 |
1.08 |
5 |
1.33 |
2 |
5.79 |
5 |
0.1 |
2 |
0.2 |
2 |
1.42 |
3 |
1.91 |
1 |
3.63 |
2 |
6.05 |
1 |
2.18 |
3 |
7.06 |
3 |
6.52 |
5 |
AdaptingBP(subpixel) |
3.67 |
1.12 |
6 |
1.37 |
4 |
6.02 |
6 |
0.1 |
3 |
0.19 |
1 |
1.43 |
4 |
1.98 |
2 |
3.6 |
1 |
6.39 |
3 |
2.24 |
4 |
7.09 |
4 |
6.71 |
6 |
AdaptWeight |
15.08 |
1.38 |
19 |
1.85 |
11 |
6.9 |
13 |
0.55 |
19 |
0.98 |
19 |
4.76 |
16 |
5.98 |
21 |
9.56 |
16 |
14.24 |
21 |
2.79 |
12 |
8.1 |
11 |
6.08 |
3 |
AdaptWeight(subpixel) |
15.67 |
1.36 |
16 |
1.85 |
12 |
6.91 |
14 |
0.58 |
20 |
0.99 |
21 |
5.11 |
18 |
5.99 |
22 |
9.55 |
15 |
14.45 |
24 |
2.68 |
10 |
8.15 |
12 |
6.22 |
4 |
CostRelax |
32.42 |
4.7 |
42 |
5.99 |
42 |
20.01 |
44 |
1.1 |
31 |
2.08 |
31 |
14.34 |
37 |
6.83 |
28 |
14.2 |
33 |
20.21 |
35 |
3.63 |
19 |
9.51 |
19 |
11.04 |
28 |
CostRelax(subpixel) |
34.67 |
4.67 |
41 |
5.97 |
41 |
19.89 |
43 |
1.28 |
36 |
2.3 |
34 |
16.68 |
40 |
7 |
30 |
14.38 |
34 |
20.68 |
41 |
3.81 |
24 |
9.66 |
20 |
11.53 |
32 |
C-SemiGlob |
10.17 |
1.34 |
13 |
1.86 |
13 |
7.16 |
17 |
0.23 |
11 |
0.5 |
11 |
3.16 |
13 |
3.74 |
8 |
8.63 |
12 |
9.76 |
7 |
2.38 |
5 |
7.38 |
5 |
7.1 |
7 |
C-SemiGlob(subpixel) |
11.92 |
1.36 |
15 |
1.91 |
15 |
7.25 |
19 |
0.23 |
12 |
0.53 |
12 |
3.24 |
14 |
3.96 |
10 |
8.11 |
10 |
10.65 |
8 |
2.48 |
8 |
7.56 |
10 |
7.42 |
10 |
DoubleBP |
3.5 |
0.88 |
1 |
1.29 |
1 |
4.76 |
1 |
0.11 |
5 |
0.42 |
9 |
1.47 |
5 |
2.03 |
3 |
5.46 |
5 |
6.38 |
2 |
1.98 |
1 |
7.5 |
8 |
5.92 |
1 |
DoubleBP(subpixel) |
4.67 |
0.92 |
2 |
1.33 |
3 |
5 |
2 |
0.11 |
6 |
0.42 |
10 |
1.47 |
6 |
2.03 |
4 |
5.59 |
6 |
6.57 |
4 |
2.02 |
2 |
7.56 |
9 |
6.04 |
2 |
DP |
41 |
4.12 |
40 |
5.04 |
40 |
11.95 |
38 |
7.28 |
50 |
8.24 |
50 |
18.03 |
43 |
9.04 |
37 |
16.77 |
37 |
15.68 |
30 |
8.1 |
43 |
16.71 |
42 |
17.56 |
42 |
DP(subpixel) |
40.58 |
4.01 |
39 |
4.92 |
39 |
11.89 |
37 |
7.07 |
49 |
8.05 |
49 |
18.51 |
45 |
9.28 |
38 |
17.04 |
38 |
15.96 |
32 |
7.67 |
40 |
16.34 |
41 |
17.02 |
40 |
GC |
34.67 |
1.94 |
28 |
4.12 |
34 |
9.39 |
28 |
0.95 |
27 |
2.6 |
38 |
8.46 |
29 |
12.07 |
42 |
21.03 |
45 |
20.26 |
36 |
5.61 |
36 |
16.18 |
40 |
12.69 |
33 |
GC(subpixel) |
34.08 |
1.92 |
27 |
4.1 |
33 |
9.37 |
27 |
0.9 |
25 |
2.54 |
37 |
8.46 |
30 |
11.9 |
41 |
20.87 |
44 |
20.3 |
37 |
5.41 |
35 |
16 |
39 |
12.7 |
34 |
GC+occ |
24.58 |
1.19 |
8 |
2.01 |
19 |
6.24 |
8 |
1.22 |
33 |
1.75 |
28 |
6.39 |
23 |
8.08 |
32 |
13.06 |
30 |
14.97 |
28 |
4.46 |
29 |
11.32 |
27 |
11.42 |
30 |
GC+occ(subpixel) |
24.25 |
1.17 |
7 |
1.99 |
17 |
6.2 |
7 |
1.17 |
32 |
1.7 |
26 |
6.38 |
22 |
8 |
31 |
13.12 |
31 |
15.02 |
29 |
4.48 |
30 |
11.32 |
28 |
11.51 |
31 |
GenModel |
30.67 |
2.46 |
37 |
4.62 |
38 |
12.52 |
41 |
1.52 |
39 |
2.8 |
39 |
15.28 |
38 |
5.03 |
15 |
12.63 |
25 |
14.81 |
27 |
3.54 |
17 |
13.53 |
33 |
9.39 |
19 |
GenModel(subpixel) |
32.75 |
2.46 |
38 |
4.61 |
37 |
12.52 |
42 |
1.62 |
40 |
2.87 |
40 |
16.22 |
39 |
5.38 |
17 |
13.16 |
32 |
15.89 |
31 |
3.72 |
21 |
13.76 |
34 |
10.07 |
22 |
Infection |
47.42 |
7.36 |
49 |
8.88 |
47 |
26.88 |
48 |
3.31 |
42 |
4.41 |
42 |
30.15 |
48 |
14.76 |
50 |
21.16 |
46 |
38.3 |
50 |
11.91 |
49 |
18.24 |
48 |
33.14 |
50 |
Infection(subpixel) |
46.67 |
7.35 |
48 |
8.92 |
48 |
27.15 |
49 |
3.13 |
41 |
4.23 |
41 |
29.84 |
47 |
14.59 |
49 |
19.57 |
43 |
37.82 |
49 |
11.91 |
50 |
18 |
46 |
33.02 |
49 |
Layered |
21.5 |
1.57 |
25 |
1.87 |
14 |
8.28 |
25 |
0.42 |
14 |
0.77 |
16 |
4.52 |
15 |
5.73 |
20 |
10.81 |
22 |
13.36 |
19 |
4.58 |
31 |
12.42 |
31 |
10.83 |
26 |
Layered(subpixel) |
22.83 |
1.76 |
26 |
2.06 |
20 |
9.32 |
26 |
0.43 |
16 |
0.82 |
18 |
4.91 |
17 |
5.65 |
19 |
10.74 |
21 |
13.35 |
18 |
4.68 |
32 |
12.5 |
32 |
11.42 |
29 |
MultiCamGC |
20.92 |
1.27 |
10 |
1.99 |
18 |
6.48 |
10 |
0.42 |
15 |
0.71 |
15 |
2.42 |
11 |
8.75 |
36 |
12.68 |
26 |
16.79 |
33 |
3.94 |
27 |
10.79 |
26 |
10.37 |
24 |
MultiCamGC(subpixel) |
20.25 |
1.26 |
9 |
1.98 |
16 |
6.46 |
9 |
0.35 |
13 |
0.64 |
14 |
2.42 |
12 |
8.62 |
35 |
12.68 |
27 |
16.84 |
34 |
3.87 |
26 |
10.72 |
25 |
10.21 |
23 |
RealtimeBP |
21.67 |
1.49 |
21 |
3.4 |
31 |
7.87 |
23 |
0.64 |
22 |
1.65 |
25 |
7.79 |
26 |
6.59 |
25 |
9.85 |
18 |
12.8 |
13 |
3.35 |
15 |
9.9 |
21 |
9.58 |
20 |
RealtimeBP(subpixel) |
22.92 |
1.49 |
22 |
3.4 |
32 |
7.9 |
24 |
0.68 |
23 |
1.73 |
27 |
8.39 |
28 |
6.75 |
26 |
9.8 |
17 |
13.24 |
17 |
3.39 |
16 |
9.94 |
22 |
9.74 |
21 |
RealTimeGPU |
30.25 |
2.04 |
32 |
4.21 |
36 |
10.57 |
35 |
1.39 |
37 |
2.43 |
35 |
17.01 |
41 |
4.86 |
13 |
10.69 |
19 |
13.08 |
15 |
5.13 |
34 |
11.94 |
30 |
13.94 |
36 |
RealTimeGPU(subpixel) |
30.08 |
1.99 |
29 |
4.15 |
35 |
10.41 |
34 |
1.43 |
38 |
2.46 |
36 |
17.71 |
42 |
4.89 |
14 |
10.71 |
20 |
13.16 |
16 |
4.96 |
33 |
11.68 |
29 |
13.91 |
35 |
RegionTreeDP |
12.67 |
1.39 |
20 |
1.64 |
8 |
6.85 |
12 |
0.1 |
4 |
0.35 |
7 |
1.14 |
2 |
5.19 |
16 |
8.05 |
9 |
12.75 |
12 |
4.11 |
28 |
9.41 |
18 |
9.14 |
16 |
RegionTreeDP(subpixel) |
12.17 |
1.35 |
14 |
1.6 |
7 |
6.81 |
11 |
0.09 |
1 |
0.36 |
8 |
1.13 |
1 |
5.61 |
18 |
8.46 |
11 |
13.74 |
20 |
3.86 |
25 |
9.2 |
15 |
8.97 |
15 |
ReliabilityDP |
30.92 |
1.36 |
17 |
3.39 |
30 |
7.25 |
20 |
1.09 |
30 |
2.19 |
33 |
8.54 |
31 |
6.55 |
24 |
12.82 |
28 |
14.31 |
22 |
11.12 |
48 |
18.13 |
47 |
17.05 |
41 |
ReliabilityDP(subpixel) |
30.17 |
1.36 |
18 |
3.38 |
29 |
7.24 |
18 |
1.05 |
29 |
2.14 |
32 |
8.34 |
27 |
6.44 |
23 |
13 |
29 |
14.68 |
26 |
10.53 |
47 |
17.55 |
45 |
16.75 |
39 |
Segm+visib |
10.33 |
1.3 |
11 |
1.57 |
5 |
6.92 |
15 |
0.45 |
17 |
0.64 |
13 |
5.29 |
19 |
3.13 |
5 |
4.64 |
3 |
8.23 |
5 |
2.82 |
13 |
7.45 |
6 |
8.12 |
12 |
Segm+visib(subpixel) |
12 |
1.3 |
12 |
1.57 |
6 |
6.92 |
16 |
0.6 |
21 |
0.79 |
17 |
5.96 |
21 |
3.37 |
6 |
4.88 |
4 |
8.97 |
6 |
2.86 |
14 |
7.46 |
7 |
8.28 |
14 |
SegTreeDP |
16.75 |
2.21 |
34 |
2.76 |
24 |
10.32 |
32 |
0.2 |
10 |
0.31 |
4 |
1.93 |
7 |
6.92 |
29 |
11.05 |
23 |
14.43 |
23 |
2.41 |
6 |
6.15 |
1 |
7.13 |
8 |
SegTreeDP(subpixel) |
16.67 |
2.1 |
33 |
2.64 |
23 |
9.96 |
30 |
0.16 |
9 |
0.27 |
3 |
1.93 |
8 |
6.83 |
27 |
11.11 |
24 |
14.5 |
25 |
2.41 |
7 |
6.2 |
2 |
7.21 |
9 |
SemiGlob |
18.83 |
2.33 |
36 |
2.97 |
28 |
10.66 |
36 |
0.49 |
18 |
0.98 |
20 |
5.82 |
20 |
4.44 |
11 |
8.67 |
13 |
12.04 |
11 |
2.64 |
9 |
8.77 |
13 |
7.78 |
11 |
SemiGlob(subpixel) |
21 |
2.25 |
35 |
2.88 |
27 |
10.38 |
33 |
0.87 |
24 |
1.36 |
22 |
10.39 |
33 |
4.7 |
12 |
8.91 |
14 |
13.07 |
14 |
2.78 |
11 |
8.91 |
14 |
8.2 |
13 |
SO |
45.5 |
5.08 |
43 |
7.22 |
45 |
12.18 |
39 |
6.1 |
47 |
7.64 |
48 |
18.26 |
44 |
13.89 |
48 |
22.66 |
50 |
20.97 |
42 |
10.31 |
46 |
20.35 |
50 |
18.52 |
44 |
SO(subpixel) |
45.33 |
5.13 |
44 |
7.27 |
46 |
12.24 |
40 |
5.91 |
46 |
7.46 |
46 |
18.61 |
46 |
13.74 |
47 |
22.51 |
49 |
21.17 |
43 |
9.97 |
45 |
20.03 |
49 |
18.11 |
43 |
SSD+MF |
44.25 |
5.23 |
46 |
7.07 |
44 |
24.14 |
46 |
3.44 |
43 |
4.79 |
43 |
10.79 |
35 |
13.38 |
46 |
21.78 |
48 |
27.23 |
46 |
8.19 |
44 |
17.47 |
44 |
22.14 |
46 |
SSD+MF(subpixel) |
43.67 |
5.17 |
45 |
7.02 |
43 |
23.93 |
45 |
3.44 |
44 |
4.79 |
44 |
11.08 |
36 |
13.25 |
45 |
21.71 |
47 |
27.14 |
45 |
7.99 |
42 |
17.36 |
43 |
21.63 |
45 |
STICA |
46.08 |
7.7 |
50 |
9.63 |
50 |
27.83 |
50 |
6.22 |
48 |
7.6 |
47 |
34.02 |
50 |
12.41 |
43 |
19.4 |
41 |
30.81 |
48 |
7.78 |
41 |
15.59 |
37 |
23.27 |
48 |
STICA(subpixel) |
44.67 |
6.99 |
47 |
8.93 |
49 |
26.49 |
47 |
5.48 |
45 |
6.83 |
45 |
33.9 |
49 |
12.43 |
44 |
19.52 |
42 |
30.68 |
47 |
7.64 |
39 |
15.5 |
35 |
22.92 |
47 |
SymBP+occ |
9.83 |
0.97 |
4 |
1.75 |
10 |
5.09 |
4 |
0.15 |
7 |
0.32 |
5 |
2.13 |
9 |
3.72 |
7 |
6.47 |
7 |
11.34 |
9 |
3.79 |
23 |
9.24 |
16 |
9.35 |
17 |
SymBP+occ(subpixel) |
10.08 |
0.94 |
3 |
1.71 |
9 |
5.03 |
3 |
0.15 |
8 |
0.32 |
6 |
2.13 |
10 |
3.86 |
9 |
6.91 |
8 |
11.84 |
10 |
3.7 |
20 |
9.25 |
17 |
9.38 |
18 |
TensorVoting |
26.58 |
1.5 |
23 |
2.49 |
22 |
7.42 |
21 |
0.94 |
26 |
1.58 |
23 |
9.64 |
32 |
8.29 |
33 |
15.56 |
35 |
20.43 |
38 |
3.6 |
18 |
10.44 |
23 |
10.53 |
25 |
TensorVoting(subpixel) |
28.33 |
1.5 |
24 |
2.48 |
21 |
7.42 |
22 |
1.01 |
28 |
1.64 |
24 |
10.61 |
34 |
8.42 |
34 |
15.69 |
36 |
21.21 |
44 |
3.76 |
22 |
10.59 |
24 |
11.03 |
27 |
TreeDP |
33.92 |
1.99 |
30 |
2.84 |
26 |
9.96 |
31 |
1.23 |
34 |
1.92 |
29 |
7.55 |
24 |
10.77 |
40 |
19.09 |
40 |
20.51 |
39 |
7.62 |
38 |
15.99 |
38 |
15.62 |
38 |
TreeDP(subpixel) |
33.58 |
1.99 |
31 |
2.83 |
25 |
9.95 |
29 |
1.23 |
35 |
1.92 |
30 |
7.55 |
25 |
10.72 |
39 |
19.07 |
39 |
20.52 |
40 |
7.12 |
37 |
15.53 |
36 |
14.86 |
37 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|