Overall comparison of algorithms with
sub-pixel estimation using two views |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
table below is the current, off-line version of Figure 3 of our paper. It
compares the performance of different stereo algorithms on four test |
|
|
|
|
|
|
image
pairs, including all the algorithms reported to the Middlebury website. The
numbers represent the percentage of "bad" pixels, i.e., pixel
whose |
|
|
|
|
|
absolute
disparity error is greater than [0.50 | 0.75 | 1.00 | 1.50 |2.00], the error
threshold can be changed by clicking on the tabs at the end of |
|
|
|
|
|
the
table. For each image pair, we report percentages for (1) non-occluded
pixels, (2) all pixels and (3) pixels near depth. All the scoring and
ranking |
|
|
|
|
|
schemes are the
same as the Middlebury benchmark. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
All
algorithms are run with constant parameter settings across all four images.
The rank of each algorithm in each column is indicated with a small red
number. |
|
|
|
The
algorithms are listed roughly in decreasing order of overall performance (as
measured by average rank). |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
difference between this benchmark and the Middlebury one is that we provide
sub-pixel estimation for all the algorithms reported to the Middlebury
website |
|
|
|
by
using only a single view (the reference view) or two views. The lines with
the blue highlighting are the experimental results with our sub-pixel
estimations. |
|
|
|
This
sub-pixel benchmark proves that the sub-pixel estimation algorithm proposed
is very robust; it works for all the algorithms. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Algorithms |
Avg. |
|
|
Tsukuba |
|
|
|
|
|
Venus |
|
|
|
|
|
Teddy |
|
|
|
|
|
Cones |
|
|
|
|
Rank |
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
AdaptingBP(subpixel) |
3.75 |
1.03 |
8 |
1.27 |
2 |
5.52 |
9 |
0.1 |
4 |
0.18 |
2 |
1.42 |
4 |
1.19 |
2 |
1.89 |
1 |
4.21 |
2 |
1.88 |
3 |
6.35 |
3 |
5.67 |
5 |
AdaptingBP |
4 |
1.05 |
9 |
1.31 |
4 |
5.66 |
10 |
0.1 |
3 |
0.18 |
1 |
1.42 |
3 |
1.05 |
1 |
2 |
2 |
3.74 |
1 |
1.89 |
4 |
6.42 |
4 |
5.69 |
6 |
AdaptWeight |
14.08 |
1.08 |
10 |
1.41 |
7 |
5.4 |
8 |
0.51 |
21 |
0.82 |
21 |
4.17 |
19 |
5.06 |
21 |
7.5 |
15 |
11.82 |
24 |
2.26 |
9 |
7.16 |
12 |
5.2 |
2 |
AdaptWeight(subpixel) |
16.83 |
1.24 |
18 |
1.69 |
15 |
6.27 |
15 |
0.51 |
22 |
0.82 |
22 |
4.21 |
21 |
5.18 |
23 |
7.56 |
16 |
12.17 |
27 |
2.26 |
10 |
7.1 |
10 |
5.33 |
3 |
CostRelax |
33.75 |
3.91 |
41 |
5.2 |
41 |
16.85 |
43 |
1.05 |
31 |
1.95 |
33 |
13.61 |
41 |
6.05 |
29 |
13.23 |
33 |
17.94 |
40 |
3.44 |
25 |
9.04 |
19 |
10.48 |
29 |
CostRelax(subpixel) |
35.33 |
4.07 |
42 |
5.37 |
42 |
17.72 |
44 |
1.07 |
34 |
1.97 |
34 |
13.81 |
42 |
6.09 |
30 |
13.23 |
34 |
18.16 |
42 |
3.55 |
27 |
9.15 |
21 |
10.8 |
32 |
C-SemiGlob |
11.08 |
1.2 |
14 |
1.7 |
16 |
6.42 |
18 |
0.23 |
13 |
0.38 |
11 |
3.15 |
13 |
3.04 |
9 |
6.37 |
12 |
7.97 |
7 |
2.13 |
7 |
6.64 |
5 |
6.37 |
8 |
C-SemiGlob(subpixel) |
13.67 |
1.32 |
23 |
1.84 |
20 |
7.07 |
24 |
0.23 |
14 |
0.43 |
14 |
3.18 |
14 |
3.15 |
10 |
6.11 |
10 |
8.5 |
9 |
2.23 |
8 |
6.75 |
8 |
6.68 |
10 |
DoubleBP |
3.83 |
0.83 |
2 |
1.24 |
1 |
4.49 |
2 |
0.1 |
5 |
0.35 |
9 |
1.46 |
5 |
1.41 |
3 |
4.13 |
5 |
4.73 |
3 |
1.71 |
1 |
7.02 |
9 |
5.16 |
1 |
DoubleBP(subpixel) |
5.17 |
0.86 |
3 |
1.27 |
3 |
4.66 |
3 |
0.1 |
6 |
0.35 |
10 |
1.46 |
6 |
1.43 |
4 |
4.13 |
6 |
4.94 |
4 |
1.78 |
2 |
7.14 |
11 |
5.37 |
4 |
DP |
40.42 |
3.43 |
39 |
4.23 |
37 |
9.85 |
36 |
6.5 |
50 |
7.43 |
50 |
17.36 |
45 |
7.11 |
34 |
14.92 |
38 |
13.43 |
32 |
6.52 |
42 |
15.07 |
41 |
15.09 |
41 |
DP(subpixel) |
40.67 |
3.72 |
40 |
4.64 |
40 |
11.3 |
39 |
6.29 |
49 |
7.24 |
49 |
17.55 |
46 |
7.06 |
33 |
14.88 |
37 |
13.28 |
31 |
6.45 |
40 |
15.07 |
42 |
15.26 |
42 |
GC |
35.25 |
1.67 |
30 |
3.75 |
34 |
8.2 |
29 |
0.83 |
27 |
2.37 |
38 |
8.12 |
31 |
9.72 |
42 |
18.84 |
46 |
17.23 |
37 |
4.51 |
36 |
14.98 |
40 |
11.21 |
33 |
GC(subpixel) |
36.25 |
1.89 |
34 |
4.05 |
36 |
9.25 |
34 |
0.83 |
28 |
2.37 |
39 |
8.12 |
32 |
9.56 |
41 |
18.7 |
45 |
17.33 |
38 |
4.44 |
35 |
14.96 |
39 |
11.29 |
34 |
GC+occ |
23.67 |
0.88 |
4 |
1.66 |
12 |
4.68 |
4 |
1.1 |
35 |
1.57 |
26 |
6.02 |
23 |
7.03 |
32 |
11.37 |
31 |
12.41 |
29 |
4.16 |
31 |
10.73 |
27 |
10.6 |
30 |
GC+occ(subpixel) |
25 |
1 |
6 |
1.82 |
19 |
5.33 |
7 |
1.1 |
36 |
1.58 |
28 |
6.02 |
24 |
6.97 |
31 |
11.31 |
30 |
12.36 |
28 |
4.2 |
32 |
10.77 |
28 |
10.72 |
31 |
GenModel |
30.08 |
2.35 |
37 |
4.5 |
38 |
12.17 |
41 |
1.11 |
37 |
2.2 |
37 |
10.42 |
35 |
3.88 |
13 |
11.05 |
29 |
11.91 |
25 |
3.07 |
19 |
12.76 |
33 |
8.1 |
17 |
GenModel(subpixel) |
33.08 |
2.35 |
38 |
4.5 |
39 |
12.2 |
42 |
1.35 |
40 |
2.5 |
40 |
12.87 |
40 |
4.28 |
17 |
11.55 |
32 |
12.73 |
30 |
3.27 |
23 |
13.11 |
34 |
8.78 |
22 |
Infection |
47 |
6.34 |
48 |
7.81 |
48 |
22.8 |
48 |
2.7 |
42 |
3.66 |
42 |
26 |
47 |
12.8 |
49 |
18.32 |
44 |
33.48 |
50 |
10.72 |
49 |
16.55 |
48 |
30.13 |
49 |
Infection(subpixel) |
47.08 |
6.82 |
50 |
8.32 |
49 |
25.32 |
50 |
2.65 |
41 |
3.62 |
41 |
26.68 |
48 |
12.84 |
50 |
17.16 |
41 |
33.44 |
49 |
10.82 |
50 |
16.46 |
46 |
30.31 |
50 |
Layered |
20.5 |
1.28 |
21 |
1.45 |
10 |
6.84 |
22 |
0.26 |
15 |
0.48 |
17 |
3.58 |
15 |
4.51 |
20 |
9.33 |
24 |
10.4 |
16 |
3.82 |
29 |
11.26 |
31 |
9.59 |
26 |
Layered(subpixel) |
22.5 |
1.51 |
28 |
1.81 |
18 |
8 |
28 |
0.26 |
16 |
0.49 |
18 |
3.61 |
16 |
4.44 |
19 |
9.3 |
23 |
10.37 |
14 |
3.9 |
30 |
11.39 |
32 |
9.81 |
28 |
MultiCamGC |
19 |
1.01 |
7 |
1.66 |
13 |
5.21 |
6 |
0.16 |
11 |
0.38 |
12 |
1.7 |
7 |
7.46 |
38 |
10.67 |
26 |
13.82 |
33 |
3.55 |
26 |
10.21 |
25 |
9.29 |
24 |
MultiCamGC(subpixel) |
20.58 |
1.1 |
11 |
1.79 |
17 |
5.68 |
11 |
0.17 |
12 |
0.39 |
13 |
1.76 |
8 |
7.43 |
37 |
10.64 |
25 |
13.87 |
34 |
3.56 |
28 |
10.24 |
26 |
9.31 |
25 |
RealtimeBP |
21.17 |
1.25 |
19 |
3.04 |
29 |
6.66 |
20 |
0.63 |
23 |
1.53 |
25 |
7.68 |
29 |
5.68 |
27 |
8.27 |
17 |
10.18 |
12 |
2.9 |
15 |
9.11 |
20 |
8.27 |
18 |
RealtimeBP(subpixel) |
24.42 |
1.43 |
27 |
3.32 |
32 |
7.64 |
27 |
0.64 |
24 |
1.58 |
27 |
7.82 |
30 |
5.87 |
28 |
8.51 |
19 |
10.81 |
19 |
2.98 |
17 |
9.31 |
22 |
8.53 |
21 |
RealTimeGPU |
27.5 |
1.34 |
24 |
3.27 |
31 |
7.17 |
26 |
1.02 |
30 |
1.9 |
32 |
12.43 |
39 |
3.9 |
14 |
8.65 |
21 |
10.38 |
15 |
4.37 |
33 |
10.82 |
29 |
12.29 |
36 |
RealTimeGPU(subpixel) |
29.25 |
1.71 |
31 |
3.84 |
35 |
8.97 |
32 |
0.97 |
29 |
1.84 |
29 |
11.61 |
38 |
3.99 |
15 |
8.75 |
22 |
10.69 |
18 |
4.43 |
34 |
10.94 |
30 |
12.51 |
38 |
RegionTreeDP |
11 |
1.2 |
15 |
1.43 |
9 |
6.01 |
13 |
0.09 |
1 |
0.3 |
7 |
1.11 |
1 |
4.22 |
16 |
6.09 |
9 |
10.69 |
17 |
2.98 |
16 |
8.1 |
13 |
7.31 |
15 |
RegionTreeDP(subpixel) |
13.08 |
1.26 |
20 |
1.51 |
11 |
6.31 |
17 |
0.09 |
2 |
0.31 |
8 |
1.13 |
2 |
4.38 |
18 |
6.12 |
11 |
11.01 |
20 |
2.99 |
18 |
8.15 |
14 |
7.5 |
16 |
ReliabilityDP |
30.83 |
1.21 |
17 |
3.18 |
30 |
6.49 |
19 |
1.05 |
32 |
2.03 |
35 |
8.27 |
33 |
5.17 |
22 |
10.68 |
27 |
11.81 |
23 |
9.05 |
48 |
16.05 |
44 |
14.31 |
40 |
ReliabilityDP(subpixel) |
32.33 |
1.31 |
22 |
3.33 |
33 |
7.04 |
23 |
1.05 |
33 |
2.03 |
36 |
8.27 |
34 |
5.2 |
24 |
10.74 |
28 |
11.93 |
26 |
8.82 |
47 |
15.84 |
43 |
14.17 |
39 |
Segm+visib |
10.42 |
1.14 |
12 |
1.37 |
6 |
6.09 |
14 |
0.35 |
18 |
0.48 |
16 |
4.16 |
18 |
2.39 |
5 |
3.82 |
3 |
6.35 |
5 |
2.36 |
11 |
6.7 |
6 |
6.89 |
11 |
Segm+visib(subpixel) |
11.25 |
1.18 |
13 |
1.43 |
8 |
6.3 |
16 |
0.34 |
17 |
0.46 |
15 |
3.99 |
17 |
2.52 |
6 |
3.94 |
4 |
6.63 |
6 |
2.43 |
13 |
6.73 |
7 |
7.04 |
13 |
SegTreeDP |
14.42 |
1.38 |
25 |
1.88 |
21 |
7.11 |
25 |
0.15 |
9 |
0.23 |
3 |
1.86 |
10 |
5.4 |
26 |
8.64 |
20 |
11.54 |
21 |
2.03 |
5 |
5.33 |
1 |
6.13 |
7 |
SegTreeDP(subpixel) |
15.5 |
1.61 |
29 |
2.16 |
22 |
8.22 |
30 |
0.15 |
10 |
0.23 |
4 |
1.79 |
9 |
5.31 |
25 |
8.49 |
18 |
11.61 |
22 |
2.11 |
6 |
5.45 |
2 |
6.37 |
9 |
SemiGlob |
18.75 |
2.01 |
35 |
2.52 |
25 |
9.15 |
33 |
0.36 |
19 |
0.68 |
19 |
4.19 |
20 |
3.46 |
11 |
6.55 |
13 |
9.26 |
11 |
2.38 |
12 |
8.15 |
15 |
7 |
12 |
SemiGlob(subpixel) |
20.5 |
2.21 |
36 |
2.83 |
28 |
10.18 |
37 |
0.38 |
20 |
0.72 |
20 |
4.43 |
22 |
3.75 |
12 |
6.71 |
14 |
10.23 |
13 |
2.43 |
14 |
8.15 |
16 |
7.15 |
14 |
SO |
45.17 |
4.23 |
43 |
6.21 |
43 |
10.71 |
38 |
5.14 |
48 |
6.58 |
48 |
16.97 |
44 |
11.27 |
46 |
20.22 |
48 |
18.56 |
44 |
8.76 |
46 |
18.78 |
50 |
16.08 |
44 |
SO(subpixel) |
44.92 |
4.53 |
44 |
6.63 |
46 |
11.65 |
40 |
4.9 |
47 |
6.36 |
47 |
16.81 |
43 |
11 |
45 |
19.98 |
47 |
18.49 |
43 |
8.66 |
45 |
18.73 |
49 |
16.06 |
43 |
SSD+MF |
44.92 |
4.7 |
45 |
6.48 |
44 |
21.93 |
46 |
3.41 |
43 |
4.63 |
43 |
10.73 |
36 |
12.03 |
48 |
20.42 |
50 |
24.24 |
45 |
7.33 |
44 |
16.5 |
47 |
20.29 |
48 |
SSD+MF(subpixel) |
45.17 |
4.83 |
46 |
6.66 |
47 |
22.63 |
47 |
3.42 |
44 |
4.65 |
44 |
10.91 |
37 |
11.87 |
47 |
20.34 |
49 |
24.28 |
46 |
7.09 |
43 |
16.38 |
45 |
19.59 |
47 |
STICA |
44.33 |
4.95 |
47 |
6.6 |
45 |
18.35 |
45 |
4.69 |
46 |
5.89 |
46 |
27.25 |
49 |
10.54 |
44 |
17.23 |
43 |
25.76 |
48 |
6.36 |
39 |
13.84 |
35 |
19.14 |
45 |
STICA(subpixel) |
45.17 |
6.49 |
49 |
8.36 |
50 |
24.38 |
49 |
4.61 |
45 |
5.81 |
45 |
27.49 |
50 |
10.48 |
43 |
17.16 |
40 |
25.7 |
47 |
6.45 |
41 |
14.03 |
37 |
19.4 |
46 |
SymBP+occ |
9.17 |
0.7 |
1 |
1.37 |
5 |
3.77 |
1 |
0.14 |
7 |
0.29 |
5 |
1.99 |
11 |
2.65 |
7 |
4.48 |
7 |
8.42 |
8 |
3.26 |
22 |
8.39 |
17 |
8.39 |
19 |
SymBP+occ(subpixel) |
11.25 |
0.9 |
5 |
1.67 |
14 |
4.84 |
5 |
0.14 |
8 |
0.29 |
6 |
1.99 |
12 |
2.74 |
8 |
4.58 |
8 |
8.6 |
10 |
3.24 |
21 |
8.44 |
18 |
8.53 |
20 |
TensorVoting |
24.92 |
1.2 |
16 |
2.18 |
23 |
5.85 |
12 |
0.68 |
25 |
1.18 |
23 |
6.69 |
25 |
7.21 |
35 |
14.4 |
35 |
17.5 |
39 |
3.12 |
20 |
9.78 |
23 |
9.2 |
23 |
TensorVoting(subpixel) |
28 |
1.38 |
26 |
2.36 |
24 |
6.81 |
21 |
0.73 |
26 |
1.24 |
24 |
7.34 |
27 |
7.35 |
36 |
14.54 |
36 |
18.06 |
41 |
3.29 |
24 |
9.97 |
24 |
9.68 |
27 |
TreeDP |
34.67 |
1.73 |
32 |
2.55 |
26 |
8.82 |
31 |
1.21 |
38 |
1.89 |
30 |
7.35 |
28 |
8.8 |
40 |
17.2 |
42 |
17.19 |
36 |
5.57 |
38 |
14.1 |
38 |
12.46 |
37 |
TreeDP(subpixel) |
34.33 |
1.88 |
33 |
2.69 |
27 |
9.58 |
35 |
1.21 |
39 |
1.89 |
31 |
7.31 |
26 |
8.64 |
39 |
17.08 |
39 |
17.09 |
35 |
5.31 |
37 |
13.86 |
36 |
12.15 |
35 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|