Overall comparison of algorithms with
sub-pixel estimation using two views |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
table below is the current, off-line version of Figure 3 of our paper. It
compares the performance of different stereo algorithms on four test |
|
|
|
|
|
|
image
pairs, including all the algorithms reported to the Middlebury website. The
numbers represent the percentage of "bad" pixels, i.e., pixel
whose |
|
|
|
|
|
absolute
disparity error is greater than [0.50 | 0.75 | 1.00 | 1.50 |2.00], the error
threshold can be changed by clicking on the tabs at the end of |
|
|
|
|
|
the
table. For each image pair, we report percentages for (1) non-occluded
pixels, (2) all pixels and (3) pixels near depth. All the scoring and
ranking |
|
|
|
|
|
schemes are the
same as the Middlebury benchmark. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
All
algorithms are run with constant parameter settings across all four images.
The rank of each algorithm in each column is indicated with a small red
number. |
|
|
|
The
algorithms are listed roughly in decreasing order of overall performance (as
measured by average rank). |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
difference between this benchmark and the Middlebury one is that we provide
sub-pixel estimation for all the algorithms reported to the Middlebury
website |
|
|
|
by
using only a single view (the reference view) or two views. The lines with
the blue highlighting are the experimental results with our sub-pixel
estimations. |
|
|
|
This
sub-pixel benchmark proves that the sub-pixel estimation algorithm proposed
is very robust; it works for all the algorithms. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Algorithms |
Avg. |
|
|
Tsukuba |
|
|
|
|
|
Venus |
|
|
|
|
|
Teddy |
|
|
|
|
|
Cones |
|
|
|
|
Rank |
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
DoubleBP(subpixel) |
4.5 |
4.91 |
6 |
5.62 |
6 |
10.99 |
1 |
0.22 |
2 |
0.57 |
4 |
3.03 |
2 |
5.46 |
1 |
11.15 |
5 |
13.62 |
2 |
4.69 |
9 |
10.63 |
9 |
11.01 |
7 |
AdaptingBP(subpixel) |
6.25 |
6.6 |
21 |
7.21 |
15 |
13.53 |
12 |
0.21 |
1 |
0.39 |
1 |
2.92 |
1 |
6.22 |
3 |
10.35 |
3 |
15.71 |
3 |
3.84 |
5 |
9.59 |
4 |
10.85 |
6 |
C-SemiGlob |
7 |
6.21 |
17 |
6.9 |
11 |
12.93 |
8 |
0.46 |
7 |
0.85 |
8 |
4.42 |
11 |
6.39 |
4 |
13.8 |
9 |
15.99 |
4 |
3.29 |
1 |
9.21 |
2 |
9.38 |
2 |
C-SemiGlob(subpixel) |
7.5 |
5.53 |
13 |
6.55 |
9 |
13.72 |
13 |
0.35 |
6 |
0.72 |
6 |
4.82 |
12 |
6.94 |
5 |
13.75 |
7 |
17.22 |
7 |
3.7 |
4 |
9.65 |
5 |
10.18 |
3 |
Segm+visib(subpixel) |
8.42 |
4.08 |
2 |
4.51 |
1 |
11.67 |
3 |
0.92 |
12 |
1.36 |
12 |
8.84 |
19 |
7.26 |
8 |
8.99 |
2 |
17.33 |
8 |
4.97 |
13 |
9.96 |
7 |
12.59 |
14 |
SymBP+occ(subpixel) |
9.5 |
5.4 |
12 |
6.48 |
8 |
11.46 |
2 |
0.26 |
3 |
0.51 |
2 |
3.65 |
6 |
8.36 |
11 |
13.76 |
8 |
20.8 |
13 |
5.71 |
18 |
11.88 |
16 |
12.66 |
15 |
RegionTreeDP(subpixel) |
11 |
4.59 |
5 |
5.07 |
4 |
11.73 |
4 |
0.27 |
4 |
0.74 |
7 |
3.11 |
3 |
8.95 |
13 |
14.51 |
12 |
20.52 |
10 |
7.45 |
29 |
13.3 |
22 |
13.8 |
19 |
AdaptingBP |
11.67 |
14.03 |
34 |
14.4 |
34 |
13.49 |
11 |
1.16 |
17 |
1.36 |
11 |
4.34 |
10 |
7.16 |
7 |
10.63 |
4 |
16.61 |
6 |
3.46 |
2 |
9.34 |
3 |
9.19 |
1 |
AdaptWeight(subpixel) |
12.08 |
5.36 |
11 |
6.17 |
7 |
13.17 |
9 |
0.71 |
9 |
1.17 |
9 |
6.98 |
15 |
9.16 |
15 |
15.7 |
15 |
21.66 |
16 |
5.46 |
17 |
11.53 |
14 |
11.23 |
8 |
DoubleBP |
13.42 |
18.66 |
38 |
19.11 |
37 |
15.76 |
21 |
0.82 |
10 |
1.18 |
10 |
3.65 |
5 |
5.82 |
2 |
11.17 |
6 |
13.42 |
1 |
4.76 |
10 |
10.78 |
11 |
11.52 |
10 |
SemiGlob(subpixel) |
15 |
5.68 |
14 |
6.65 |
10 |
15.99 |
25 |
1 |
13 |
1.7 |
15 |
12.03 |
31 |
8.12 |
10 |
15.28 |
14 |
21.34 |
15 |
4.42 |
8 |
11.49 |
13 |
12.13 |
12 |
SegTreeDP(subpixel) |
15.25 |
10.79 |
29 |
11.52 |
28 |
18.28 |
31 |
0.31 |
5 |
0.54 |
3 |
3.84 |
7 |
11.08 |
25 |
17.69 |
25 |
22 |
19 |
4.13 |
6 |
9.08 |
1 |
10.35 |
4 |
Segm+visib |
17.67 |
12.69 |
32 |
12.93 |
31 |
15.81 |
22 |
2.64 |
32 |
3.08 |
29 |
10.25 |
23 |
7.04 |
6 |
8.71 |
1 |
16.25 |
5 |
4.86 |
12 |
9.86 |
6 |
12.21 |
13 |
SemiGlob |
17.67 |
7.34 |
23 |
8.37 |
23 |
18.39 |
33 |
1.96 |
29 |
2.6 |
24 |
12.83 |
33 |
7.47 |
9 |
14.43 |
11 |
19.58 |
9 |
3.61 |
3 |
10.67 |
10 |
10.44 |
5 |
Layered(subpixel) |
18.17 |
4.25 |
4 |
4.75 |
2 |
14.14 |
15 |
1.12 |
16 |
1.83 |
17 |
8.98 |
20 |
9.75 |
20 |
16.05 |
16 |
21.06 |
14 |
8.14 |
31 |
16.28 |
32 |
17.57 |
31 |
RealtimeBP(subpixel) |
18.5 |
5.15 |
9 |
7.05 |
13 |
13.45 |
10 |
0.84 |
11 |
2.06 |
19 |
10.61 |
26 |
10.95 |
24 |
16.49 |
19 |
21.7 |
17 |
6.15 |
23 |
13.45 |
24 |
15.16 |
27 |
SymBP+occ |
19.75 |
20.74 |
43 |
21.57 |
43 |
19.51 |
36 |
0.48 |
8 |
0.7 |
5 |
3.44 |
4 |
9.03 |
14 |
13.98 |
10 |
22.26 |
22 |
5.85 |
19 |
12.06 |
17 |
12.82 |
16 |
GC+occ(subpixel) |
19.75 |
3.96 |
1 |
4.95 |
3 |
12.2 |
5 |
1.72 |
27 |
2.34 |
21 |
8.25 |
16 |
12.8 |
32 |
19.82 |
32 |
23.07 |
23 |
6.62 |
26 |
13.82 |
25 |
15.09 |
26 |
MultiCamGC(subpixel) |
21.67 |
4.19 |
3 |
5.14 |
5 |
12.77 |
7 |
3.58 |
36 |
4.08 |
35 |
5.57 |
13 |
13.56 |
33 |
20.2 |
33 |
24.97 |
31 |
6 |
22 |
12.97 |
21 |
14.01 |
21 |
TensorVoting(subpixel) |
22 |
6.26 |
18 |
7.4 |
18 |
13.89 |
14 |
1.07 |
14 |
1.74 |
16 |
11.05 |
29 |
11.4 |
29 |
18.76 |
31 |
27.83 |
39 |
5.16 |
14 |
12.47 |
20 |
14.47 |
22 |
AdaptWeight |
23.25 |
18.1 |
37 |
18.8 |
36 |
18.59 |
34 |
1.32 |
19 |
1.9 |
18 |
8.27 |
17 |
10.11 |
22 |
16.29 |
17 |
23.45 |
26 |
6.17 |
24 |
12.15 |
18 |
12.11 |
11 |
SegTreeDP |
23.58 |
25.42 |
49 |
25.96 |
47 |
24.6 |
41 |
1.29 |
18 |
1.53 |
13 |
4.21 |
9 |
12.34 |
31 |
18.45 |
29 |
21.96 |
18 |
4.78 |
11 |
10.09 |
8 |
11.45 |
9 |
GenModel |
24 |
5.28 |
10 |
7.47 |
19 |
15.84 |
23 |
2.04 |
31 |
3.45 |
31 |
17.66 |
38 |
9.19 |
17 |
17.42 |
23 |
23.12 |
24 |
5.97 |
21 |
16.28 |
33 |
13.4 |
18 |
RegionTreeDP |
24.5 |
20.95 |
44 |
21.12 |
42 |
18.33 |
32 |
1.09 |
15 |
1.57 |
14 |
4.21 |
8 |
10 |
21 |
15.12 |
13 |
20.75 |
12 |
9.8 |
35 |
15.45 |
29 |
16.13 |
29 |
RealTimeGPU(subpixel) |
24.58 |
8.06 |
25 |
10.2 |
25 |
15.74 |
20 |
1.61 |
24 |
2.7 |
25 |
18.49 |
40 |
8.42 |
12 |
16.53 |
20 |
20.67 |
11 |
7.84 |
30 |
15.38 |
28 |
19.02 |
35 |
GenModel(subpixel) |
25.08 |
6.08 |
15 |
8.26 |
21 |
17.09 |
28 |
1.91 |
28 |
3.21 |
30 |
16.56 |
37 |
9.41 |
18 |
17.54 |
24 |
23.58 |
27 |
5.94 |
20 |
16.14 |
30 |
14.7 |
23 |
ReliabilityDP(subpixel) |
25.67 |
4.96 |
7 |
7.01 |
12 |
12.24 |
6 |
1.56 |
22 |
2.8 |
27 |
10.76 |
27 |
11.26 |
28 |
18.75 |
30 |
23.39 |
25 |
13.93 |
45 |
21.03 |
41 |
22.1 |
38 |
CostRelax(subpixel) |
25.83 |
9.54 |
27 |
10.94 |
27 |
24.95 |
43 |
1.36 |
20 |
2.42 |
23 |
17.69 |
39 |
9.43 |
19 |
17.36 |
22 |
26.46 |
35 |
5.24 |
16 |
11.78 |
15 |
14.8 |
24 |
TensorVoting |
26 |
16.84 |
35 |
17.68 |
35 |
15.23 |
17 |
1.66 |
26 |
2.37 |
22 |
12.35 |
32 |
11.09 |
26 |
18.4 |
28 |
26.81 |
37 |
5.18 |
15 |
12.36 |
19 |
13.96 |
20 |
GC+occ |
26.08 |
6.1 |
16 |
7.11 |
14 |
14.55 |
16 |
3.2 |
34 |
3.8 |
34 |
8.4 |
18 |
14.67 |
34 |
21.27 |
34 |
24.41 |
30 |
6.81 |
28 |
14 |
27 |
15.8 |
28 |
CostRelax |
26.92 |
19.48 |
40 |
20.65 |
40 |
29.54 |
44 |
1.65 |
25 |
2.73 |
26 |
19.45 |
41 |
9.18 |
16 |
17.14 |
21 |
26.29 |
34 |
4.37 |
7 |
10.89 |
12 |
12.92 |
17 |
MultiCamGC |
28.42 |
6.56 |
20 |
7.55 |
20 |
15.68 |
19 |
6.33 |
44 |
6.75 |
43 |
6.67 |
14 |
15.51 |
36 |
21.69 |
36 |
26.81 |
36 |
6.35 |
25 |
13.31 |
23 |
14.84 |
25 |
RealtimeBP |
29.17 |
19.94 |
42 |
21.62 |
44 |
22.16 |
37 |
1.61 |
23 |
2.82 |
28 |
11.01 |
28 |
11.24 |
27 |
16.43 |
18 |
22.12 |
20 |
6.66 |
27 |
13.93 |
26 |
16.87 |
30 |
GC(subpixel) |
29.33 |
5.09 |
8 |
7.21 |
16 |
15.38 |
18 |
2.03 |
30 |
3.7 |
33 |
10.27 |
24 |
17.91 |
40 |
26.36 |
44 |
27.51 |
38 |
9.04 |
33 |
19.39 |
36 |
17.59 |
32 |
TreeDP(subpixel) |
30.17 |
6.32 |
19 |
7.29 |
17 |
16.34 |
26 |
1.38 |
21 |
2.15 |
20 |
9.04 |
21 |
17.99 |
41 |
25.88 |
42 |
30.23 |
42 |
11.49 |
38 |
19.71 |
38 |
21.87 |
37 |
Layered |
32.75 |
12.95 |
33 |
13.51 |
33 |
18.67 |
35 |
4.71 |
41 |
5.33 |
39 |
10.57 |
25 |
12.01 |
30 |
17.86 |
27 |
23.72 |
28 |
9.17 |
34 |
17.33 |
34 |
18.31 |
34 |
RealTimeGPU |
35 |
24.21 |
47 |
26.02 |
48 |
24.94 |
42 |
3.3 |
35 |
4.43 |
36 |
22.01 |
43 |
10.15 |
23 |
17.8 |
26 |
22.14 |
21 |
8.68 |
32 |
16.17 |
31 |
20.58 |
36 |
GC |
35.83 |
7.71 |
24 |
9.82 |
24 |
17.4 |
29 |
3.96 |
38 |
5.6 |
41 |
11.28 |
30 |
21.32 |
48 |
29.41 |
48 |
29.29 |
40 |
9.84 |
36 |
20.1 |
39 |
18.28 |
33 |
DP(subpixel) |
36.08 |
7.23 |
22 |
8.34 |
22 |
16.77 |
27 |
9.11 |
47 |
10.11 |
46 |
21.9 |
42 |
16.9 |
38 |
24.42 |
38 |
24.03 |
29 |
12.07 |
41 |
20.54 |
40 |
23.83 |
41 |
ReliabilityDP |
38.25 |
18.99 |
39 |
20.73 |
41 |
17.54 |
30 |
3.98 |
39 |
5.2 |
38 |
15.94 |
36 |
14.69 |
35 |
21.64 |
35 |
25.12 |
32 |
15.66 |
48 |
22.55 |
46 |
23.49 |
40 |
SSD+MF(subpixel) |
39 |
11.23 |
31 |
13.1 |
32 |
31.79 |
46 |
3.71 |
37 |
5.15 |
37 |
13.08 |
34 |
17.34 |
39 |
25.6 |
40 |
35.28 |
45 |
11.9 |
40 |
21.09 |
42 |
28.19 |
45 |
STICA(subpixel) |
39.75 |
10.43 |
28 |
12.4 |
29 |
31.36 |
45 |
5.99 |
43 |
7.39 |
44 |
36.1 |
49 |
16.09 |
37 |
24.14 |
37 |
38.29 |
47 |
10.12 |
37 |
18.22 |
35 |
29.21 |
46 |
TreeDP |
39.83 |
22.42 |
46 |
23.05 |
45 |
22.33 |
38 |
2.86 |
33 |
3.6 |
32 |
10.19 |
22 |
21.32 |
47 |
28.86 |
47 |
33.72 |
44 |
13.65 |
42 |
21.72 |
43 |
23.29 |
39 |
SO(subpixel) |
40.75 |
8.31 |
26 |
10.43 |
26 |
15.96 |
24 |
8.75 |
46 |
10.29 |
47 |
22.48 |
44 |
22.38 |
49 |
30.37 |
49 |
29.37 |
41 |
14.54 |
46 |
24.24 |
49 |
24.76 |
42 |
Infection(subpixel) |
42.17 |
11 |
30 |
12.6 |
30 |
32.36 |
47 |
4.21 |
40 |
5.33 |
40 |
31.44 |
47 |
18.51 |
43 |
25.53 |
39 |
45.51 |
49 |
15.49 |
47 |
22.37 |
45 |
40.25 |
49 |
DP |
42.75 |
19.58 |
41 |
20.59 |
39 |
22.84 |
39 |
13.6 |
50 |
14.5 |
49 |
24.08 |
45 |
19.17 |
44 |
26.31 |
43 |
25.64 |
33 |
13.76 |
43 |
22.09 |
44 |
25.71 |
43 |
SSD+MF |
45.25 |
28.53 |
50 |
29.99 |
50 |
38.06 |
49 |
4.98 |
42 |
6.43 |
42 |
14.51 |
35 |
19.79 |
45 |
27.84 |
46 |
38.24 |
46 |
13.8 |
44 |
22.79 |
47 |
31.24 |
47 |
SO |
45.42 |
17.85 |
36 |
19.81 |
38 |
23.37 |
40 |
13.06 |
49 |
14.52 |
50 |
25.25 |
46 |
25.61 |
50 |
33.28 |
50 |
31.26 |
43 |
16.02 |
49 |
25.55 |
50 |
26.8 |
44 |
STICA |
45.67 |
24.33 |
48 |
26.1 |
49 |
44.76 |
50 |
9.65 |
48 |
11.02 |
48 |
42.57 |
50 |
18.21 |
42 |
25.76 |
41 |
42.79 |
48 |
11.59 |
39 |
19.56 |
37 |
33.08 |
48 |
Infection |
47.17 |
21.94 |
45 |
23.29 |
46 |
37.02 |
48 |
6.5 |
45 |
7.67 |
45 |
33.56 |
48 |
20.11 |
46 |
27.67 |
45 |
49.22 |
50 |
16.53 |
50 |
23.89 |
48 |
42.11 |
50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|