Overall comparison of algorithms with
sub-pixel estimation using two views |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
table below is the current, off-line version of Figure 3 of our paper. It
compares the performance of different stereo algorithms on four test |
|
|
|
|
|
|
image
pairs, including all the algorithms reported to the Middlebury website. The
numbers represent the percentage of "bad" pixels, i.e., pixel
whose |
|
|
|
|
|
absolute
disparity error is greater than [0.50 | 0.75 | 1.00 | 1.50 |2.00], the error
threshold can be changed by clicking on the tabs at the end of |
|
|
|
|
|
the
table. For each image pair, we report percentages for (1) non-occluded
pixels, (2) all pixels and (3) pixels near depth. All the scoring and
ranking |
|
|
|
|
|
schemes are the
same as the Middlebury benchmark. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
All
algorithms are run with constant parameter settings across all four images.
The rank of each algorithm in each column is indicated with a small red
number. |
|
|
|
The
algorithms are listed roughly in decreasing order of overall performance (as
measured by average rank). |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
difference between this benchmark and the Middlebury one is that we provide
sub-pixel estimation for all the algorithms reported to the Middlebury
website |
|
|
|
by
using only a single view (the reference view) or two views. The lines with
the blue highlighting are the experimental results with our sub-pixel
estimations. |
|
|
|
This
sub-pixel benchmark proves that the sub-pixel estimation algorithm proposed
is very robust; it works for all the algorithms. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Algorithms |
Avg. |
|
|
Tsukuba |
|
|
|
|
|
Venus |
|
|
|
|
|
Teddy |
|
|
|
|
|
Cones |
|
|
|
|
Rank |
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
DoubleBP(subpixel) |
5.75 |
8.78 |
11 |
9.45 |
7 |
14.95 |
3 |
0.72 |
2 |
1.12 |
2 |
5.24 |
2 |
10.12 |
2 |
16.42 |
3 |
21.32 |
1 |
8.49 |
14 |
14.71 |
14 |
16.52 |
8 |
AdaptingBP(subpixel) |
6.92 |
12.67 |
20 |
13.14 |
21 |
16.81 |
10 |
0.62 |
1 |
0.87 |
1 |
5.1 |
1 |
11.72 |
7 |
16.66 |
4 |
25.05 |
7 |
6.09 |
3 |
12.25 |
2 |
15.02 |
6 |
C-SemiGlob(subpixel) |
7.25 |
9.24 |
13 |
10.19 |
12 |
17.57 |
16 |
0.84 |
4 |
1.33 |
5 |
7.49 |
6 |
11.34 |
6 |
18.56 |
8 |
24.81 |
6 |
6.24 |
4 |
12.52 |
3 |
14.48 |
4 |
Segm+visib(subpixel) |
9 |
8.28 |
9 |
8.63 |
6 |
15.33 |
5 |
2.25 |
12 |
3 |
11 |
13.98 |
21 |
11.25 |
5 |
13.8 |
2 |
24.76 |
5 |
7.9 |
12 |
13.12 |
8 |
17.23 |
12 |
C-SemiGlob |
11.75 |
13.92 |
28 |
14.71 |
26 |
18.88 |
24 |
3.3 |
20 |
3.82 |
17 |
10.95 |
13 |
9.82 |
1 |
17.43 |
6 |
22.76 |
2 |
5.37 |
2 |
11.69 |
1 |
12.77 |
1 |
SymBP+occ(subpixel) |
12 |
12.71 |
22 |
13.67 |
23 |
17.76 |
18 |
0.77 |
3 |
1.13 |
3 |
5.96 |
4 |
12.59 |
10 |
18.67 |
9 |
27.87 |
11 |
8.73 |
15 |
15.19 |
15 |
16.99 |
11 |
SemiGlob(subpixel) |
12.33 |
9.17 |
12 |
10.1 |
11 |
19.69 |
28 |
1.42 |
9 |
2.37 |
9 |
14.53 |
22 |
12.53 |
9 |
20.02 |
12 |
28.35 |
13 |
6.74 |
6 |
14.05 |
10 |
16.26 |
7 |
AdaptWeight(subpixel) |
12.42 |
9.42 |
14 |
10.19 |
13 |
17.23 |
11 |
1.3 |
7 |
1.94 |
7 |
9.42 |
9 |
13.5 |
15 |
20.5 |
14 |
29.27 |
16 |
9.26 |
17 |
15.44 |
16 |
16.78 |
10 |
RegionTreeDP(subpixel) |
14 |
11.46 |
18 |
11.84 |
17 |
18.07 |
19 |
0.88 |
5 |
1.51 |
6 |
5.89 |
3 |
13.54 |
16 |
19.47 |
10 |
27.68 |
10 |
11.49 |
25 |
17.5 |
20 |
18.85 |
19 |
GC+occ(subpixel) |
14.42 |
5.91 |
1 |
6.87 |
1 |
14.43 |
1 |
2.78 |
16 |
3.5 |
14 |
10.72 |
11 |
17.22 |
25 |
24.49 |
26 |
29.6 |
17 |
9.75 |
19 |
16.96 |
19 |
19.46 |
23 |
SemiGlob |
15 |
13.42 |
25 |
14.3 |
25 |
20.35 |
32 |
4.55 |
23 |
5.38 |
24 |
15.7 |
25 |
10.96 |
4 |
18.48 |
7 |
26.07 |
8 |
4.93 |
1 |
12.54 |
4 |
13.51 |
2 |
AdaptingBP |
15.33 |
19.06 |
37 |
19.34 |
35 |
17.35 |
12 |
4.84 |
26 |
5.08 |
22 |
7.84 |
7 |
12.76 |
12 |
16.67 |
5 |
26.31 |
9 |
7.02 |
7 |
13.15 |
9 |
13.99 |
3 |
Segm+visib |
16.08 |
12.69 |
21 |
12.93 |
20 |
15.81 |
8 |
10.4 |
36 |
11.01 |
36 |
19.49 |
32 |
10.96 |
3 |
13.2 |
1 |
23.7 |
3 |
8.12 |
13 |
13.11 |
7 |
17.29 |
13 |
TensorVoting(subpixel) |
17.67 |
11.86 |
19 |
12.9 |
19 |
20.14 |
30 |
1.34 |
8 |
2.12 |
8 |
12.28 |
16 |
14.93 |
20 |
22.21 |
23 |
33.24 |
30 |
7.12 |
10 |
14.6 |
13 |
18.04 |
16 |
MultiCamGC(subpixel) |
17.83 |
6.23 |
3 |
7.17 |
3 |
15.05 |
4 |
6.29 |
30 |
6.91 |
30 |
9.39 |
8 |
18.12 |
28 |
25.07 |
30 |
31.95 |
26 |
9.3 |
18 |
16.27 |
17 |
18.29 |
17 |
Layered(subpixel) |
18.08 |
7.49 |
5 |
8 |
5 |
17.71 |
17 |
2.84 |
18 |
3.75 |
16 |
13.78 |
18 |
15.08 |
21 |
21.61 |
20 |
29.8 |
19 |
11.11 |
23 |
19.32 |
27 |
21.77 |
28 |
SegTreeDP(subpixel) |
18.17 |
21.7 |
42 |
22.3 |
42 |
24.73 |
41 |
0.94 |
6 |
1.3 |
4 |
6.86 |
5 |
15.28 |
23 |
22.19 |
22 |
28.47 |
14 |
7.1 |
9 |
12.84 |
5 |
14.53 |
5 |
RealtimeBP(subpixel) |
18.92 |
9.81 |
15 |
11.72 |
15 |
20.09 |
29 |
1.66 |
11 |
3 |
12 |
13.83 |
20 |
15.25 |
22 |
21.45 |
18 |
28.71 |
15 |
10.35 |
21 |
17.75 |
23 |
21.13 |
26 |
CostRelax(subpixel) |
20.17 |
15.57 |
29 |
16.87 |
29 |
31.59 |
43 |
1.64 |
10 |
2.74 |
10 |
18.43 |
30 |
12.7 |
11 |
20.62 |
15 |
31.75 |
25 |
7.15 |
11 |
14.18 |
11 |
18.33 |
18 |
GenModel(subpixel) |
20.83 |
10.18 |
16 |
12.26 |
18 |
20.49 |
33 |
2.8 |
17 |
4.15 |
21 |
18.61 |
31 |
13.47 |
14 |
21.51 |
19 |
29.64 |
18 |
8.75 |
16 |
18.81 |
25 |
19.03 |
22 |
DoubleBP |
21.5 |
18.66 |
35 |
19.11 |
33 |
15.76 |
7 |
7.82 |
33 |
8.22 |
32 |
11.32 |
14 |
14.4 |
17 |
19.88 |
11 |
24.35 |
4 |
11.77 |
26 |
17.65 |
22 |
19.67 |
24 |
GenModel |
22.17 |
7.89 |
8 |
10.01 |
10 |
18.52 |
21 |
4.59 |
24 |
6.03 |
25 |
23.48 |
40 |
14.76 |
19 |
22.75 |
24 |
31.75 |
24 |
10.2 |
20 |
20.2 |
30 |
19.03 |
21 |
ReliabilityDP(subpixel) |
23.5 |
8.76 |
10 |
10.72 |
14 |
16.51 |
9 |
2.68 |
15 |
4.08 |
19 |
14.71 |
24 |
17.56 |
26 |
24.82 |
28 |
31.39 |
22 |
17.08 |
41 |
24.08 |
38 |
26.43 |
36 |
CostRelax |
23.58 |
26.25 |
49 |
27.33 |
49 |
33.46 |
44 |
2.92 |
19 |
4.06 |
18 |
20.8 |
35 |
12.28 |
8 |
20.24 |
13 |
32.4 |
28 |
6.33 |
5 |
13.09 |
6 |
16.72 |
9 |
RealTimeGPU(subpixel) |
24 |
17.31 |
32 |
19.24 |
34 |
20.18 |
31 |
2.37 |
14 |
3.63 |
15 |
20.22 |
34 |
13.04 |
13 |
21.41 |
17 |
27.95 |
12 |
11.77 |
27 |
19.56 |
28 |
23.97 |
31 |
GC(subpixel) |
24.5 |
7.51 |
6 |
9.57 |
8 |
17.43 |
14 |
3.58 |
22 |
5.23 |
23 |
12.42 |
17 |
23.42 |
38 |
31.34 |
41 |
33.76 |
32 |
12.44 |
30 |
22.44 |
33 |
21.91 |
30 |
TensorVoting |
24.83 |
25.46 |
48 |
26.19 |
48 |
21.23 |
34 |
3.32 |
21 |
4.12 |
20 |
14.58 |
23 |
14.61 |
18 |
21.78 |
21 |
33.26 |
31 |
7.05 |
8 |
14.46 |
12 |
17.37 |
14 |
SymBP+occ |
25 |
20.74 |
40 |
21.57 |
40 |
19.51 |
27 |
5.96 |
28 |
6.27 |
27 |
10.19 |
10 |
15.66 |
24 |
20.92 |
16 |
31.71 |
23 |
11.4 |
24 |
17.54 |
21 |
18.94 |
20 |
GC+occ |
25.33 |
6.1 |
2 |
7.11 |
2 |
14.55 |
2 |
10.71 |
37 |
11.33 |
37 |
16.91 |
28 |
23.68 |
39 |
30.09 |
38 |
34.56 |
35 |
12.25 |
29 |
19.2 |
26 |
21.9 |
29 |
MultiCamGC |
28.17 |
6.56 |
4 |
7.55 |
4 |
15.68 |
6 |
16.4 |
48 |
16.83 |
48 |
17.57 |
29 |
24.33 |
41 |
30.43 |
39 |
36.94 |
40 |
12.04 |
28 |
18.76 |
24 |
21.15 |
27 |
SegTreeDP |
28.33 |
25.42 |
47 |
25.96 |
45 |
24.6 |
40 |
6.23 |
29 |
6.59 |
29 |
10.79 |
12 |
19.71 |
32 |
25.87 |
31 |
30.86 |
20 |
10.86 |
22 |
16.48 |
18 |
17.73 |
15 |
AdaptWeight |
29.25 |
18.1 |
34 |
18.8 |
32 |
18.59 |
22 |
7.77 |
32 |
8.4 |
33 |
15.83 |
26 |
17.57 |
27 |
23.93 |
25 |
34.03 |
34 |
14 |
32 |
19.67 |
29 |
20.63 |
25 |
TreeDP(subpixel) |
30.5 |
13.39 |
24 |
14.24 |
24 |
23.62 |
39 |
2.32 |
13 |
3.23 |
13 |
11.54 |
15 |
24.55 |
43 |
31.94 |
43 |
39.34 |
42 |
15.91 |
36 |
23.8 |
37 |
27.6 |
37 |
DP(subpixel) |
31.33 |
10.63 |
17 |
11.75 |
16 |
18.97 |
25 |
10.87 |
38 |
11.87 |
38 |
25.59 |
41 |
22.49 |
37 |
29.56 |
37 |
31.24 |
21 |
15.41 |
34 |
23.65 |
34 |
28.44 |
38 |
RegionTreeDP |
32.75 |
20.95 |
41 |
21.12 |
39 |
18.33 |
20 |
9.08 |
35 |
9.74 |
34 |
13.79 |
19 |
19.72 |
33 |
24.76 |
27 |
32.12 |
27 |
19.74 |
44 |
24.76 |
41 |
25.36 |
33 |
GC |
33.67 |
7.71 |
7 |
9.82 |
9 |
17.4 |
13 |
13.48 |
45 |
14.96 |
45 |
20.91 |
36 |
31.45 |
48 |
38.52 |
48 |
38.88 |
41 |
15.95 |
37 |
25.54 |
43 |
24.68 |
32 |
RealtimeBP |
34 |
19.94 |
39 |
21.62 |
41 |
22.16 |
35 |
8.68 |
34 |
9.93 |
35 |
20.12 |
33 |
19.24 |
29 |
24.83 |
29 |
33.82 |
33 |
14.22 |
33 |
21.25 |
32 |
25.9 |
35 |
SSD+MF(subpixel) |
34.75 |
16.22 |
30 |
17.97 |
30 |
36.16 |
46 |
4.78 |
25 |
6.22 |
26 |
15.98 |
27 |
21.22 |
34 |
29.19 |
36 |
41.74 |
44 |
15.73 |
35 |
24.6 |
40 |
33.62 |
44 |
Layered |
34.83 |
12.95 |
23 |
13.51 |
22 |
18.67 |
23 |
15.6 |
47 |
16.33 |
46 |
21.78 |
38 |
22.16 |
36 |
27.83 |
34 |
35.38 |
36 |
16.91 |
40 |
24.53 |
39 |
25.83 |
34 |
STICA(subpixel) |
35.25 |
13.89 |
27 |
15.8 |
28 |
34.07 |
45 |
6.53 |
31 |
7.94 |
31 |
36.77 |
48 |
19.49 |
30 |
27.58 |
33 |
43.68 |
45 |
12.49 |
31 |
20.57 |
31 |
33.21 |
43 |
SO(subpixel) |
37.83 |
13.5 |
26 |
15.49 |
27 |
19.23 |
26 |
11.11 |
40 |
12.61 |
41 |
26.11 |
42 |
27.96 |
45 |
35.41 |
45 |
36.01 |
37 |
17.82 |
42 |
27.19 |
44 |
29.43 |
39 |
RealTimeGPU |
38.42 |
24.21 |
45 |
26.02 |
46 |
24.94 |
42 |
10.95 |
39 |
12.12 |
39 |
27.58 |
44 |
19.62 |
31 |
27.01 |
32 |
33 |
29 |
16.55 |
39 |
23.75 |
35 |
29.52 |
40 |
Infection(subpixel) |
38.75 |
16.55 |
31 |
18.04 |
31 |
38.26 |
49 |
5.3 |
27 |
6.55 |
28 |
33.13 |
46 |
21.78 |
35 |
29.18 |
35 |
50.53 |
49 |
17.91 |
43 |
25.13 |
42 |
44.02 |
49 |
ReliabilityDP |
40.17 |
18.99 |
36 |
20.73 |
38 |
17.54 |
15 |
12.72 |
43 |
13.96 |
43 |
26.13 |
43 |
26.32 |
44 |
32.49 |
44 |
36.76 |
39 |
23.69 |
48 |
29.89 |
48 |
31.48 |
41 |
DP |
43.92 |
19.58 |
38 |
20.59 |
37 |
22.84 |
37 |
23.49 |
50 |
24.3 |
49 |
32.85 |
45 |
30.04 |
47 |
36.28 |
47 |
36.09 |
38 |
22.04 |
47 |
29.64 |
47 |
33.75 |
45 |
TreeDP |
44 |
22.42 |
44 |
23.05 |
43 |
22.33 |
36 |
12.08 |
42 |
12.92 |
42 |
21.72 |
37 |
32.39 |
49 |
38.95 |
49 |
45.59 |
46 |
23.7 |
49 |
30.77 |
49 |
31.74 |
42 |
STICA |
44.67 |
24.33 |
46 |
26.1 |
47 |
44.76 |
50 |
15.24 |
46 |
16.55 |
47 |
46.35 |
50 |
23.97 |
40 |
31.28 |
40 |
50.41 |
48 |
15.95 |
38 |
23.75 |
36 |
39.5 |
48 |
Infection |
44.83 |
22.13 |
43 |
23.49 |
44 |
37.63 |
47 |
11.13 |
41 |
12.33 |
40 |
37.32 |
49 |
24.34 |
42 |
31.71 |
42 |
56.37 |
50 |
20.16 |
45 |
27.8 |
45 |
48.17 |
50 |
SO |
45.17 |
17.85 |
33 |
19.81 |
36 |
23.37 |
38 |
23.14 |
49 |
24.44 |
50 |
34.59 |
47 |
36.3 |
50 |
42.87 |
50 |
41.41 |
43 |
23.71 |
50 |
32.4 |
50 |
34.42 |
46 |
SSD+MF |
46.08 |
28.53 |
50 |
29.99 |
50 |
38.06 |
48 |
12.83 |
44 |
14.2 |
44 |
23.41 |
39 |
28.16 |
46 |
35.42 |
46 |
48.65 |
47 |
21.08 |
46 |
29.42 |
46 |
38.61 |
47 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|