Overall
comparison of algorithms with sub-pixel estimation using a single view |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
table below is the current, off-line version of Figure 3 of our paper. It
compares the performance of different stereo algorithms on four test |
|
|
|
|
|
|
|
image
pairs, including all the algorithms reported to the Middlebury website. The
numbers represent the percentage of "bad" pixels, i.e., pixel
whose |
|
|
|
|
|
|
absolute
disparity error is greater than [0.50 | 0.75 | 1.00 | 1.50 |2.00], the error
threshold can be changed by clicking on the tabs at the end of |
|
|
|
|
|
|
the
table. For each image pair, we report percentages for (1) non-occluded
pixels, (2) all pixels and (3) pixels near depth. All the scoring and
ranking |
|
|
|
|
|
|
schemes are the
same as the Middlebury benchmark. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
All
algorithms are run with constant parameter settings across all four images.
The rank of each algorithm in each column is indicated with a small red
number. |
|
|
|
|
The
algorithms are listed roughly in decreasing order of overall performance (as
measured by average rank). |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
difference between this benchmark and the Middlebury one is that we provide
sub-pixel estimation for all the algorithms reported to the Middlebury
website |
|
|
|
|
by
using only a single view (the reference view) or two views. The lines with
the blue highlighting are the experimental results with our sub-pixel
estimations. |
|
|
|
|
This
sub-pixel benchmark proves that the sub-pixel estimation algorithm proposed
is very robust; it works for all the algorithms. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Algorithms |
Avg. |
|
|
Tsukuba |
|
|
|
|
|
Venus |
|
|
|
|
|
Teddy |
|
|
|
|
|
Cones |
|
|
|
|
|
Rank |
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
|
C-SemiGlob(subpixel) |
4.33 |
4.85 |
3 |
5.54 |
1 |
12.14 |
1 |
0.46 |
5 |
0.82 |
6 |
4.97 |
12 |
6.46 |
4 |
13.76 |
8 |
15.89 |
5 |
3.26 |
1 |
9.08 |
3 |
9.23 |
3 |
|
C-SemiGlob |
5.92 |
6.21 |
10 |
6.9 |
6 |
12.93 |
5 |
0.46 |
4 |
0.85 |
7 |
4.42 |
11 |
6.39 |
3 |
13.8 |
9 |
15.99 |
6 |
3.29 |
2 |
9.21 |
4 |
9.38 |
4 |
|
AdaptingBP(subpixel) |
7.08 |
8.4 |
16 |
8.86 |
14 |
12.48 |
3 |
1.14 |
13 |
1.34 |
9 |
4.34 |
10 |
6.78 |
6 |
10.14 |
3 |
15.71 |
4 |
3.33 |
3 |
9.02 |
2 |
9.23 |
2 |
|
DoubleBP(subpixel) |
7.25 |
12.53 |
25 |
12.91 |
22 |
12.18 |
2 |
0.39 |
1 |
0.73 |
4 |
3.42 |
3 |
5.24 |
1 |
10.7 |
5 |
13.58 |
2 |
3.9 |
8 |
9.84 |
7 |
10.17 |
7 |
|
Segm+visib(subpixel) |
9.67 |
8.79 |
17 |
9.09 |
15 |
14.1 |
8 |
1.05 |
11 |
1.44 |
11 |
7.87 |
16 |
6.52 |
5 |
8.62 |
1 |
15.67 |
3 |
4.4 |
10 |
9.48 |
6 |
11.91 |
13 |
|
SymBP+occ(subpixel) |
10.58 |
11.38 |
20 |
12.14 |
19 |
12.72 |
4 |
0.39 |
2 |
0.6 |
1 |
3.35 |
2 |
8.41 |
11 |
13.24 |
7 |
20.95 |
14 |
5.25 |
19 |
11.33 |
16 |
11.91 |
12 |
|
AdaptingBP |
10.75 |
14.03 |
31 |
14.4 |
28 |
13.49 |
7 |
1.16 |
14 |
1.36 |
10 |
4.34 |
9 |
7.16 |
8 |
10.63 |
4 |
16.61 |
8 |
3.46 |
4 |
9.34 |
5 |
9.19 |
1 |
|
SemiGlob(subpixel) |
12.58 |
5.24 |
5 |
6.1 |
4 |
15.53 |
15 |
1.94 |
25 |
2.53 |
19 |
11.81 |
30 |
7.55 |
10 |
14.32 |
12 |
19.83 |
10 |
3.48 |
5 |
10.22 |
10 |
9.95 |
6 |
|
DoubleBP |
13.25 |
18.66 |
37 |
19.11 |
36 |
15.76 |
19 |
0.82 |
8 |
1.18 |
8 |
3.65 |
6 |
5.82 |
2 |
11.17 |
6 |
13.42 |
1 |
4.76 |
13 |
10.78 |
12 |
11.52 |
11 |
|
AdaptWeight(subpixel) |
14.42 |
9.96 |
18 |
10.43 |
18 |
14.19 |
9 |
1.03 |
10 |
1.48 |
12 |
7.14 |
15 |
9.19 |
17 |
15.16 |
15 |
21.77 |
18 |
5.24 |
18 |
11.08 |
14 |
10.45 |
9 |
|
SemiGlob |
16.08 |
7.34 |
13 |
8.37 |
13 |
18.39 |
31 |
1.96 |
26 |
2.6 |
21 |
12.83 |
32 |
7.47 |
9 |
14.43 |
13 |
19.58 |
9 |
3.61 |
7 |
10.67 |
11 |
10.44 |
8 |
|
SegTreeDP(subpixel) |
16.33 |
16.79 |
33 |
17.26 |
33 |
19.96 |
35 |
0.5 |
7 |
0.7 |
3 |
3.48 |
5 |
11.21 |
27 |
17.42 |
24 |
21.69 |
17 |
3.61 |
6 |
8.86 |
1 |
9.71 |
5 |
|
RegionTreeDP(subpixel) |
16.58 |
13.64 |
29 |
13.86 |
26 |
16.24 |
24 |
0.44 |
3 |
0.81 |
5 |
3.01 |
1 |
8.83 |
13 |
14.06 |
11 |
20.1 |
11 |
7.43 |
30 |
13.14 |
23 |
13.45 |
23 |
|
Segm+visib |
17.08 |
12.69 |
26 |
12.93 |
23 |
15.81 |
20 |
2.64 |
31 |
3.08 |
27 |
10.25 |
24 |
7.04 |
7 |
8.71 |
2 |
16.25 |
7 |
4.86 |
15 |
9.86 |
8 |
12.21 |
15 |
|
GenModel(subpixel) |
19.42 |
4.15 |
1 |
6.36 |
5 |
15.55 |
16 |
1.76 |
22 |
3.14 |
28 |
17.14 |
37 |
8.89 |
14 |
17.01 |
21 |
22.67 |
23 |
5.5 |
20 |
15.64 |
30 |
12.35 |
16 |
|
SymBP+occ |
19.67 |
20.74 |
43 |
21.57 |
43 |
19.51 |
34 |
0.48 |
6 |
0.7 |
2 |
3.44 |
4 |
9.03 |
15 |
13.98 |
10 |
22.26 |
22 |
5.85 |
21 |
12.06 |
18 |
12.82 |
18 |
|
TensorVoting(subpixel) |
20.5 |
11.73 |
22 |
12.68 |
21 |
15.57 |
17 |
1.16 |
15 |
1.83 |
15 |
11.64 |
28 |
10.39 |
24 |
17.72 |
26 |
25.73 |
33 |
4.49 |
11 |
11.71 |
17 |
12.76 |
17 |
|
RealtimeBP(subpixel) |
20.58 |
13.16 |
28 |
14.86 |
30 |
17.65 |
29 |
0.98 |
9 |
2.07 |
17 |
9.54 |
22 |
10.63 |
25 |
15.96 |
16 |
21.32 |
15 |
4.94 |
16 |
12.22 |
20 |
13.22 |
20 |
|
GC+occ(subpixel) |
20.92 |
4.8 |
2 |
5.65 |
2 |
12.96 |
6 |
2.43 |
28 |
3 |
26 |
7.99 |
17 |
14.12 |
33 |
20.74 |
32 |
24.17 |
29 |
6.27 |
25 |
13.47 |
25 |
14.68 |
26 |
|
Layered(subpixel) |
21.75 |
7.11 |
12 |
7.62 |
11 |
15.81 |
21 |
1.93 |
24 |
2.61 |
22 |
9.28 |
21 |
10.12 |
22 |
16.04 |
17 |
21.6 |
16 |
8.08 |
31 |
16.25 |
32 |
16.96 |
32 |
|
AdaptWeight |
23 |
18.1 |
36 |
18.8 |
35 |
18.59 |
32 |
1.32 |
17 |
1.9 |
16 |
8.27 |
18 |
10.11 |
21 |
16.29 |
18 |
23.45 |
26 |
6.17 |
24 |
12.15 |
19 |
12.11 |
14 |
|
GenModel |
23.08 |
5.28 |
6 |
7.47 |
9 |
15.84 |
22 |
2.04 |
27 |
3.45 |
29 |
17.66 |
38 |
9.19 |
18 |
17.42 |
25 |
23.12 |
25 |
5.97 |
23 |
16.28 |
33 |
13.4 |
22 |
|
SegTreeDP |
23.83 |
25.42 |
49 |
25.96 |
47 |
24.6 |
41 |
1.29 |
16 |
1.53 |
13 |
4.21 |
8 |
12.34 |
30 |
18.45 |
30 |
21.96 |
19 |
4.78 |
14 |
10.09 |
9 |
11.45 |
10 |
|
MultiCamGC(subpixel) |
24 |
5.13 |
4 |
6 |
3 |
14.39 |
10 |
5.56 |
42 |
5.97 |
41 |
5.49 |
13 |
15.07 |
36 |
21.28 |
34 |
26.48 |
36 |
5.9 |
22 |
12.89 |
22 |
14.01 |
25 |
|
RegionTreeDP |
24.17 |
20.95 |
44 |
21.12 |
42 |
18.33 |
30 |
1.09 |
12 |
1.57 |
14 |
4.21 |
7 |
10 |
20 |
15.12 |
14 |
20.75 |
13 |
9.8 |
36 |
15.45 |
29 |
16.13 |
29 |
|
GC+occ |
24.58 |
6.1 |
9 |
7.11 |
8 |
14.55 |
11 |
3.2 |
34 |
3.8 |
33 |
8.4 |
19 |
14.67 |
34 |
21.27 |
33 |
24.41 |
30 |
6.81 |
29 |
14 |
27 |
15.8 |
28 |
|
TensorVoting |
25.42 |
16.84 |
34 |
17.68 |
34 |
15.23 |
13 |
1.66 |
20 |
2.37 |
18 |
12.35 |
31 |
11.09 |
26 |
18.4 |
29 |
26.81 |
38 |
5.18 |
17 |
12.36 |
21 |
13.96 |
24 |
|
RealTimeGPU(subpixel) |
25.75 |
12.51 |
24 |
14.51 |
29 |
17.56 |
28 |
2.47 |
29 |
3.54 |
30 |
19.23 |
39 |
8.68 |
12 |
16.5 |
20 |
20.23 |
12 |
6.74 |
28 |
14.3 |
28 |
16.63 |
30 |
|
CostRelax(subpixel) |
26.17 |
13.67 |
30 |
14.9 |
31 |
27.5 |
43 |
1.68 |
21 |
2.76 |
24 |
19.31 |
40 |
9.22 |
19 |
17.21 |
23 |
26.31 |
35 |
4.57 |
12 |
11.08 |
15 |
13.3 |
21 |
|
CostRelax |
26.67 |
19.48 |
40 |
20.65 |
40 |
29.54 |
44 |
1.65 |
19 |
2.73 |
23 |
19.45 |
41 |
9.18 |
16 |
17.14 |
22 |
26.29 |
34 |
4.37 |
9 |
10.89 |
13 |
12.92 |
19 |
|
MultiCamGC |
27.25 |
6.56 |
11 |
7.55 |
10 |
15.68 |
18 |
6.33 |
43 |
6.75 |
43 |
6.67 |
14 |
15.51 |
37 |
21.69 |
37 |
26.81 |
37 |
6.35 |
26 |
13.31 |
24 |
14.84 |
27 |
|
RealtimeBP |
28.58 |
19.94 |
42 |
21.62 |
44 |
22.16 |
37 |
1.61 |
18 |
2.82 |
25 |
11.01 |
26 |
11.24 |
28 |
16.43 |
19 |
22.12 |
20 |
6.66 |
27 |
13.93 |
26 |
16.87 |
31 |
|
ReliabilityDP(subpixel) |
30.5 |
11.64 |
21 |
13.5 |
24 |
14.77 |
12 |
2.48 |
30 |
3.72 |
32 |
13.67 |
34 |
13.13 |
31 |
20.29 |
31 |
24.14 |
28 |
13.77 |
45 |
20.85 |
41 |
21.16 |
37 |
|
DP(subpixel) |
30.5 |
5.97 |
7 |
7 |
7 |
15.31 |
14 |
8.75 |
47 |
9.74 |
46 |
21.79 |
42 |
13.98 |
32 |
21.56 |
35 |
22.86 |
24 |
10.53 |
38 |
19.13 |
36 |
21.24 |
38 |
|
Layered |
31.17 |
12.95 |
27 |
13.51 |
25 |
18.67 |
33 |
4.71 |
40 |
5.33 |
38 |
10.57 |
25 |
12.01 |
29 |
17.86 |
28 |
23.72 |
27 |
9.17 |
33 |
17.33 |
34 |
18.31 |
35 |
|
GC(subpixel) |
31.75 |
6.07 |
8 |
8.16 |
12 |
16.83 |
25 |
3.15 |
33 |
4.77 |
35 |
11.68 |
29 |
20.72 |
47 |
28.85 |
47 |
28.84 |
40 |
9.54 |
34 |
19.76 |
38 |
17.91 |
33 |
|
GC |
34.08 |
7.71 |
14 |
9.82 |
16 |
17.4 |
26 |
3.96 |
37 |
5.6 |
40 |
11.28 |
27 |
21.32 |
49 |
29.41 |
49 |
29.29 |
41 |
9.84 |
37 |
20.1 |
39 |
18.28 |
34 |
|
TreeDP(subpixel) |
34.67 |
15.93 |
32 |
16.65 |
32 |
21.63 |
36 |
1.84 |
23 |
2.59 |
20 |
8.98 |
20 |
20.52 |
46 |
28.13 |
45 |
33.24 |
43 |
12.28 |
40 |
20.41 |
40 |
22.88 |
39 |
|
RealTimeGPU |
34.92 |
24.21 |
47 |
26.02 |
48 |
24.94 |
42 |
3.3 |
35 |
4.43 |
34 |
22.01 |
43 |
10.15 |
23 |
17.8 |
27 |
22.14 |
21 |
8.68 |
32 |
16.17 |
31 |
20.58 |
36 |
|
SO(subpixel) |
37.75 |
8.01 |
15 |
10.11 |
17 |
16 |
23 |
8.51 |
46 |
10.05 |
47 |
24.45 |
45 |
20.12 |
45 |
28.35 |
46 |
28.75 |
39 |
13.38 |
42 |
23.18 |
48 |
23.18 |
40 |
|
ReliabilityDP |
38 |
18.99 |
39 |
20.73 |
41 |
17.54 |
27 |
3.98 |
38 |
5.2 |
37 |
15.94 |
36 |
14.69 |
35 |
21.64 |
36 |
25.12 |
31 |
15.66 |
48 |
22.55 |
46 |
23.49 |
42 |
|
STICA(subpixel) |
39.33 |
12.48 |
23 |
14.36 |
27 |
32.52 |
45 |
7.71 |
45 |
9.1 |
45 |
37.98 |
49 |
16.13 |
38 |
23.84 |
38 |
38.97 |
47 |
9.55 |
35 |
17.68 |
35 |
28.07 |
45 |
|
TreeDP |
40.08 |
22.42 |
46 |
23.05 |
45 |
22.33 |
38 |
2.86 |
32 |
3.6 |
31 |
10.19 |
23 |
21.32 |
48 |
28.86 |
48 |
33.72 |
44 |
13.65 |
43 |
21.72 |
42 |
23.29 |
41 |
|
Infection(subpixel) |
40.17 |
11 |
19 |
12.54 |
20 |
32.89 |
46 |
4.33 |
39 |
5.44 |
39 |
32.37 |
47 |
18.79 |
41 |
26.51 |
41 |
47.04 |
49 |
15.21 |
47 |
22.53 |
45 |
39.74 |
49 |
|
SSD+MF(subpixel) |
40.42 |
18.92 |
38 |
20.57 |
38 |
34.97 |
47 |
3.71 |
36 |
5.08 |
36 |
13.04 |
33 |
18.57 |
40 |
26.66 |
42 |
37.24 |
45 |
12.99 |
41 |
21.96 |
43 |
31.02 |
46 |
|
DP |
42.25 |
19.58 |
41 |
20.59 |
39 |
22.84 |
39 |
13.6 |
50 |
14.5 |
49 |
24.08 |
44 |
19.17 |
42 |
26.31 |
40 |
25.64 |
32 |
13.76 |
44 |
22.09 |
44 |
25.71 |
43 |
|
SSD+MF |
45 |
28.53 |
50 |
29.99 |
50 |
38.06 |
49 |
4.98 |
41 |
6.43 |
42 |
14.51 |
35 |
19.79 |
43 |
27.84 |
44 |
38.24 |
46 |
13.8 |
46 |
22.79 |
47 |
31.24 |
47 |
|
SO |
45.17 |
17.85 |
35 |
19.81 |
37 |
23.37 |
40 |
13.06 |
49 |
14.52 |
50 |
25.25 |
46 |
25.61 |
50 |
33.28 |
50 |
31.26 |
42 |
16.02 |
49 |
25.55 |
50 |
26.8 |
44 |
|
STICA |
45.25 |
24.33 |
48 |
26.1 |
49 |
44.76 |
50 |
9.65 |
48 |
11.02 |
48 |
42.57 |
50 |
18.21 |
39 |
25.76 |
39 |
42.79 |
48 |
11.59 |
39 |
19.56 |
37 |
33.08 |
48 |
|
Infection |
46.75 |
21.94 |
45 |
23.29 |
46 |
37.02 |
48 |
6.5 |
44 |
7.67 |
44 |
33.56 |
48 |
20.11 |
44 |
27.67 |
43 |
49.22 |
50 |
16.53 |
50 |
23.89 |
49 |
42.11 |
50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|