Overall
comparison of algorithms with sub-pixel estimation using a single view |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
table below is the current, off-line version of Figure 3 of our paper. It
compares the performance of different stereo algorithms on four test |
|
|
|
|
|
|
|
image
pairs, including all the algorithms reported to the Middlebury website. The
numbers represent the percentage of "bad" pixels, i.e., pixel
whose |
|
|
|
|
|
|
absolute
disparity error is greater than [0.50 | 0.75 | 1.00 | 1.50 |2.00], the error
threshold can be changed by clicking on the tabs at the end of |
|
|
|
|
|
|
the
table. For each image pair, we report percentages for (1) non-occluded
pixels, (2) all pixels and (3) pixels near depth. All the scoring and
ranking |
|
|
|
|
|
|
schemes are the
same as the Middlebury benchmark. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
All
algorithms are run with constant parameter settings across all four images.
The rank of each algorithm in each column is indicated with a small red
number. |
|
|
|
|
The
algorithms are listed roughly in decreasing order of overall performance (as
measured by average rank). |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
difference between this benchmark and the Middlebury one is that we provide
sub-pixel estimation for all the algorithms reported to the Middlebury
website |
|
|
|
|
by
using only a single view (the reference view) or two views. The lines with
the blue highlighting are the experimental results with our sub-pixel
estimations. |
|
|
|
|
This
sub-pixel benchmark proves that the sub-pixel estimation algorithm proposed
is very robust; it works for all the algorithms. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Algorithms |
Avg. |
|
|
Tsukuba |
|
|
|
|
|
Venus |
|
|
|
|
|
Teddy |
|
|
|
|
|
Cones |
|
|
|
|
|
Rank |
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
|
AdaptingBP |
3.17 |
1.08 |
5 |
1.33 |
3 |
5.79 |
5 |
0.1 |
2 |
0.2 |
2 |
1.42 |
3 |
1.91 |
2 |
3.63 |
2 |
6.05 |
2 |
2.18 |
3 |
7.06 |
4 |
6.52 |
5 |
|
AdaptingBP(subpixel) |
3.33 |
1.08 |
6 |
1.33 |
4 |
5.79 |
6 |
0.1 |
3 |
0.18 |
1 |
1.42 |
4 |
1.78 |
1 |
3.55 |
1 |
5.66 |
1 |
2.19 |
4 |
7.05 |
3 |
6.56 |
6 |
|
AdaptWeight |
16.33 |
1.38 |
18 |
1.85 |
12 |
6.9 |
13 |
0.55 |
22 |
0.98 |
21 |
4.76 |
20 |
5.98 |
22 |
9.56 |
17 |
14.24 |
23 |
2.79 |
13 |
8.1 |
12 |
6.08 |
3 |
|
AdaptWeight(subpixel) |
15.58 |
1.38 |
19 |
1.82 |
11 |
7.02 |
16 |
0.51 |
21 |
0.83 |
20 |
4.21 |
17 |
5.91 |
21 |
9.37 |
15 |
14.17 |
22 |
2.66 |
10 |
8.09 |
11 |
6.1 |
4 |
|
CostRelax |
33.17 |
4.7 |
42 |
5.99 |
42 |
20.01 |
44 |
1.1 |
33 |
2.08 |
34 |
14.34 |
39 |
6.83 |
28 |
14.2 |
33 |
20.21 |
35 |
3.63 |
20 |
9.51 |
19 |
11.04 |
29 |
|
CostRelax(subpixel) |
33.42 |
4.7 |
43 |
5.98 |
41 |
20 |
43 |
1.08 |
31 |
1.99 |
32 |
13.95 |
38 |
6.86 |
29 |
14.23 |
34 |
20.3 |
38 |
3.67 |
22 |
9.62 |
20 |
11.15 |
30 |
|
C-SemiGlob |
10.67 |
1.34 |
13 |
1.86 |
13 |
7.16 |
17 |
0.23 |
11 |
0.5 |
13 |
3.16 |
13 |
3.74 |
9 |
8.63 |
13 |
9.76 |
7 |
2.38 |
6 |
7.38 |
5 |
7.1 |
8 |
|
C-SemiGlob(subpixel) |
11.67 |
1.35 |
14 |
1.86 |
14 |
7.23 |
18 |
0.23 |
12 |
0.44 |
11 |
3.16 |
14 |
3.74 |
10 |
8.55 |
12 |
9.76 |
8 |
2.42 |
8 |
7.47 |
9 |
7.22 |
10 |
|
DoubleBP |
3.92 |
0.88 |
1 |
1.29 |
1 |
4.76 |
1 |
0.11 |
5 |
0.42 |
10 |
1.47 |
5 |
2.03 |
3 |
5.46 |
6 |
6.38 |
3 |
1.98 |
1 |
7.5 |
10 |
5.92 |
1 |
|
DoubleBP(subpixel) |
4.25 |
0.88 |
2 |
1.29 |
2 |
4.76 |
2 |
0.11 |
6 |
0.35 |
8 |
1.47 |
6 |
2.09 |
4 |
5.44 |
5 |
6.45 |
4 |
1.98 |
2 |
7.47 |
8 |
5.95 |
2 |
|
DP |
41.42 |
4.12 |
40 |
5.04 |
40 |
11.95 |
38 |
7.28 |
50 |
8.24 |
50 |
18.03 |
45 |
9.04 |
38 |
16.77 |
38 |
15.68 |
32 |
8.1 |
42 |
16.71 |
42 |
17.56 |
42 |
|
DP(subpixel) |
38.92 |
3.98 |
39 |
4.9 |
39 |
11.78 |
37 |
6.72 |
49 |
7.65 |
49 |
17.61 |
44 |
7.78 |
31 |
15.59 |
37 |
14.81 |
29 |
7.21 |
37 |
15.88 |
37 |
16.6 |
39 |
|
GC |
35 |
1.94 |
29 |
4.12 |
35 |
9.39 |
28 |
0.95 |
27 |
2.6 |
38 |
8.46 |
30 |
12.07 |
42 |
21.03 |
44 |
20.26 |
36 |
5.61 |
36 |
16.18 |
41 |
12.69 |
34 |
|
GC(subpixel) |
34.75 |
1.92 |
28 |
4.09 |
34 |
9.37 |
27 |
0.95 |
28 |
2.51 |
37 |
8.48 |
31 |
12.09 |
43 |
21.03 |
45 |
20.28 |
37 |
5.44 |
35 |
15.98 |
39 |
12.62 |
33 |
|
GC+occ |
25.25 |
1.19 |
8 |
2.01 |
20 |
6.24 |
8 |
1.22 |
34 |
1.75 |
28 |
6.39 |
23 |
8.08 |
32 |
13.06 |
32 |
14.97 |
30 |
4.46 |
30 |
11.32 |
27 |
11.42 |
31 |
|
GC+occ(subpixel) |
25.33 |
1.17 |
7 |
1.99 |
18 |
6.19 |
7 |
1.22 |
35 |
1.7 |
27 |
6.39 |
24 |
8.09 |
33 |
13.05 |
31 |
14.99 |
31 |
4.46 |
31 |
11.36 |
28 |
11.43 |
32 |
|
GenModel |
31.42 |
2.46 |
37 |
4.62 |
37 |
12.52 |
41 |
1.52 |
40 |
2.8 |
40 |
15.28 |
41 |
5.03 |
16 |
12.63 |
26 |
14.81 |
28 |
3.54 |
17 |
13.53 |
34 |
9.39 |
20 |
|
GenModel(subpixel) |
31.17 |
2.46 |
38 |
4.62 |
38 |
12.52 |
42 |
1.5 |
39 |
2.77 |
39 |
14.91 |
40 |
4.85 |
14 |
12.43 |
25 |
14.5 |
27 |
3.56 |
18 |
13.47 |
33 |
9.42 |
21 |
|
Infection |
47.58 |
7.36 |
49 |
8.88 |
48 |
26.88 |
47 |
3.31 |
42 |
4.41 |
42 |
30.15 |
49 |
14.76 |
50 |
21.16 |
46 |
38.3 |
50 |
11.91 |
50 |
18.24 |
48 |
33.14 |
50 |
|
Infection(subpixel) |
46.5 |
7.28 |
48 |
8.82 |
47 |
26.99 |
48 |
2.9 |
41 |
3.9 |
41 |
27.45 |
47 |
14.45 |
49 |
20.89 |
43 |
37.72 |
49 |
11.78 |
49 |
18.19 |
47 |
32.76 |
49 |
|
Layered |
22.83 |
1.57 |
25 |
1.87 |
15 |
8.28 |
25 |
0.42 |
17 |
0.77 |
19 |
4.52 |
19 |
5.73 |
20 |
10.81 |
22 |
13.36 |
20 |
4.58 |
32 |
12.42 |
32 |
10.83 |
28 |
|
Layered(subpixel) |
21.42 |
1.57 |
26 |
1.87 |
16 |
8.28 |
26 |
0.34 |
13 |
0.65 |
16 |
4.16 |
16 |
5.52 |
19 |
10.58 |
20 |
13.17 |
19 |
4.26 |
29 |
12.03 |
31 |
10.56 |
26 |
|
MultiCamGC |
21.92 |
1.27 |
10 |
1.99 |
19 |
6.48 |
10 |
0.42 |
18 |
0.71 |
18 |
2.42 |
12 |
8.75 |
36 |
12.68 |
29 |
16.79 |
34 |
3.94 |
27 |
10.79 |
26 |
10.37 |
24 |
|
MultiCamGC(subpixel) |
20.58 |
1.26 |
9 |
1.98 |
17 |
6.46 |
9 |
0.38 |
16 |
0.63 |
14 |
2.3 |
11 |
8.75 |
37 |
12.66 |
27 |
16.78 |
33 |
3.93 |
26 |
10.76 |
25 |
10.33 |
23 |
|
RealtimeBP |
23.08 |
1.49 |
22 |
3.4 |
32 |
7.87 |
24 |
0.64 |
24 |
1.65 |
26 |
7.79 |
28 |
6.59 |
26 |
9.85 |
18 |
12.8 |
17 |
3.35 |
16 |
9.9 |
22 |
9.58 |
22 |
|
RealtimeBP(subpixel) |
21.08 |
1.48 |
21 |
3.37 |
29 |
7.86 |
23 |
0.63 |
23 |
1.56 |
24 |
7.76 |
27 |
6.38 |
23 |
9.42 |
16 |
12.32 |
13 |
3.23 |
15 |
9.73 |
21 |
9.25 |
18 |
|
RealTimeGPU |
31.08 |
2.04 |
32 |
4.21 |
36 |
10.57 |
35 |
1.39 |
38 |
2.43 |
36 |
17.01 |
42 |
4.86 |
15 |
10.69 |
21 |
13.08 |
18 |
5.13 |
34 |
11.94 |
30 |
13.94 |
36 |
|
RealTimeGPU(subpixel) |
27.67 |
1.91 |
27 |
4.08 |
33 |
10.11 |
31 |
1.08 |
30 |
1.99 |
31 |
13.06 |
37 |
4.7 |
13 |
10.47 |
19 |
12.43 |
14 |
4.87 |
33 |
11.68 |
29 |
13.63 |
35 |
|
RegionTreeDP |
13.33 |
1.39 |
20 |
1.64 |
8 |
6.85 |
12 |
0.1 |
4 |
0.35 |
9 |
1.14 |
1 |
5.19 |
18 |
8.05 |
10 |
12.75 |
16 |
4.11 |
28 |
9.41 |
18 |
9.14 |
16 |
|
RegionTreeDP(subpixel) |
11.58 |
1.37 |
17 |
1.62 |
7 |
6.81 |
11 |
0.09 |
1 |
0.31 |
5 |
1.14 |
2 |
5.08 |
17 |
7.83 |
9 |
12.56 |
15 |
3.87 |
25 |
9.19 |
15 |
8.96 |
15 |
|
ReliabilityDP |
31.75 |
1.36 |
16 |
3.39 |
31 |
7.25 |
20 |
1.09 |
32 |
2.19 |
35 |
8.54 |
32 |
6.55 |
25 |
12.82 |
30 |
14.31 |
25 |
11.12 |
48 |
18.13 |
46 |
17.05 |
41 |
|
ReliabilityDP(subpixel) |
30 |
1.35 |
15 |
3.38 |
30 |
7.23 |
19 |
1.05 |
29 |
2.03 |
33 |
8.23 |
29 |
6.39 |
24 |
12.66 |
28 |
14.04 |
21 |
10.59 |
47 |
17.62 |
45 |
16.73 |
40 |
|
Segm+visib |
11.33 |
1.3 |
11 |
1.57 |
5 |
6.92 |
14 |
0.45 |
19 |
0.64 |
15 |
5.29 |
21 |
3.13 |
6 |
4.64 |
4 |
8.23 |
6 |
2.82 |
14 |
7.45 |
7 |
8.12 |
14 |
|
Segm+visib(subpixel) |
9.83 |
1.3 |
12 |
1.57 |
6 |
6.92 |
15 |
0.35 |
14 |
0.48 |
12 |
4.16 |
15 |
2.99 |
5 |
4.5 |
3 |
7.87 |
5 |
2.77 |
12 |
7.4 |
6 |
8.08 |
13 |
|
SegTreeDP |
17.75 |
2.21 |
34 |
2.76 |
24 |
10.32 |
33 |
0.2 |
10 |
0.31 |
6 |
1.93 |
8 |
6.92 |
30 |
11.05 |
24 |
14.43 |
26 |
2.41 |
7 |
6.15 |
2 |
7.13 |
9 |
|
SegTreeDP(subpixel) |
16.17 |
2.15 |
33 |
2.69 |
23 |
10.25 |
32 |
0.15 |
9 |
0.25 |
3 |
1.91 |
7 |
6.7 |
27 |
10.85 |
23 |
14.24 |
24 |
2.34 |
5 |
6.04 |
1 |
7.01 |
7 |
|
SemiGlob |
19.58 |
2.33 |
36 |
2.97 |
28 |
10.66 |
36 |
0.49 |
20 |
0.98 |
22 |
5.82 |
22 |
4.44 |
12 |
8.67 |
14 |
12.04 |
12 |
2.64 |
9 |
8.77 |
13 |
7.78 |
11 |
|
SemiGlob(subpixel) |
18 |
2.3 |
35 |
2.92 |
27 |
10.56 |
34 |
0.38 |
15 |
0.71 |
17 |
4.35 |
18 |
4.27 |
11 |
8.37 |
11 |
11.51 |
11 |
2.66 |
11 |
8.79 |
14 |
7.84 |
12 |
|
SO |
46.08 |
5.08 |
44 |
7.22 |
46 |
12.18 |
40 |
6.1 |
47 |
7.64 |
48 |
18.26 |
46 |
13.89 |
48 |
22.66 |
50 |
20.97 |
44 |
10.31 |
46 |
20.35 |
50 |
18.52 |
44 |
|
SO(subpixel) |
43.83 |
4.68 |
41 |
6.82 |
43 |
11.99 |
39 |
5.32 |
46 |
6.77 |
46 |
17.41 |
43 |
12.28 |
44 |
21.2 |
47 |
20.37 |
40 |
9.52 |
45 |
19.59 |
49 |
17.64 |
43 |
|
SSD+MF |
44.58 |
5.23 |
45 |
7.07 |
45 |
24.14 |
45 |
3.44 |
44 |
4.79 |
44 |
10.79 |
36 |
13.38 |
47 |
21.78 |
49 |
27.23 |
46 |
8.19 |
44 |
17.47 |
44 |
22.14 |
46 |
|
SSD+MF(subpixel) |
44.08 |
5.23 |
46 |
7.06 |
44 |
24.14 |
46 |
3.41 |
43 |
4.63 |
43 |
10.73 |
35 |
13.22 |
46 |
21.53 |
48 |
27.04 |
45 |
8.14 |
43 |
17.43 |
43 |
22.21 |
47 |
|
STICA |
46.25 |
7.7 |
50 |
9.63 |
50 |
27.83 |
50 |
6.22 |
48 |
7.6 |
47 |
34.02 |
50 |
12.41 |
45 |
19.4 |
42 |
30.81 |
48 |
7.78 |
41 |
15.59 |
36 |
23.27 |
48 |
|
STICA(subpixel) |
44 |
7.17 |
47 |
9.09 |
49 |
27.25 |
49 |
5.09 |
45 |
6.33 |
45 |
28.89 |
48 |
11.97 |
41 |
18.97 |
39 |
29.47 |
47 |
7.31 |
38 |
15.06 |
35 |
21.98 |
45 |
|
SymBP+occ |
10.67 |
0.97 |
4 |
1.75 |
10 |
5.09 |
4 |
0.15 |
7 |
0.32 |
7 |
2.13 |
10 |
3.72 |
8 |
6.47 |
8 |
11.34 |
10 |
3.79 |
24 |
9.24 |
17 |
9.35 |
19 |
|
SymBP+occ(subpixel) |
9.58 |
0.94 |
3 |
1.68 |
9 |
5.06 |
3 |
0.15 |
8 |
0.3 |
4 |
2.1 |
9 |
3.36 |
7 |
6.21 |
7 |
10.5 |
9 |
3.75 |
23 |
9.21 |
16 |
9.25 |
17 |
|
TensorVoting |
27.58 |
1.5 |
23 |
2.49 |
22 |
7.42 |
22 |
0.94 |
26 |
1.58 |
25 |
9.64 |
34 |
8.29 |
35 |
15.56 |
36 |
20.43 |
41 |
3.6 |
19 |
10.44 |
23 |
10.53 |
25 |
|
TensorVoting(subpixel) |
27.25 |
1.5 |
24 |
2.47 |
21 |
7.41 |
21 |
0.86 |
25 |
1.42 |
23 |
8.7 |
33 |
8.15 |
34 |
15.41 |
35 |
20.31 |
39 |
3.64 |
21 |
10.45 |
24 |
10.67 |
27 |
|
TreeDP |
34.83 |
1.99 |
31 |
2.84 |
26 |
9.96 |
29 |
1.23 |
36 |
1.92 |
30 |
7.55 |
25 |
10.77 |
40 |
19.09 |
40 |
20.51 |
43 |
7.62 |
40 |
15.99 |
40 |
15.62 |
38 |
|
TreeDP(subpixel) |
34.42 |
1.96 |
30 |
2.81 |
25 |
9.98 |
30 |
1.23 |
37 |
1.91 |
29 |
7.55 |
26 |
10.76 |
39 |
19.09 |
41 |
20.48 |
42 |
7.55 |
39 |
15.93 |
38 |
15.5 |
37 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|