Overall
comparison of algorithms with sub-pixel estimation using a single view |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
table below is the current, off-line version of Figure 3 of our paper. It
compares the performance of different stereo algorithms on four test |
|
|
|
|
|
|
|
image
pairs, including all the algorithms reported to the Middlebury website. The
numbers represent the percentage of "bad" pixels, i.e., pixel
whose |
|
|
|
|
|
|
absolute
disparity error is greater than [0.50 | 0.75 | 1.00 | 1.50 |2.00], the error
threshold can be changed by clicking on the tabs at the end of |
|
|
|
|
|
|
the
table. For each image pair, we report percentages for (1) non-occluded
pixels, (2) all pixels and (3) pixels near depth. All the scoring and
ranking |
|
|
|
|
|
|
schemes are the
same as the Middlebury benchmark. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
All
algorithms are run with constant parameter settings across all four images.
The rank of each algorithm in each column is indicated with a small red
number. |
|
|
|
|
The
algorithms are listed roughly in decreasing order of overall performance (as
measured by average rank). |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
difference between this benchmark and the Middlebury one is that we provide
sub-pixel estimation for all the algorithms reported to the Middlebury
website |
|
|
|
|
by
using only a single view (the reference view) or two views. The lines with
the blue highlighting are the experimental results with our sub-pixel
estimations. |
|
|
|
|
This
sub-pixel benchmark proves that the sub-pixel estimation algorithm proposed
is very robust; it works for all the algorithms. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Algorithms |
Avg. |
|
|
Tsukuba |
|
|
|
|
|
Venus |
|
|
|
|
|
Teddy |
|
|
|
|
|
Cones |
|
|
|
|
|
Rank |
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
|
AdaptingBP |
2.5 |
1.11 |
3 |
1.37 |
2 |
5.79 |
3 |
0.1 |
1 |
0.21 |
1 |
1.44 |
2 |
4.22 |
4 |
7.06 |
4 |
11.79 |
4 |
2.48 |
1 |
7.92 |
4 |
7.32 |
1 |
|
AdaptingBP(subpixel) |
2.92 |
1.18 |
4 |
1.52 |
3 |
6.26 |
6 |
0.1 |
2 |
0.21 |
2 |
1.46 |
3 |
3.96 |
3 |
6.83 |
3 |
10.99 |
3 |
2.55 |
2 |
7.85 |
2 |
7.48 |
2 |
|
AdaptWeight |
15.5 |
1.38 |
12 |
1.85 |
8 |
6.9 |
12 |
0.71 |
17 |
1.19 |
17 |
6.13 |
18 |
7.88 |
21 |
13.29 |
18 |
18.64 |
24 |
3.97 |
17 |
9.79 |
14 |
8.26 |
8 |
|
AdaptWeight(subpixel) |
15 |
1.63 |
21 |
2.17 |
17 |
7.76 |
17 |
0.52 |
14 |
0.88 |
14 |
4.31 |
15 |
7.32 |
18 |
12.46 |
15 |
17.9 |
20 |
3.73 |
13 |
9.45 |
11 |
7.95 |
5 |
|
CostRelax |
30.5 |
4.76 |
41 |
6.08 |
41 |
20.32 |
43 |
1.41 |
28 |
2.48 |
31 |
18.53 |
41 |
8.18 |
23 |
15.94 |
28 |
23.8 |
37 |
3.91 |
15 |
10.17 |
15 |
11.84 |
23 |
|
CostRelax(subpixel) |
29.83 |
4.86 |
42 |
6.19 |
42 |
20.6 |
44 |
1.14 |
23 |
2.15 |
28 |
14.76 |
37 |
8.13 |
22 |
15.91 |
27 |
23.69 |
36 |
3.94 |
16 |
10.27 |
17 |
11.93 |
24 |
|
C-SemiGlob |
13.58 |
2.61 |
34 |
3.29 |
24 |
9.89 |
28 |
0.25 |
11 |
0.57 |
10 |
3.24 |
13 |
5.14 |
8 |
11.79 |
12 |
12.96 |
8 |
2.77 |
4 |
8.35 |
5 |
8.2 |
6 |
|
C-SemiGlob(subpixel) |
12.67 |
2.25 |
30 |
2.87 |
21 |
9.18 |
25 |
0.23 |
10 |
0.46 |
9 |
3.16 |
12 |
5.13 |
7 |
11.71 |
11 |
12.75 |
7 |
2.8 |
5 |
8.38 |
6 |
8.29 |
9 |
|
DoubleBP |
4.33 |
0.88 |
1 |
1.29 |
1 |
4.76 |
1 |
0.13 |
5 |
0.45 |
8 |
1.87 |
5 |
3.53 |
2 |
8.3 |
6 |
9.63 |
2 |
2.9 |
7 |
8.78 |
10 |
7.79 |
4 |
|
DoubleBP(subpixel) |
4.92 |
1.37 |
11 |
1.86 |
9 |
6.18 |
4 |
0.11 |
4 |
0.37 |
7 |
1.47 |
4 |
3.34 |
1 |
8.18 |
5 |
9.39 |
1 |
2.67 |
3 |
8.42 |
7 |
7.77 |
3 |
|
DP |
41.08 |
4.12 |
39 |
5.04 |
39 |
11.95 |
34 |
10.1 |
50 |
11.03 |
50 |
21.03 |
45 |
14 |
38 |
21.58 |
38 |
20.56 |
32 |
10.54 |
43 |
19.1 |
42 |
21.1 |
43 |
|
DP(subpixel) |
38.5 |
4.19 |
40 |
5.11 |
40 |
12.34 |
38 |
7.68 |
47 |
8.62 |
47 |
19.62 |
42 |
10.48 |
33 |
18.28 |
37 |
18.64 |
25 |
8.76 |
38 |
17.47 |
36 |
18.94 |
39 |
|
GC |
35.83 |
1.94 |
24 |
4.12 |
33 |
9.39 |
26 |
1.79 |
36 |
3.44 |
39 |
8.75 |
27 |
16.47 |
46 |
25.03 |
48 |
24.93 |
41 |
7.7 |
36 |
18.15 |
39 |
15.27 |
35 |
|
GC(subpixel) |
34.5 |
1.93 |
23 |
4.11 |
32 |
9.39 |
27 |
1.54 |
32 |
3.1 |
37 |
8.73 |
26 |
16.26 |
45 |
24.82 |
46 |
24.67 |
40 |
7.57 |
35 |
17.98 |
38 |
15.03 |
33 |
|
GC+occ |
24.42 |
1.19 |
5 |
2.01 |
15 |
6.24 |
5 |
1.64 |
34 |
2.19 |
29 |
6.75 |
20 |
11.18 |
35 |
17.4 |
34 |
19.77 |
30 |
5.36 |
28 |
12.45 |
28 |
12.97 |
30 |
|
GC+occ(subpixel) |
23.83 |
1.2 |
6 |
2.04 |
16 |
6.3 |
7 |
1.53 |
31 |
2.04 |
26 |
6.72 |
19 |
11.12 |
34 |
17.26 |
33 |
19.93 |
31 |
5.14 |
27 |
12.24 |
27 |
12.64 |
29 |
|
GenModel |
30.58 |
2.57 |
33 |
4.74 |
37 |
13.03 |
42 |
1.72 |
35 |
3.08 |
36 |
16.93 |
39 |
6.86 |
16 |
14.99 |
25 |
19.21 |
27 |
4.64 |
23 |
14.89 |
34 |
11.38 |
20 |
|
GenModel(subpixel) |
28.83 |
2.52 |
32 |
4.69 |
35 |
12.77 |
41 |
1.55 |
33 |
2.87 |
34 |
15.59 |
38 |
6.64 |
14 |
14.69 |
24 |
18.93 |
26 |
4.31 |
19 |
14.41 |
32 |
10.91 |
18 |
|
Infection |
47.92 |
7.95 |
48 |
9.54 |
47 |
28.88 |
48 |
4.41 |
44 |
5.53 |
44 |
31.67 |
48 |
17.7 |
49 |
25.09 |
49 |
44.41 |
50 |
14.34 |
50 |
21.33 |
48 |
38.04 |
50 |
|
Infection(subpixel) |
46.92 |
7.95 |
49 |
9.55 |
48 |
29.32 |
50 |
3.44 |
42 |
4.55 |
41 |
29.96 |
47 |
16.8 |
48 |
24.27 |
44 |
42.79 |
49 |
13.55 |
49 |
20.47 |
47 |
36.5 |
49 |
|
Layered |
24.17 |
1.57 |
20 |
1.87 |
10 |
8.28 |
21 |
1.34 |
27 |
1.85 |
22 |
6.85 |
22 |
8.64 |
25 |
14.31 |
21 |
18.54 |
23 |
6.59 |
34 |
14.71 |
33 |
14.39 |
32 |
|
Layered(subpixel) |
21.25 |
1.63 |
22 |
1.99 |
12 |
8.62 |
22 |
0.69 |
16 |
1.13 |
16 |
5.65 |
16 |
7.86 |
20 |
13.49 |
19 |
17.59 |
19 |
6.13 |
31 |
14.2 |
31 |
13.58 |
31 |
|
MultiCamGC |
25.33 |
1.27 |
7 |
1.99 |
13 |
6.48 |
8 |
2.79 |
40 |
3.13 |
38 |
3.6 |
14 |
11.99 |
37 |
17.63 |
36 |
21.99 |
34 |
4.89 |
26 |
11.83 |
25 |
12.09 |
26 |
|
MultiCamGC(subpixel) |
24.33 |
1.28 |
8 |
2 |
14 |
6.51 |
9 |
2.44 |
39 |
2.69 |
33 |
2.93 |
11 |
11.9 |
36 |
17.48 |
35 |
21.87 |
33 |
4.79 |
25 |
11.74 |
24 |
11.98 |
25 |
|
RealtimeBP |
21.83 |
1.49 |
15 |
3.4 |
27 |
7.87 |
18 |
0.77 |
18 |
1.9 |
24 |
9 |
28 |
8.72 |
26 |
13.24 |
17 |
17.17 |
16 |
4.61 |
22 |
11.64 |
23 |
12.44 |
28 |
|
RealtimeBP(subpixel) |
19.17 |
1.52 |
17 |
3.44 |
28 |
7.92 |
19 |
0.64 |
15 |
1.6 |
20 |
7.81 |
25 |
8.46 |
24 |
12.94 |
16 |
17.19 |
17 |
3.8 |
14 |
10.79 |
19 |
10.8 |
16 |
|
RealTimeGPU |
30.42 |
2.05 |
27 |
4.22 |
34 |
10.64 |
32 |
1.92 |
37 |
2.98 |
35 |
20.27 |
44 |
7.23 |
17 |
14.38 |
22 |
17.57 |
18 |
6.41 |
33 |
13.7 |
30 |
16.46 |
36 |
|
RealTimeGPU(subpixel) |
28.5 |
2.44 |
31 |
4.69 |
36 |
12.09 |
35 |
1.45 |
30 |
2.5 |
32 |
16.99 |
40 |
6.6 |
13 |
13.76 |
20 |
16.38 |
13 |
5.58 |
29 |
12.75 |
29 |
15.05 |
34 |
|
RegionTreeDP |
15.08 |
1.39 |
13 |
1.64 |
5 |
6.85 |
11 |
0.22 |
9 |
0.57 |
11 |
1.93 |
6 |
7.42 |
19 |
11.91 |
13 |
16.75 |
14 |
6.31 |
32 |
11.9 |
26 |
11.84 |
22 |
|
RegionTreeDP(subpixel) |
13.5 |
1.57 |
19 |
1.89 |
11 |
7.44 |
16 |
0.1 |
3 |
0.35 |
6 |
1.2 |
1 |
6.76 |
15 |
11.26 |
9 |
16.35 |
12 |
5.66 |
30 |
11.24 |
21 |
10.99 |
19 |
|
ReliabilityDP |
32.58 |
1.36 |
10 |
3.39 |
26 |
7.25 |
15 |
2.35 |
38 |
3.48 |
40 |
12.22 |
36 |
9.82 |
32 |
16.87 |
31 |
19.54 |
29 |
12.88 |
47 |
19.94 |
46 |
19.71 |
41 |
|
ReliabilityDP(subpixel) |
30.25 |
1.55 |
18 |
3.66 |
29 |
8.25 |
20 |
1.33 |
26 |
2.37 |
30 |
9.14 |
29 |
9.39 |
28 |
16.45 |
29 |
19.43 |
28 |
12.23 |
46 |
19.32 |
43 |
18.89 |
37 |
|
Segm+visib |
10.83 |
1.3 |
9 |
1.57 |
4 |
6.92 |
13 |
0.79 |
19 |
1.06 |
15 |
6.76 |
21 |
5 |
6 |
6.54 |
2 |
12.33 |
6 |
3.72 |
12 |
8.62 |
9 |
10.23 |
14 |
|
Segm+visib(subpixel) |
10.08 |
1.48 |
14 |
1.75 |
7 |
7.21 |
14 |
0.52 |
13 |
0.71 |
13 |
5.81 |
17 |
4.71 |
5 |
6.38 |
1 |
11.92 |
5 |
3.51 |
11 |
8.45 |
8 |
10 |
13 |
|
SegTreeDP |
17.92 |
2.21 |
29 |
2.76 |
19 |
10.32 |
31 |
0.46 |
12 |
0.6 |
12 |
2.44 |
10 |
9.58 |
30 |
15.17 |
26 |
18.43 |
22 |
3.23 |
10 |
7.86 |
3 |
8.83 |
11 |
|
SegTreeDP(subpixel) |
16.33 |
2.69 |
35 |
3.31 |
25 |
11.82 |
33 |
0.17 |
8 |
0.28 |
3 |
1.95 |
7 |
9.05 |
27 |
14.66 |
23 |
18.38 |
21 |
2.89 |
6 |
7.46 |
1 |
8.22 |
7 |
|
SemiGlob |
20.92 |
3.26 |
37 |
3.96 |
31 |
12.75 |
40 |
1 |
21 |
1.57 |
19 |
11.27 |
33 |
6.02 |
11 |
12.19 |
14 |
16.32 |
11 |
3.06 |
9 |
9.75 |
13 |
8.9 |
12 |
|
SemiGlob(subpixel) |
19.25 |
3.03 |
36 |
3.75 |
30 |
12.22 |
37 |
0.89 |
20 |
1.39 |
18 |
9.22 |
30 |
5.79 |
10 |
11.7 |
10 |
15.92 |
10 |
2.98 |
8 |
9.52 |
12 |
8.72 |
10 |
|
SO |
45.92 |
5.08 |
43 |
7.22 |
44 |
12.18 |
36 |
9.44 |
49 |
10.94 |
49 |
21.92 |
46 |
19.95 |
50 |
28.17 |
50 |
26.26 |
42 |
13.01 |
48 |
22.85 |
50 |
22.25 |
44 |
|
SO(subpixel) |
44 |
5.24 |
45 |
7.4 |
45 |
12.51 |
39 |
6.64 |
46 |
8.13 |
46 |
20.06 |
43 |
16.15 |
44 |
24.75 |
45 |
24.39 |
39 |
11.41 |
45 |
21.38 |
49 |
20.45 |
42 |
|
SSD+MF |
44 |
5.23 |
44 |
7.07 |
43 |
24.14 |
45 |
3.74 |
43 |
5.16 |
43 |
11.89 |
35 |
16.53 |
47 |
24.85 |
47 |
32.91 |
46 |
10.6 |
44 |
19.82 |
45 |
26.26 |
46 |
|
SSD+MF(subpixel) |
43.08 |
5.74 |
46 |
7.66 |
46 |
25.94 |
46 |
3.43 |
41 |
4.69 |
42 |
10.73 |
32 |
16.01 |
43 |
24.2 |
43 |
32.78 |
45 |
10.47 |
42 |
19.64 |
44 |
26.52 |
47 |
|
STICA |
45.17 |
7.7 |
47 |
9.63 |
49 |
27.83 |
47 |
8.19 |
48 |
9.58 |
48 |
40.3 |
50 |
15.83 |
41 |
23.22 |
40 |
37.74 |
48 |
9.81 |
39 |
17.8 |
37 |
28.7 |
48 |
|
STICA(subpixel) |
44.17 |
8.28 |
50 |
10.25 |
50 |
29.32 |
49 |
6.41 |
45 |
7.8 |
45 |
35.49 |
49 |
14.48 |
39 |
21.94 |
39 |
35.34 |
47 |
8.58 |
37 |
16.61 |
35 |
25.56 |
45 |
|
SymBP+occ |
10.42 |
0.97 |
2 |
1.75 |
6 |
5.09 |
2 |
0.16 |
7 |
0.33 |
5 |
2.19 |
9 |
6.47 |
12 |
10.72 |
8 |
17.03 |
15 |
4.79 |
24 |
10.72 |
18 |
10.88 |
17 |
|
SymBP+occ(subpixel) |
11.58 |
1.51 |
16 |
2.43 |
18 |
6.8 |
10 |
0.15 |
6 |
0.31 |
4 |
2.11 |
8 |
5.76 |
9 |
9.92 |
7 |
15.6 |
9 |
4.43 |
21 |
10.22 |
16 |
10.31 |
15 |
|
TensorVoting |
29.17 |
3.79 |
38 |
4.79 |
38 |
8.86 |
23 |
1.23 |
24 |
1.88 |
23 |
11.53 |
34 |
9.76 |
31 |
17.05 |
32 |
24.03 |
38 |
4.38 |
20 |
11.44 |
22 |
12.23 |
27 |
|
TensorVoting(subpixel) |
25.17 |
2.15 |
28 |
3.21 |
23 |
8.94 |
24 |
1 |
22 |
1.65 |
21 |
10.44 |
31 |
9.41 |
29 |
16.71 |
30 |
23.49 |
35 |
4.07 |
18 |
11.15 |
20 |
11.8 |
21 |
|
TreeDP |
33.5 |
1.99 |
25 |
2.84 |
20 |
9.96 |
29 |
1.41 |
29 |
2.1 |
27 |
7.74 |
24 |
15.92 |
42 |
23.9 |
42 |
27.09 |
44 |
10.05 |
41 |
18.35 |
41 |
18.92 |
38 |
|
TreeDP(subpixel) |
32.92 |
2.02 |
26 |
2.87 |
22 |
10.13 |
30 |
1.29 |
25 |
1.99 |
25 |
7.55 |
23 |
15.58 |
40 |
23.6 |
41 |
26.82 |
43 |
9.95 |
40 |
18.22 |
40 |
19.08 |
40 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|