Overall
comparison of algorithms with sub-pixel estimation using a single view |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
table below is the current, off-line version of Figure 3 of our paper. It
compares the performance of different stereo algorithms on four test |
|
|
|
|
|
|
|
image
pairs, including all the algorithms reported to the Middlebury website. The
numbers represent the percentage of "bad" pixels, i.e., pixel
whose |
|
|
|
|
|
|
absolute
disparity error is greater than [0.50 | 0.75 | 1.00 | 1.50 |2.00], the error
threshold can be changed by clicking on the tabs at the end of |
|
|
|
|
|
|
the
table. For each image pair, we report percentages for (1) non-occluded
pixels, (2) all pixels and (3) pixels near depth. All the scoring and
ranking |
|
|
|
|
|
|
schemes are the
same as the Middlebury benchmark. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
All
algorithms are run with constant parameter settings across all four images.
The rank of each algorithm in each column is indicated with a small red
number. |
|
|
|
|
The
algorithms are listed roughly in decreasing order of overall performance (as
measured by average rank). |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The
difference between this benchmark and the Middlebury one is that we provide
sub-pixel estimation for all the algorithms reported to the Middlebury
website |
|
|
|
|
by
using only a single view (the reference view) or two views. The lines with
the blue highlighting are the experimental results with our sub-pixel
estimations. |
|
|
|
|
This
sub-pixel benchmark proves that the sub-pixel estimation algorithm proposed
is very robust; it works for all the algorithms. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Algorithms |
Avg. |
|
|
Tsukuba |
|
|
|
|
|
Venus |
|
|
|
|
|
Teddy |
|
|
|
|
|
Cones |
|
|
|
|
|
Rank |
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
nonocc |
|
all |
|
disc |
|
|
C-SemiGlob(subpixel) |
5.25 |
12.16 |
11 |
12.82 |
12 |
16.27 |
9 |
2.9 |
3 |
3.38 |
2 |
9.47 |
7 |
10.32 |
3 |
17.9 |
7 |
22.93 |
3 |
5.55 |
4 |
11.51 |
1 |
12.45 |
1 |
|
C-SemiGlob |
7.75 |
13.92 |
17 |
14.71 |
16 |
18.88 |
25 |
3.3 |
6 |
3.82 |
4 |
10.95 |
10 |
9.82 |
1 |
17.43 |
5 |
22.76 |
2 |
5.37 |
3 |
11.69 |
2 |
12.77 |
2 |
|
AdaptingBP(subpixel) |
9.17 |
17.41 |
20 |
17.73 |
20 |
17.01 |
11 |
4.46 |
13 |
4.71 |
10 |
7.38 |
2 |
12.74 |
9 |
16.52 |
3 |
25.77 |
7 |
6.04 |
7 |
12.08 |
4 |
13.35 |
4 |
|
SemiGlob(subpixel) |
9.83 |
10.6 |
9 |
11.4 |
9 |
18.87 |
24 |
4.19 |
11 |
4.94 |
13 |
14.43 |
16 |
11.64 |
6 |
19.14 |
10 |
26.54 |
10 |
5.11 |
2 |
12.09 |
5 |
13.29 |
3 |
|
Segm+visib(subpixel) |
11.17 |
12.46 |
12 |
12.71 |
11 |
15.82 |
7 |
6.99 |
24 |
7.75 |
23 |
17.45 |
23 |
10.3 |
2 |
12.9 |
1 |
22.42 |
1 |
7.44 |
12 |
12.5 |
6 |
16.49 |
12 |
|
DoubleBP(subpixel) |
11.33 |
17.69 |
21 |
18.08 |
21 |
16.04 |
8 |
4.46 |
14 |
4.8 |
11 |
8.25 |
5 |
11.9 |
7 |
17.49 |
6 |
23.9 |
5 |
8.08 |
14 |
14.1 |
13 |
16.49 |
11 |
|
SemiGlob |
12 |
13.42 |
15 |
14.3 |
15 |
20.35 |
29 |
4.55 |
15 |
5.38 |
16 |
15.7 |
19 |
10.96 |
5 |
18.48 |
8 |
26.07 |
8 |
4.93 |
1 |
12.54 |
7 |
13.51 |
6 |
|
AdaptingBP |
12.58 |
19.06 |
27 |
19.34 |
24 |
17.35 |
13 |
4.84 |
18 |
5.08 |
15 |
7.84 |
3 |
12.76 |
10 |
16.67 |
4 |
26.31 |
9 |
7.02 |
10 |
13.15 |
11 |
13.99 |
7 |
|
Segm+visib |
14.67 |
12.69 |
13 |
12.93 |
13 |
15.81 |
6 |
10.4 |
34 |
11.01 |
33 |
19.49 |
27 |
10.96 |
4 |
13.2 |
2 |
23.7 |
4 |
8.12 |
15 |
13.11 |
10 |
17.29 |
15 |
|
SymBP+occ(subpixel) |
14.92 |
20.55 |
35 |
21.16 |
33 |
18.59 |
20 |
4.1 |
10 |
4.36 |
9 |
7.35 |
1 |
13.2 |
13 |
18.63 |
9 |
29.09 |
13 |
7.92 |
13 |
14.3 |
14 |
16.05 |
9 |
|
AdaptWeight(subpixel) |
14.92 |
16.12 |
19 |
16.55 |
19 |
17.1 |
12 |
4.34 |
12 |
4.91 |
12 |
12.62 |
12 |
14.29 |
16 |
20.8 |
15 |
30.82 |
17 |
9.3 |
19 |
15.11 |
16 |
16.42 |
10 |
|
TensorVoting(subpixel) |
16.33 |
24.47 |
45 |
25.14 |
43 |
21.12 |
31 |
1.52 |
1 |
2.3 |
1 |
12.84 |
13 |
13.11 |
12 |
20.41 |
14 |
30.5 |
15 |
5.58 |
5 |
12.96 |
8 |
14.87 |
8 |
|
SegTreeDP(subpixel) |
17.42 |
24.31 |
43 |
24.78 |
41 |
23.02 |
39 |
3.2 |
5 |
3.51 |
3 |
8 |
4 |
16.6 |
23 |
23.12 |
23 |
29.35 |
14 |
5.97 |
6 |
11.77 |
3 |
13.44 |
5 |
|
GenModel(subpixel) |
17.58 |
9.21 |
8 |
11.31 |
8 |
19.97 |
28 |
3.59 |
9 |
4.99 |
14 |
22.25 |
37 |
14.17 |
15 |
22.11 |
20 |
30.64 |
16 |
9.09 |
17 |
19.02 |
26 |
16.51 |
13 |
|
RegionTreeDP(subpixel) |
18.17 |
19.78 |
32 |
19.88 |
27 |
18.37 |
18 |
3.42 |
8 |
3.97 |
6 |
8.26 |
6 |
14.52 |
18 |
20.17 |
12 |
28.59 |
12 |
13.01 |
30 |
18.53 |
24 |
19.84 |
25 |
|
DoubleBP |
18.42 |
18.66 |
25 |
19.11 |
23 |
15.76 |
5 |
7.82 |
27 |
8.22 |
25 |
11.32 |
11 |
14.4 |
17 |
19.88 |
11 |
24.35 |
6 |
11.77 |
26 |
17.65 |
22 |
19.67 |
23 |
|
GC+occ(subpixel) |
19.08 |
6.63 |
3 |
7.53 |
2 |
14.39 |
1 |
7.75 |
25 |
8.39 |
26 |
14.34 |
15 |
21.43 |
32 |
28.14 |
32 |
33.69 |
29 |
9.83 |
20 |
16.97 |
20 |
19.75 |
24 |
|
GenModel |
20.25 |
7.89 |
6 |
10.01 |
7 |
18.52 |
19 |
4.59 |
16 |
6.03 |
18 |
23.48 |
40 |
14.76 |
20 |
22.75 |
22 |
31.75 |
22 |
10.2 |
21 |
20.2 |
31 |
19.03 |
21 |
|
CostRelax(subpixel) |
21.17 |
24.03 |
41 |
25.05 |
42 |
31.6 |
43 |
2.82 |
2 |
3.92 |
5 |
20.67 |
29 |
12.94 |
11 |
20.86 |
16 |
33.08 |
27 |
7 |
9 |
13.67 |
12 |
17.7 |
17 |
|
CostRelax |
21.58 |
26.25 |
48 |
27.33 |
48 |
33.46 |
44 |
2.92 |
4 |
4.06 |
7 |
20.8 |
31 |
12.28 |
8 |
20.24 |
13 |
32.4 |
25 |
6.33 |
8 |
13.09 |
9 |
16.72 |
14 |
|
RealtimeBP(subpixel) |
21.92 |
19.45 |
29 |
21.04 |
31 |
21.69 |
33 |
4.72 |
17 |
6 |
17 |
15.6 |
18 |
16.33 |
22 |
22.12 |
21 |
31.32 |
19 |
9.01 |
16 |
16.44 |
18 |
19.27 |
22 |
|
TensorVoting |
22.08 |
25.46 |
47 |
26.19 |
47 |
21.23 |
32 |
3.32 |
7 |
4.12 |
8 |
14.58 |
17 |
14.61 |
19 |
21.78 |
18 |
33.26 |
28 |
7.05 |
11 |
14.46 |
15 |
17.37 |
16 |
|
SymBP+occ |
22.42 |
20.74 |
36 |
21.57 |
35 |
19.51 |
27 |
5.96 |
20 |
6.27 |
19 |
10.19 |
8 |
15.66 |
21 |
20.92 |
17 |
31.71 |
21 |
11.4 |
24 |
17.54 |
21 |
18.94 |
20 |
|
MultiCamGC(subpixel) |
23.08 |
6.98 |
4 |
7.86 |
4 |
15.43 |
3 |
13.87 |
45 |
14.39 |
42 |
16.61 |
21 |
22.17 |
35 |
28.51 |
34 |
35.74 |
35 |
9.12 |
18 |
16.06 |
17 |
18.65 |
19 |
|
GC+occ |
24 |
6.1 |
1 |
7.11 |
1 |
14.55 |
2 |
10.71 |
36 |
11.33 |
34 |
16.91 |
22 |
23.68 |
37 |
30.09 |
37 |
34.56 |
33 |
12.25 |
29 |
19.2 |
27 |
21.9 |
29 |
|
RealTimeGPU(subpixel) |
24.67 |
20.24 |
34 |
22.05 |
38 |
21.7 |
34 |
5.71 |
19 |
6.89 |
21 |
21.6 |
33 |
14.03 |
14 |
22.05 |
19 |
27.63 |
11 |
10.23 |
22 |
18.04 |
23 |
21.33 |
28 |
|
Layered(subpixel) |
25 |
11.82 |
10 |
12.32 |
10 |
19.23 |
26 |
9.87 |
32 |
10.65 |
32 |
20.73 |
30 |
16.88 |
24 |
23.17 |
24 |
31.78 |
23 |
12.15 |
28 |
20.13 |
30 |
23.1 |
31 |
|
AdaptWeight |
25.58 |
18.1 |
24 |
18.8 |
22 |
18.59 |
21 |
7.77 |
26 |
8.4 |
27 |
15.83 |
20 |
17.57 |
25 |
23.93 |
25 |
34.03 |
31 |
14 |
32 |
19.67 |
28 |
20.63 |
26 |
|
SegTreeDP |
26.33 |
25.42 |
46 |
25.96 |
44 |
24.6 |
41 |
6.23 |
21 |
6.59 |
20 |
10.79 |
9 |
19.71 |
29 |
25.87 |
28 |
30.86 |
18 |
10.86 |
23 |
16.48 |
19 |
17.73 |
18 |
|
MultiCamGC |
27 |
6.56 |
2 |
7.55 |
3 |
15.68 |
4 |
16.4 |
48 |
16.83 |
48 |
17.57 |
25 |
24.33 |
39 |
30.43 |
38 |
36.94 |
38 |
12.04 |
27 |
18.76 |
25 |
21.15 |
27 |
|
GC(subpixel) |
29.25 |
7.93 |
7 |
9.97 |
6 |
17.98 |
16 |
10.01 |
33 |
11.53 |
35 |
18.6 |
26 |
29.98 |
46 |
37.22 |
47 |
38.38 |
40 |
13.27 |
31 |
23.13 |
34 |
23.07 |
30 |
|
ReliabilityDP(subpixel) |
29.5 |
17.9 |
23 |
19.61 |
25 |
16.68 |
10 |
6.62 |
23 |
7.96 |
24 |
21.89 |
36 |
21.83 |
33 |
28.55 |
35 |
34.12 |
32 |
17.6 |
40 |
24.43 |
37 |
26.07 |
36 |
|
RegionTreeDP |
29.75 |
20.95 |
37 |
21.12 |
32 |
18.33 |
17 |
9.08 |
31 |
9.74 |
30 |
13.79 |
14 |
19.72 |
30 |
24.76 |
26 |
32.12 |
24 |
19.74 |
44 |
24.76 |
39 |
25.36 |
33 |
|
DP(subpixel) |
31.17 |
14.17 |
18 |
15.14 |
17 |
18.68 |
23 |
13.42 |
43 |
14.4 |
43 |
28.15 |
43 |
21.09 |
31 |
28.18 |
33 |
31.65 |
20 |
14.12 |
33 |
22.46 |
33 |
26.36 |
37 |
|
RealtimeBP |
31.42 |
19.94 |
33 |
21.62 |
36 |
22.16 |
35 |
8.68 |
30 |
9.93 |
31 |
20.12 |
28 |
19.24 |
26 |
24.83 |
27 |
33.82 |
30 |
14.22 |
34 |
21.25 |
32 |
25.9 |
35 |
|
Layered |
32.25 |
12.95 |
14 |
13.51 |
14 |
18.67 |
22 |
15.6 |
47 |
16.33 |
46 |
21.78 |
35 |
22.16 |
34 |
27.83 |
31 |
35.38 |
34 |
16.91 |
38 |
24.53 |
38 |
25.83 |
34 |
|
GC |
32.42 |
7.71 |
5 |
9.82 |
5 |
17.4 |
14 |
13.48 |
44 |
14.96 |
45 |
20.91 |
32 |
31.45 |
48 |
38.52 |
48 |
38.88 |
41 |
15.95 |
35 |
25.54 |
40 |
24.68 |
32 |
|
STICA(subpixel) |
35.58 |
19.61 |
31 |
21.43 |
34 |
37.15 |
45 |
10.7 |
35 |
12.1 |
36 |
40.24 |
49 |
19.45 |
27 |
27.31 |
30 |
44.85 |
43 |
11.7 |
25 |
19.77 |
29 |
32.34 |
43 |
|
RealTimeGPU |
36.58 |
24.21 |
42 |
26.02 |
45 |
24.94 |
42 |
10.95 |
37 |
12.12 |
37 |
27.58 |
42 |
19.62 |
28 |
27.01 |
29 |
33 |
26 |
16.55 |
37 |
23.75 |
35 |
29.52 |
39 |
|
TreeDP(subpixel) |
36.67 |
21.22 |
38 |
21.86 |
37 |
22.62 |
37 |
6.23 |
22 |
7.1 |
22 |
17.52 |
24 |
29.57 |
45 |
36.43 |
46 |
44.89 |
44 |
18.68 |
43 |
26.34 |
42 |
30.08 |
40 |
|
SO(subpixel) |
36.75 |
13.47 |
16 |
15.47 |
18 |
20.87 |
30 |
13.39 |
42 |
14.84 |
44 |
29.11 |
44 |
28.37 |
44 |
35.8 |
44 |
37.8 |
39 |
17.12 |
39 |
26.56 |
43 |
28.71 |
38 |
|
ReliabilityDP |
37.5 |
18.99 |
26 |
20.73 |
30 |
17.54 |
15 |
12.72 |
40 |
13.96 |
40 |
26.13 |
41 |
26.32 |
42 |
32.49 |
42 |
36.76 |
37 |
23.69 |
48 |
29.89 |
48 |
31.48 |
41 |
|
Infection(subpixel) |
38.08 |
19.27 |
28 |
20.65 |
29 |
37.4 |
46 |
7.84 |
28 |
9.06 |
28 |
33.65 |
46 |
22.2 |
36 |
29.81 |
36 |
53.32 |
49 |
17.93 |
41 |
25.72 |
41 |
44.4 |
49 |
|
SSD+MF(subpixel) |
41.75 |
26.35 |
49 |
27.83 |
49 |
37.61 |
47 |
7.9 |
29 |
9.28 |
29 |
22.74 |
38 |
24.52 |
41 |
32.13 |
41 |
46.96 |
46 |
18.02 |
42 |
26.58 |
44 |
37.85 |
46 |
|
DP |
42.17 |
19.58 |
30 |
20.59 |
28 |
22.84 |
38 |
23.49 |
50 |
24.3 |
49 |
32.85 |
45 |
30.04 |
47 |
36.28 |
45 |
36.09 |
36 |
22.04 |
47 |
29.64 |
47 |
33.75 |
44 |
|
TreeDP |
42.5 |
22.42 |
40 |
23.05 |
39 |
22.33 |
36 |
12.08 |
39 |
12.92 |
39 |
21.72 |
34 |
32.39 |
49 |
38.95 |
49 |
45.59 |
45 |
23.7 |
49 |
30.77 |
49 |
31.74 |
42 |
|
SO |
43.42 |
17.85 |
22 |
19.81 |
26 |
23.37 |
40 |
23.14 |
49 |
24.44 |
50 |
34.59 |
47 |
36.3 |
50 |
42.87 |
50 |
41.41 |
42 |
23.71 |
50 |
32.4 |
50 |
34.42 |
45 |
|
Infection |
43.42 |
22.13 |
39 |
23.49 |
40 |
37.63 |
48 |
11.13 |
38 |
12.33 |
38 |
37.32 |
48 |
24.34 |
40 |
31.71 |
40 |
56.37 |
50 |
20.16 |
45 |
27.8 |
45 |
48.17 |
50 |
|
STICA |
44 |
24.33 |
44 |
26.1 |
46 |
44.76 |
50 |
15.24 |
46 |
16.55 |
47 |
46.35 |
50 |
23.97 |
38 |
31.28 |
39 |
50.41 |
48 |
15.95 |
36 |
23.75 |
36 |
39.5 |
48 |
|
SSD+MF |
45.17 |
28.53 |
50 |
29.99 |
50 |
38.06 |
49 |
12.83 |
41 |
14.2 |
41 |
23.41 |
39 |
28.16 |
43 |
35.42 |
43 |
48.65 |
47 |
21.08 |
46 |
29.42 |
46 |
38.61 |
47 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|