Leaderboard

Evaluation Metrics

  • AUPR : pixel-wise Area Under Precision Recall curve
  • FPR95 : pixel-wise False Positive Rate at a true positive rate of 95%
  • sIoU gt : adjusted Intersection over Union averaged over all ground truth segmentation components
  • PPV : predictive positive value (or precision) averaged over all predicted segmentation components
  • mean F1 : component-wise F1-score averaged over different detection thresholds
For a more detailed explanation of the metrics, we refer to our paper.



Anomaly Track

Method
OoD Data
Pixel Level
AUPR
FPR95
Component Level
sIoU gt
PPV
mean F1
SynBoost [paper] [code]
56.44%
61.86%
34.68%
17.81%
9.99%
Embedding Density [paper]
37.52%
70.76%
33.86%
20.54%
7.90%
Void Classifier [paper]
36.61%
63.49%
21.14%
22.13%
6.49%
MC Dropout [paper]
28.87%
69.47%
20.49%
17.26%
4.26%
Maximized Entropy [paper] [code]
85.47%
15.00%
49.21%
39.51%
28.72%
ODIN [paper]
33.06%
71.68%
19.53%
17.88%
5.15%
Maximum Softmax [paper]
27.97%
72.05%
15.48%
15.29%
5.37%
Mahalanobis [paper]
20.04%
86.99%
14.82%
10.22%
2.68%
Image Resynthesis [paper] [code]
52.28%
25.93%
39.68%
10.95%
12.51%
NFlowJS [paper]
56.92%
34.71%
36.94%
18.01%
14.89%
JSRNet [paper] [code]
33.64%
43.85%
20.20%
29.27%
13.66%
Ensemble [paper]
17.66%
91.06%
16.44%
20.77%
3.39%
DenseHybrid [paper] [code]
77.96%
9.81%
54.17%
24.13%
31.08%
ObsNet [paper] [code]
75.44%
26.69%
44.22%
52.56%
45.08%
PEBAL [paper] [code]
49.14%
40.82%
38.88%
27.20%
14.48%
DenseHybrid (DeepLabv3+)
42.05%
62.25%
36.90%
18.70%
11.32%
cDNP [paper] [code]
88.90%
11.42%
50.44%
29.04%
28.12%
EAM [paper]
93.75%
4.09%
67.09%
53.77%
60.86%
RbA [paper]
94.46%
4.60%
64.93%
47.51%
51.87%
RbA [paper]
86.13%
15.94%
56.26%
41.35%
42.04%
RPL+CoroCL [paper] [code]
83.49%
11.68%
49.77%
29.96%
30.16%
FlowEneDet [paper] [code]
36.74%
77.82%
15.47%
16.83%
3.40%
CSL [paper] [code]
80.08%
7.16%
46.46%
50.02%
50.39%
Maskomaly [paper] [code]
93.35%
6.87%
55.43%
51.46%
49.90%
ContMAV [paper] [code]
90.20%
3.83%
54.55%
61.86%
63.64%
Mask2Anomaly [paper] [code]
88.72%
14.63%
55.28%
51.68%
47.16%
ATTA [paper] [code]
67.04%
31.57%
44.58%
29.55%
20.64%
UNO [paper]
96.33%
1.98%
68.51%
55.77%
62.61%
UNO [paper]
96.10%
2.27%
68.01%
51.86%
58.87%
VLAD
92.94%
3.25%
71.58%
53.71%
65.40%
VLAD
82.48%
82.42%
66.99%
51.86%
61.14%
PixOOD [paper] [code]
68.88%
54.33%
44.15%
24.32%
19.82%
PGN [paper] [code]
36.70%
61.41%
21.59%
17.48%
6.21%
Con2MAV
90.00%
2.68%
59.10%
68.32%
69.38%
OodDINO+RPL
87.33%
7.83%
48.06%
52.41%
56.09%
OodDINO+RbA
85.64%
7.79%
46.21%
55.24%
54.90%
FlowCLAS [paper]
91.32%
17.12%
49.22%
44.96%
46.48%

Obstacle Track

Method
OoD Data
Pixel Level
AUPR
FPR95
Component Level
sIoU gt
PPV
mean F1
SynBoost [paper] [code]
71.34%
3.15%
44.28%
41.75%
37.57%
Void Classifier [paper]
10.44%
41.54%
6.34%
20.27%
5.41%
MC Dropout [paper]
4.88%
50.31%
5.49%
5.77%
1.05%
Embedding Density [paper]
0.82%
46.38%
35.64%
2.87%
2.31%
Maximized Entropy [paper] [code]
85.07%
0.75%
47.87%
62.64%
48.51%
ODIN [paper]
22.12%
15.28%
21.62%
18.50%
9.37%
Mahalanobis [paper]
20.90%
13.08%
13.52%
21.79%
4.70%
Maximum Softmax [paper]
15.72%
16.60%
19.72%
15.93%
6.25%
Road Inpainting [paper]
54.14%
47.12%
57.64%
39.50%
36.01%
Image Resynthesis [paper] [code]
37.71%
4.70%
16.61%
20.48%
8.38%
NFlowJS [paper]
85.55%
0.41%
45.53%
49.53%
50.36%
JSRNet [paper] [code]
28.09%
28.86%
18.55%
24.46%
11.02%
Ensemble [paper]
1.06%
77.20%
8.63%
4.71%
1.28%
DenseHybrid [paper] [code]
87.08%
0.24%
45.74%
50.10%
50.72%
RAOS (training)
87.40%
0.28%
46.75%
46.60%
47.57%
RAOS (training free)
79.67%
0.81%
42.42%
32.58%
32.70%
ObsFormer [code]
24.75%
39.59%
20.32%
39.82%
19.98%
DenseHybrid (DeepLabv3+)
80.79%
6.02%
48.48%
60.16%
55.59%
PEBAL [paper] [code]
4.98%
12.68%
29.91%
7.55%
5.54%
EAM [paper]
92.87%
0.52%
65.86%
76.50%
75.58%
DaCUP [paper] [code]
81.50%
1.13%
37.68%
60.13%
46.01%
RbA [paper]
95.12%
0.08%
54.34%
59.08%
57.44%
RbA [paper]
87.85%
3.33%
47.44%
56.16%
50.42%
RPL+CoroCL [paper] [code]
85.93%
0.58%
52.62%
56.65%
56.69%
CSL [paper] [code]
87.10%
0.67%
44.70%
53.13%
51.02%
FlowEneDet [paper] [code]
73.71%
0.97%
42.62%
42.25%
39.96%
Mask2Anomaly [paper] [code]
93.22%
0.20%
55.72%
75.42%
68.15%
ATTA [paper] [code]
76.46%
2.81%
43.93%
37.66%
36.57%
UNO [paper]
93.19%
0.16%
70.97%
72.17%
77.65%
UNO [paper]
88.97%
0.61%
66.87%
74.86%
76.32%
VLAD
78.70%
0.57%
36.42%
22.40%
24.36%
VLAD
76.43%
0.58%
42.47%
19.05%
23.63%
PixOOD [paper] [code]
88.90%
0.30%
42.68%
57.49%
50.82%
PGN [paper] [code]
16.52%
19.69%
19.42%
14.89%
7.39%
OodDINO+RbA
94.52%
0.05%
73.01%
80.44%
89.91%
OodDINO+RPL
94.05%
0.06%
67.72%
81.36%
86.53%
FlowCLAS [paper]
87.29%
0.42%
31.74%
63.05%
48.83%

LostAndFound NoKnown

Method
OoD Data
Pixel Level
AUPR
FPR95
Component Level
sIoU gt
PPV
mean F1
SynBoost [paper] [code]
81.71%
4.64%
36.83%
72.32%
48.72%
Void Classifier [paper]
4.81%
47.02%
1.76%
35.08%
1.87%
MC Dropout [paper]
36.78%
35.55%
17.35%
34.71%
12.99%
Embedding Density [paper]
61.70%
10.36%
37.75%
35.21%
27.55%
Maximized Entropy [paper] [code]
77.90%
9.70%
45.90%
63.06%
49.92%
ODIN [paper]
52.93%
30.04%
39.79%
49.33%
34.53%
Mahalanobis [paper]
54.97%
12.89%
33.83%
31.71%
22.09%
Maximum Softmax [paper]
30.14%
33.20%
14.20%
62.23%
10.32%
Road Inpainting [paper]
82.93%
35.75%
49.21%
60.67%
52.25%
Image Resynthesis [paper] [code]
57.08%
8.82%
27.16%
30.69%
19.17%
NFlowJS [paper]
89.28%
0.65%
54.63%
59.74%
61.75%
JSRNet [paper] [code]
74.17%
6.59%
34.28%
45.89%
35.97%
Ensemble [paper]
2.89%
82.03%
6.66%
7.64%
2.68%
DenseHybrid [paper] [code]
78.67%
2.12%
46.90%
52.14%
52.33%
RAOS (training)
89.81%
0.95%
53.06%
57.86%
60.21%
RAOS (training free)
81.13%
3.23%
53.54%
46.20%
50.64%
DaCUP [paper] [code]
81.37%
7.36%
38.34%
67.29%
51.14%
FlowEneDet [paper] [code]
79.75%
2.92%
43.79%
52.83%
48.05%
PixOOD [paper] [code]
85.07%
4.46%
30.18%
78.47%
44.41%


For a description of the test datasets, we refer to our paper.