Hello：
There is a obvious difference between the actual evaluation results of the algorithm and those described in the paper, the cause of the problem has not been found yet, please give me some suggestions.
1.Evaluation results for the model provided in github of SamsungLabs/fcaf3d：(12 epoches)
1.1 sunrgbd
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/5050, elapsed: [>>>>>>>>>>>>>>>>] 5050/5050, 15.9 task/s, elapsed: 318s, ETA: 0s
+-------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed | 0.0496 | 0.7612 | 0.0094 | 0.5883 |
| table | 0.1570 | 0.8514 | 0.1222 | 0.7909 |
| sofa | 0.2261 | 0.8469 | 0.1440 | 0.7735 |
| chair | 0.1985 | 0.7662 | 0.1671 | 0.7272 |
| toilet | 0.3586 | 0.9379 | 0.2778 | 0.8552 |
| desk | 0.0970 | 0.7641 | 0.0665 | 0.6982 |
| dresser | 0.2063 | 0.9128 | 0.1956 | 0.8670 |
| night_stand | 0.4174 | 0.9608 | 0.4127 | 0.9451 |
| bookshelf | 0.1585 | 0.8865 | 0.1489 | 0.8511 |
| bathtub | 0.3658 | 0.9592 | 0.3561 | 0.8571 |
+-------------+---------+---------+---------+---------+
| Overall | 0.2235 | 0.8647 | 0.1900 | 0.7954 |
+-------------+---------+---------+---------+---------+
{'bed_AP_0.25': 0.049614179879426956, 'table_AP_0.25': 0.15698249638080597, 'sofa_AP_0.25': 0.2261233627796173, 'chair_AP_0.25': 0.19846875965595245, 'toilet_AP_0.25': 0.35864582657814026, 'desk_AP_0.25': 0.09701932221651077, 'dresser_AP_0.25': 0.20626720786094666, 'night_stand_AP_0.25': 0.41742968559265137, 'bookshelf_AP_0.25': 0.15852327644824982, 'bathtub_AP_0.25': 0.3657997250556946, 'mAP_0.25': 0.22348737716674805, 'bed_rec_0.25': 0.7611650485436893, 'table_rec_0.25': 0.8513628620102215, 'sofa_rec_0.25': 0.84688995215311, 'chair_rec_0.25': 0.7661741214057508, 'toilet_rec_0.25': 0.9379310344827586, 'desk_rec_0.25': 0.7640807651434643, 'dresser_rec_0.25': 0.9128440366972477, 'night_stand_rec_0.25': 0.9607843137254902, 'bookshelf_rec_0.25': 0.8865248226950354, 'bathtub_rec_0.25': 0.9591836734693877, 'mAR_0.25': 0.8646940630326154, 'bed_AP_0.50': 0.009411284700036049, 'table_AP_0.50': 0.12223674356937408, 'sofa_AP_0.50': 0.14403118193149567, 'chair_AP_0.50': 0.1670922487974167, 'toilet_AP_0.50': 0.27775609493255615, 'desk_AP_0.50': 0.06645272672176361, 'dresser_AP_0.50': 0.19564399123191833, 'night_stand_AP_0.50': 0.41268840432167053, 'bookshelf_AP_0.50': 0.14886672794818878, 'bathtub_AP_0.50': 0.35609957575798035, 'mAP_0.50': 0.19002790749073029, 'bed_rec_0.50': 0.5883495145631068, 'table_rec_0.50': 0.790885860306644, 'sofa_rec_0.50': 0.773524720893142, 'chair_rec_0.50': 0.7272364217252396, 'toilet_rec_0.50': 0.8551724137931035, 'desk_rec_0.50': 0.6981934112646121, 'dresser_rec_0.50': 0.8669724770642202, 'night_stand_rec_0.50': 0.9450980392156862, 'bookshelf_rec_0.50': 0.851063829787234, 'bathtub_rec_0.50': 0.8571428571428571, 'mAR_0.50': 0.7953639545755846}
1.2 scannet
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune
the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/312, elapsed: 0[>>>>>>>>>>>>>>>>>>>] 312/312, 10.4 task/s, elapsed: 30s, ETA: 0s
+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.0077 | 0.2742 | 0.0052 | 0.2419 |
| bed | 0.0003 | 0.0370 | 0.0000 | 0.0247 |
| chair | 0.0193 | 0.2661 | 0.0138 | 0.2354 |
| sofa | 0.0015 | 0.0825 | 0.0005 | 0.0825 |
| table | 0.0016 | 0.1429 | 0.0011 | 0.1229 |
| door | 0.0104 | 0.2441 | 0.0079 | 0.2313 |
| window | 0.0028 | 0.1773 | 0.0021 | 0.1631 |
| bookshelf | 0.0081 | 0.1558 | 0.0042 | 0.1429 |
| picture | 0.0028 | 0.2658 | 0.0025 | 0.2568 |
| counter | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| desk | 0.0012 | 0.0866 | 0.0010 | 0.0709 |
| curtain | 0.0000 | 0.0448 | 0.0000 | 0.0448 |
| refrigerator | 0.0000 | 0.0351 | 0.0000 | 0.0000 |
| showercurtrain | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| toilet | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| sink | 0.0000 | 0.0102 | 0.0000 | 0.0000 |
| bathtub | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| garbagebin | 0.0048 | 0.2113 | 0.0039 | 0.1811 |
+----------------+---------+---------+---------+---------+
| Overall | 0.0034 | 0.1130 | 0.0024 | 0.0999 |
+----------------+---------+---------+---------+---------+
{'cabinet_AP_0.25': 0.007672073319554329, 'bed_AP_0.25': 0.0003270927118137479, 'chair_AP_0.25': 0.019300922751426697, 'sofa_AP_0.25': 0.0015425414312630892, 'table_AP_0.25': 0.0016294954111799598, 'door_AP_0.25': 0.010358083993196487, 'window_AP_0.25': 0.0027569225057959557, 'bookshelf_AP_0.25': 0.008131678216159344, 'picture_AP_0.25': 0.002778079127892852, 'counter_AP_0.25': 0.0, 'desk_AP_0.25': 0.0011750475969165564, 'curtain_AP_0.25': 2.2466720110969618e-05, 'refrigerator_AP_0.25': 1.3386352293309756e-05, 'showercurtrain_AP_0.25': 0.0, 'toilet_AP_0.25': 0.0, 'sink_AP_0.25': 1.440034111510613e-06, 'bathtub_AP_0.25': 0.0, 'garbagebin_AP_0.25': 0.00475729675963521, 'mAP_0.25': 0.003359251655638218, 'cabinet_rec_0.25': 0.27419354838709675, 'bed_rec_0.25': 0.037037037037037035, 'chair_rec_0.25': 0.26608187134502925, 'sofa_rec_0.25': 0.08247422680412371, 'table_rec_0.25': 0.14285714285714285, 'door_rec_0.25': 0.24411134903640258, 'window_rec_0.25': 0.1773049645390071, 'bookshelf_rec_0.25': 0.15584415584415584, 'picture_rec_0.25': 0.26576576576576577, 'counter_rec_0.25': 0.0, 'desk_rec_0.25': 0.08661417322834646, 'curtain_rec_0.25': 0.04477611940298507, 'refrigerator_rec_0.25': 0.03508771929824561, 'showercurtrain_rec_0.25': 0.0, 'toilet_rec_0.25': 0.0, 'sink_rec_0.25': 0.01020408163265306, 'bathtub_rec_0.25': 0.0, 'garbagebin_rec_0.25': 0.21132075471698114, 'mAR_0.25': 0.11298182832749847, 'cabinet_AP_0.50': 0.005249418318271637, 'bed_AP_0.50': 7.024568276392529e-06, 'chair_AP_0.50': 0.013795703649520874, 'sofa_AP_0.50': 0.0005232820985838771, 'table_AP_0.50': 0.001064108102582395, 'door_AP_0.50': 0.007914243265986443, 'window_AP_0.50': 0.0021169877145439386, 'bookshelf_AP_0.50': 0.0042017120867967606, 'picture_AP_0.50': 0.0025250331964343786, 'counter_AP_0.50': 0.0, 'desk_AP_0.50': 0.0010463166981935501, 'curtain_AP_0.50': 2.2466720110969618e-05, 'refrigerator_AP_0.50': 0.0, 'showercurtrain_AP_0.50': 0.0, 'toilet_AP_0.50': 0.0, 'sink_AP_0.50': 0.0, 'bathtub_AP_0.50': 0.0, 'garbagebin_AP_0.50': 0.003941362723708153, 'mAP_0.50': 0.0023559811525046825, 'cabinet_rec_0.
50': 0.24193548387096775, 'bed_rec_0.50': 0.024691358024691357, 'chair_rec_0.50': 0.23538011695906433, 'sofa_rec_0.50': 0.08247422680412371,
'table_rec_0.50': 0.12285714285714286, 'door_rec_0.50': 0.23126338329764454, 'window_rec_0.50': 0.16312056737588654, 'bookshelf_rec_0.50':
0.14285714285714285, 'picture_rec_0.50': 0.25675675675675674, 'counter_rec_0.50': 0.0, 'desk_rec_0.50': 0.07086614173228346, 'curtain_rec_0.
50': 0.04477611940298507, 'refrigerator_rec_0.50': 0.0, 'showercurtrain_rec_0.50': 0.0, 'toilet_rec_0.50': 0.0, 'sink_rec_0.50': 0.0, 'batht
ub_rec_0.50': 0.0, 'garbagebin_rec_0.50': 0.1811320754716981, 'mAR_0.50': 0.0998950286339104}
2.Evaluation results for the model trained by myself：(12 epoches)
2.1 sunrgbd
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/5050, elapsed: [>>>>>>>>>>>>>>>] 5050/5050, 0.3 task/s, elapsed: 16300s, ETA: 0s
+-------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed | 0.5560 | 0.9903 | 0.4873 | 0.8738 |
| table | 0.3110 | 0.9817 | 0.2184 | 0.7785 |
| sofa | 0.4583 | 0.9841 | 0.4016 | 0.8740 |
| chair | 0.5615 | 0.9633 | 0.4967 | 0.8643 |
| toilet | 0.6838 | 0.9931 | 0.6078 | 0.8966 |
| desk | 0.1938 | 0.9644 | 0.0907 | 0.6791 |
| dresser | 0.2366 | 0.9541 | 0.1876 | 0.7752 |
| night_stand | 0.4379 | 0.9765 | 0.3945 | 0.8824 |
| bookshelf | 0.2321 | 0.9184 | 0.0987 | 0.5426 |
| bathtub | 0.3912 | 0.9796 | 0.3296 | 0.8367 |
+-------------+---------+---------+---------+---------+
| Overall | 0.4062 | 0.9705 | 0.3313 | 0.8003 |
+-------------+---------+---------+---------+---------+
{'bed_AP_0.25': 0.5559849143028259, 'table_AP_0.25': 0.3109968602657318, 'sofa_AP_0.25': 0.4582793712615967, 'chair_AP_0.25': 0.5614634156227112, 'toilet_AP_0.25': 0.6837515830993652, 'desk_AP_0.25': 0.1938285529613495, 'dresser_AP_0.25': 0.23655451834201813, 'night_stand_AP_0.25': 0.43787485361099243, 'bookshelf_AP_0.25': 0.2321251481771469, 'bathtub_AP_0.25': 0.3912363648414612, 'mAP_0.25': 0.4062095582485199, 'bed_rec_0.25': 0.9902912621359223, 'table_rec_0.25': 0.981686541737649, 'sofa_rec_0.25': 0.9840510366826156, 'chair_rec_0.25': 0.963258785942492, 'toilet_rec_0.25': 0.993103448275862, 'desk_rec_0.25': 0.9643995749202976, 'dresser_rec_0.25': 0.9541284403669725, 'night_stand_rec_0.25': 0.9764705882352941, 'bookshelf_rec_0.25': 0.9184397163120568, 'bathtub_rec_0.25': 0.9795918367346939, 'mAR_0.25': 0.9705421231343856, 'bed_AP_0.50': 0.4872896671295166, 'table_AP_0.50': 0.21839171648025513, 'sofa_AP_0.50': 0.40156134963035583, 'chair_AP_0.50': 0.4966593384742737, 'toilet_AP_0.50': 0.607752799987793, 'desk_AP_0.50': 0.09067006409168243, 'dresser_AP_0.50': 0.18762624263763428, 'night_stand_AP_0.50': 0.39452967047691345, 'bookshelf_AP_0.50': 0.09874459356069565, 'bathtub_AP_0.50': 0.32958272099494934, 'mAP_0.50': 0.33128082752227783, 'bed_rec_0.50': 0.8737864077669902, 'table_rec_0.50': 0.778534923339012, 'sofa_rec_0.50': 0.8740031897926634, 'chair_rec_0.50': 0.8643170926517572, 'toilet_rec_0.50': 0.896551724137931, 'desk_rec_0.50': 0.6790648246546227, 'dresser_rec_0.50': 0.7752293577981652, 'night_stand_rec_0.50': 0.8823529411764706, 'bookshelf_rec_0.50': 0.5425531914893617, 'bathtub_rec_0.50': 0.8367346938775511, 'mAR_0.50': 0.8003128346684527}
2.2 scannet
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tun$
the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/312, elapsed: 0[>>>>>>> ] 116/312, 0.2 task/s, elapsed[ ] 117/312, 0.2 t$
sk/s, elapsed: 547s,[ ] 118/312, 0.2 task/s, elapsed: 547s, ETA: 89[>>>>>>>>>>>>> ] 240/312, [> ] 241/312, 0.2 task/s, elapsed: 1262$
[> ] 242/312, 0.2 task/s, elapsed: 1262s[> ] 243/312, 0.2 task/s, elap[>>>>>>>>>>>>>>>>>>] 312/312, 0.2 task/s, elapsed: 1531s, ETA: 0s
+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.0089 | 0.2500 | 0.0071 | 0.2419 |
| bed | 0.0022 | 0.0494 | 0.0005 | 0.0123 |
| chair | 0.0195 | 0.2661 | 0.0131 | 0.2383 |
| sofa | 0.0009 | 0.0825 | 0.0000 | 0.0309 |
| table | 0.0030 | 0.1400 | 0.0021 | 0.1257 |
| door | 0.0072 | 0.2698 | 0.0066 | 0.2548 |
| window | 0.0030 | 0.1844 | 0.0020 | 0.1525 |
| bookshelf | 0.0041 | 0.1429 | 0.0031 | 0.1429 |
| picture | 0.0049 | 0.2523 | 0.0045 | 0.2342 |
| counter | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| desk | 0.0041 | 0.0709 | 0.0041 | 0.0551 |
| curtain | 0.0001 | 0.0597 | 0.0001 | 0.0597 |
| refrigerator | 0.0001 | 0.0877 | 0.0001 | 0.0877 |
| showercurtrain | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| toilet | 0.0005 | 0.0345 | 0.0001 | 0.0345 |
| sink | 0.0000 | 0.0408 | 0.0000 | 0.0408 |
| bathtub | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| garbagebin | 0.0076 | 0.2038 | 0.0063 | 0.1887 |
+----------------+---------+---------+---------+---------+
| Overall | 0.0037 | 0.1186 | 0.0028 | 0.1056 |
+----------------+---------+---------+---------+---------+
{'cabinet_AP_0.25': 0.0088879419490695, 'bed_AP_0.25': 0.002196365036070347, 'chair_AP_0.25': 0.019513951614499092, 'sofa_AP_0.25': 0.00092$
0919025614858, 'table_AP_0.25': 0.0029890856239944696, 'door_AP_0.25': 0.007219596300274134, 'window_AP_0.25': 0.002968191634863615, 'books$
elf_AP_0.25': 0.0041231452487409115, 'picture_AP_0.25': 0.004887890536338091, 'counter_AP_0.25': 0.0, 'desk_AP_0.25': 0.004088526125997305,
'curtain_AP_0.25': 0.0001449216069886461, 'refrigerator_AP_0.25': 0.00010905037925112993, 'showercurtrain_AP_0.25': 0.0, 'toilet_AP_0.25': $
.0005089639453217387, 'sink_AP_0.25': 3.358132016728632e-05, 'bathtub_AP_0.25': 0.0, 'garbagebin_AP_0.25': 0.007584888022392988, 'mAP_0.25'$
0.003676677355542779, 'cabinet_rec_0.25': 0.25, 'bed_rec_0.25': 0.04938271604938271, 'chair_rec_0.25': 0.26608187134502925, 'sofa_rec_0.25$
: 0.08247422680412371, 'table_rec_0.25': 0.14, 'door_rec_0.25': 0.2698072805139186, 'window_rec_0.25': 0.18439716312056736, 'bookshelf_rec_0
.25': 0.14285714285714285, 'picture_rec_0.25': 0.25225225225225223, 'counter_rec_0.25': 0.0, 'desk_rec_0.25': 0.07086614173228346, 'curtain_
rec_0.25': 0.05970149253731343, 'refrigerator_rec_0.25': 0.08771929824561403, 'showercurtrain_rec_0.25': 0.0, 'toilet_rec_0.25': 0.034482758
620689655, 'sink_rec_0.25': 0.04081632653061224, 'bathtub_rec_0.25': 0.0, 'garbagebin_rec_0.25': 0.2037735849056604, 'mAR_0.25': 0.118589569
75081056, 'cabinet_AP_0.50': 0.007079910486936569, 'bed_AP_0.50': 0.0005367686389945447, 'chair_AP_0.50': 0.013134335167706013, 'sofa_AP_0.5
0': 4.525698022916913e-05, 'table_AP_0.50': 0.0021134275011718273, 'door_AP_0.50': 0.00658177025616169, 'window_AP_0.50': 0.0019997931085526
943, 'bookshelf_AP_0.50': 0.0031221157405525446, 'picture_AP_0.50': 0.004526467993855476, 'counter_AP_0.50': 0.0, 'desk_AP_0.50': 0.00407800
916582346, 'curtain_AP_0.50': 0.0001449216069886461, 'refrigerator_AP_0.50': 0.0001018800976453349, 'showercurtrain_AP_0.50': 0.0, 'toilet_A
P_0.50': 5.409148434409872e-05, 'sink_AP_0.50': 3.358132016728632e-05, 'bathtub_AP_0.50': 0.0, 'garbagebin_AP_0.50': 0.006303954869508743, '
mAP_0.50': 0.0027697933837771416, 'cabinet_rec_0.50': 0.24193548387096775, 'bed_rec_0.50': 0.012345679012345678, 'chair_rec_0.50': 0.2383040
9356725146, 'sofa_rec_0.50': 0.030927835051546393, 'table_rec_0.50': 0.12571428571428572, 'door_rec_0.50': 0.25481798715203424, 'window_rec_
0.50': 0.1524822695035461, 'bookshelf_rec_0.50': 0.14285714285714285, 'picture_rec_0.50': 0.23423423423423423, 'counter_rec_0.50': 0.0, 'des
k_rec_0.50': 0.05511811023622047, 'curtain_rec_0.50': 0.05970149253731343, 'refrigerator_rec_0.50': 0.08771929824561403, 'showercurtrain_rec
472 Dec 22, 2022
215 Nov 28, 2022
4 Dec 15, 2022
4 Jul 19, 2022
158 Jan 4, 2023
146 Dec 24, 2022
8 Dec 11, 2022
23 Oct 20, 2022
148 Dec 21, 2022
5 Dec 10, 2022
3 Jan 26, 2022
115 Dec 28, 2022
111 Dec 27, 2022
158 Dec 21, 2022
606 Dec 21, 2022
1.6k Jan 7, 2023