👁️ Score (SN44) proposes Smart mAP as a fairer standard for object detection 👁️
Score (SN44) is developing Smart mAP, a size-aware scoring system designed to make computer-vision rankings better reflect real detection quality.
🔑 Key points
🔹 Fixed IoU thresholds create bias: Traditional mAP often applies the same overlap requirement to every object, regardless of its size in the image.
🔹 Small objects are disproportionately penalized: A few pixels of error can significantly reduce the score of a tiny object.
🔹 Smart mAP adjusts thresholds by scale: Smaller objects receive more tolerance, while larger objects are evaluated more strictly.
🔹 Small objects can use a 0.30 threshold: Objects below the smallest size range receive a lower threshold to account for unavoidable localization error.
🔹 Medium objects retain standard scoring: Objects in the middle range continue using the conventional 0.50 threshold.
🔹 Large objects can require 0.70 IoU: Bigger objects are expected to have more precise boundaries because small errors matter less proportionally.
🔹 Hard limits prevent manipulation: Smart mAP uses minimum and maximum thresholds so scoring cannot become excessively loose or strict.
🔹 Rankings and rewards are affected: Because SN44 uses model performance to rank miners, the scoring system influences which models receive recognition and emissions.
🔎 Why it matters
🔹 A fair benchmark should measure model capability rather than object size.
🔹 Size-aware scoring could improve evaluations in surveillance, manufacturing, sports, robotics, and autonomous systems.
🔹 Better metrics can reduce incentive gaming and encourage miners to build models that perform well across realistic conditions.
🔹 Smart mAP is not a complete solution. Dataset quality, annotation accuracy, class balance, and unusual edge cases still affect results.
🎯 Bottom line: Smart mAP is an attempt to make object-detection rankings more representative by accounting for object scale. If validated across diverse datasets, the metric could improve both competition and reward allocation on SN44—but the scoring method still needs independent testing before it can be treated as a universal standard.
https://taodaily.io/score-sn44s-smart-map-as-a-fairer-standard-for-object-detection/