👁️ Score introduces Smart mAP for fairer computer-vision rankings 👁️
Score (SN44) has introduced Smart mAP, an evaluation system that adjusts object-detection scoring based on object size to make model rankings more accurate.
🔑 Key points
🔹 Fixed thresholds replaced: Traditional mAP often applies the same Intersection over Union threshold to every object, regardless of size.
🔹 Small objects receive more tolerance: For objects smaller than 1% of an image, the IoU threshold can gradually fall from 0.50 to 0.30.
🔹 Medium objects remain standard: Objects between 1% and 5% of the image continue using a 0.50 threshold.
🔹 Large objects face stricter scoring: For objects larger than 5%, the threshold can rise toward 0.70.
🔹 Hard limits prevent extremes: Objects smaller than 0.05% are capped at 0.30, while objects larger than 25% are capped at 0.70.
🔹 Scale bias is reduced: A minor pixel-level error can drastically hurt the score of a tiny object while barely affecting the score of a large one.
🔹 Rankings become more proportional: Smart mAP is designed to measure localization quality more fairly across different object sizes.
🔹 Rewards are affected: Since SN44 uses model rankings to guide incentives, the scoring change can influence which miners receive recognition and rewards.
🔎 Why it matters
🔹 A fixed IoU threshold can reward or punish models based on object scale rather than actual detection quality.
🔹 More balanced scoring could improve competition across sports footage, surveillance, manufacturing, and other real-world environments.
🔹 Better evaluation metrics are essential if decentralized AI networks are going to distribute rewards based on useful model performance.
🔹 The change does not automatically make rankings perfect. Dataset quality, labeling accuracy, edge cases, and benchmark design still matter.
🎯 Bottom line: Smart mAP gives Score a more scale-aware way to evaluate computer-vision models. By reducing the bias against tiny objects and demanding greater precision from large ones, SN44 is trying to make its leaderboard—and the rewards tied to it—better reflect real-world detection performance.
https://www.tao.media/score-introduces-smart-map-for-fairer-computer-vision-model-rankings/