Small object detection explained

Small object detection is a particular case of object detection where various techniques are employed to detect small objects in digital images and videos. "Small objects" are objects having a small pixel footprint in the input image. In areas such as aerial imagery, state-of-the-art object detection techniques under performed because of small objects.

Uses

Small object detection has applications in various fields such as Video surveillance (Traffic video Surveillance,^{[1] [2]} Small object retrieval,^{[3] [4]} Anomaly detection,^[5] Maritime surveillance, Drone surveying, Traffic flow analysis,^[6] and Object tracking.

Problems with small objects

• Modern-day object detection algorithms such as You Only Look Once(YOLO)^{[7] [8] [9] [10] [11] [12] [13]} heavily uses convolution layers to learn features. As an object passes through convolution layers, its size gets reduced. Therefore, the small object disappears after several layers and becomes undetectable.
• Sometimes, the shadow of an object is detected as a part of object itself.^[14] So, the placement of the bounding box tends to centre around a shadow rather than an object. In the case of vehicle detection, pedestrian and two-wheeler detection suffer because of this.
• At present, drones are very widely used in aerial imagery.^[15] They are equipped with hardware (sensors) and software (algorithms) that help maintain a particular stable position during their flight. In windy conditions, the drone automatically makes fine moves to maintain its position and that changes the view near the boundary. It may be possible that some new objects appear near the image boundary. Overall, these affect classification, detection, and eventually tracking accuracy.

Methods

Various methods^[16] are available to detect small objects, which fall under three categories:

Improvising existing techniques

There are various ways to detect small objects with existing techniques. Some of them are mentioned below,

Choosing a data set that has small objects

The machine learning model's output depends on "How well it is trained."^[17] So, the data set must include small objects to detect such objects. Also, modern-day detectors, such as YOLO, rely on anchors. Latest versions of YOLO (starting from YOLOv5^[18]) uses an auto-anchor algorithm to find good anchors based on the nature of object sizes in the data set. Therefore, it is mandatory to have smaller objects in the data set.

Generating more data via augmentation, if required

Deep learning models have billions of neurons that settle down to some weights after training. Therefore, it requires a good amount of quantitative and qualitative data for better training.^[19] Data augmentation is useful technique to generate more diverse data from an existing data set.

Increasing image capture resolution and model’s input resolution

These help to get more features from objects and eventually learn the best from them. For example, a bike object in the 1280 X 1280 resolution image has more features than the 640 X 640 resolution.

Auto learning anchors

Selecting anchor size plays a vital role in small object detection.^[20] Instead of hand picking it, use algorithms that identify it based on the data set. YOLOv5 uses a K-means algorithm to define anchor size.

Tiling approach during training and inference

State-of-the-art object detectors allow only the fixed size of image and change the input image size according to it. This change may deform the small objects in the image. The tiling approach^[21] helps when an image has a high resolution than the model's fixed input size; instead of scaling it down, the image is broken down into tiles and then used in training. The same approach is used during inference as well.

Feature Pyramid Network (FPN)

Use a feature pyramid network^[22] to learn features at a multi-scale: e.g., Twin Feature Pyramid Networks (TFPN),^[23] Extended Feature Pyramid Network (EFPN).^[24] FPN helps to sustain features of small objects against convolution layers.

Add-on techniques

Instead of modifying existing methods, some add-on techniques are there, which can be directly placed on top of existing approaches to detect smaller objects. One such technique is Slicing Aided Hyper Inference(SAHI).^[25] The image is sliced into different-sized multiple overlapping patches. Hyper-parameters define their dimensions. Then patches are resized, while maintaining the aspect ratio during fine-tuning. These patches are then provided for training the model.

Well-Optimised techniques for small object detection

Various deep learning techniques are available that focus on such object detection problems: e.g., Feature-Fused SSD,^[26] YOLO-Z.^[27] Such methods work on "How to sustain features of small objects while they pass through convolution networks."

Other applications

• Crowd counting^{[28] [29] [30] [31]}
• Vehicle re-identification^[32]
• Animal detection^{[33] [34] [35] [36]}
• Fish detection^[37]

See also

• Convolutional neural network
• Use of UAVs in law enforcement
• Underwater computer vision
• Intelligent transportation system

References

1. Book: Saran K B . Sreelekha G . 2015 International Conference on Control Communication & Computing India (ICCC) . Traffic video surveillance: Vehicle detection and classification . https://ieeexplore.ieee.org/document/7432948 . 2015 . Trivandrum, Kerala, India . IEEE . 516–521 . 10.1109/ICCC.2015.7432948 . 978-1-4673-7349-4. 14779393 .
2. Nemade . Bhushan . 2016-01-01 . Automatic Traffic Surveillance Using Video Tracking . Procedia Computer Science . Proceedings of International Conference on Communication, Computing and Virtualization (ICCCV) 2016 . en . 79 . 402–409 . 10.1016/j.procs.2016.03.052 . 1877-0509. free .
3. Book: Guo . Haiyun . Wang . Jinqiao . Xu . Min . Zha . Zheng-Jun . Lu . Hanqing . Proceedings of the 23rd ACM international conference on Multimedia . Learning Multi-view Deep Features for Small Object Retrieval in Surveillance Scenarios . 2015-10-13 . https://doi.org/10.1145/2733373.2806349 . MM '15 . New York, NY, USA . Association for Computing Machinery . 859–862 . 10.1145/2733373.2806349 . 978-1-4503-3459-4. 9041849 .
4. Galiyawala . Hiren . Raval . Mehul S. . Patel . Meet . 2022-05-20 . Person retrieval in surveillance videos using attribute recognition . Journal of Ambient Intelligence and Humanized Computing . en . 10.1007/s12652-022-03891-0 . 248951090 . 1868-5145.
5. Ingle . Palash Yuvraj . Kim . Young-Gab . 2022-05-19 . Real-Time Abnormal Object Detection for Video Surveillance in Smart Cities . Sensors . en . 22 . 10 . 3862 . 10.3390/s22103862 . 1424-8220 . 9143895 . 35632270. 2022Senso..22.3862I . free .
6. Tsuboi . Tsutomu . Yoshikawa . Noriaki . 2020-03-01 . Traffic flow analysis in Ahmedabad (India) . Case Studies on Transport Policy . en . 8 . 1 . 215–228 . 10.1016/j.cstp.2019.06.001 . 195543435 . 2213-624X. free .
7. Redmon . Joseph . Divvala . Santosh . Girshick . Ross . Farhadi . Ali . 2016-05-09 . You Only Look Once: Unified, Real-Time Object Detection . cs.CV . 1506.02640.
8. Redmon . Joseph . Farhadi . Ali . 2016-12-25 . YOLO9000: Better, Faster, Stronger . cs.CV . 1612.08242.
9. Redmon . Joseph . Farhadi . Ali . 2018-04-08 . YOLOv3: An Incremental Improvement . cs.CV . 1804.02767.
10. Bochkovskiy . Alexey . Wang . Chien-Yao . Liao . Hong-Yuan Mark . 2020-04-22 . YOLOv4: Optimal Speed and Accuracy of Object Detection . cs.CV . 2004.10934.
11. Wang . Chien-Yao . Bochkovskiy . Alexey . Liao . Hong-Yuan Mark . 2021-02-21 . Scaled-YOLOv4: Scaling Cross Stage Partial Network . cs.CV . 2011.08036.
12. Li . Chuyi . Li . Lulu . Jiang . Hongliang . Weng . Kaiheng . Geng . Yifei . Li . Liang . Ke . Zaidan . Li . Qingyuan . Cheng . Meng . Nie . Weiqiang . Li . Yiduo . Zhang . Bo . Liang . Yufei . Zhou . Linyuan . Xu . Xiaoming . 2022-09-07 . YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications . cs.CV . 2209.02976.
13. Wang . Chien-Yao . Bochkovskiy . Alexey . Liao . Hong-Yuan Mark . 2022-07-06 . YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors . cs.CV . 2207.02696.
14. Book: Zhang . Mingrui . Zhao . Wenbing . Li . Xiying . Wang . Dan . 2020 IEEE 9th Joint International Information Technology and Artificial Intelligence Conference (ITAIC) . Shadow Detection of Moving Objects in Traffic Monitoring Video . 2020-12-11 . https://ieeexplore.ieee.org/document/9338958 . 9 . Chongqing, China . IEEE . 1983–1987 . 10.1109/ITAIC49862.2020.9338958 . 978-1-7281-5244-8. 231824327 .
15. Book: https://ieeexplore.ieee.org/document/7486437 . 2016 . Herndon, VA . IEEE . 1–17 . 10.1109/ICNSURV.2016.7486437 . 978-1-5090-2149-9. 21388151 . Interactive workshop "How drones are changing the world we live in" . 2016 Integrated Communications Navigation and Surveillance (ICNS) .
16. An Evaluation of Deep Learning Methods for Small Object Detection . Journal of Electrical and Computer Engineering . 2020 . en . 10.1155/2020/3189691. free . Nguyen . Nhat-Duy . Do . Tien . Ngo . Thanh Duc . Le . Duy-Dinh . 2020 . 1–18 .
17. Gong . Zhiqiang . Zhong . Ping . Hu . Weidong . 2019 . Diversity in Machine Learning . IEEE Access . 7 . 64323–64350 . 10.1109/ACCESS.2019.2917620 . 206491718 . 2169-3536. free . 1807.01477 .
18. Jocher . Glenn . ultralytics/yolov5: v6.2 - YOLOv5 Classification Models, Apple M1, Reproducibility, ClearML and Deci.ai integrations . 2022-08-17 . 10.5281/zenodo.3908559 . 2022-09-14 . Chaurasia . Ayush . Stoken . Alex . Borovec . Jirka . NanoCode012 . Kwon . Yonghye . TaoXie . Michael . Kalen . Fang . Jiacong.
19. Web site: The Size and Quality of a Data Set Machine Learning . 2022-09-14 . Google Developers . en.
20. Zhong . Yuanyi . Wang . Jianfeng . Peng . Jian . Zhang . Lei . 2020-01-26 . Anchor Box Optimization for Object Detection . cs.CV . 1812.00469.
21. Book: Unel . F. Ozge . Ozkalayci . Burak O. . Cigla . Cevahir . 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . The Power of Tiling for Small Object Detection . https://ieeexplore.ieee.org/document/9025422 . 2019 . Long Beach, CA, USA . IEEE . 582–591 . 10.1109/CVPRW.2019.00084 . 978-1-7281-2506-0. 198903617 .
22. Lin . Tsung-Yi . Dollár . Piotr . Girshick . Ross . He . Kaiming . Hariharan . Bharath . Belongie . Serge . 2017-04-19 . Feature Pyramid Networks for Object Detection . cs.CV . 1612.03144.
23. Book: Liang . Yi . Changjian . Wang . Fangzhao . Li . Yuxing . Peng . Qin . Lv . Yuan . Yuan . Zhen . Huang . 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI) . TFPN: Twin Feature Pyramid Networks for Object Detection . https://ieeexplore.ieee.org/document/8995365 . 2019 . Portland, OR, USA . IEEE . 1702–1707 . 10.1109/ICTAI.2019.00251 . 978-1-7281-3798-8. 211211764 .
24. Deng . Chunfang . Wang . Mengmeng . Liu . Liang . Liu . Yong . 2020-04-09 . Extended Feature Pyramid Network for Small Object Detection . cs.CV . 2003.07021.
25. Book: Akyon . Fatih Cagatay . Altinuc . Sinan Onur . Temizel . Alptekin . 2022-07-12 . 2022 IEEE International Conference on Image Processing (ICIP) . Slicing Aided Hyper Inference and Fine-Tuning for Small Object Detection . 966–970 . 10.1109/ICIP46576.2022.9897990 . 2202.06934. 978-1-6654-9620-9 . 246823962 .
26. Book: Cao . Guimei . Xie . Xuemei . Yang . Wenzhe . Liao . Quan . Shi . Guangming . Wu . Jinjian . Ninth International Conference on Graphic and Image Processing (ICGIP 2017) . Feature-fused SSD: Fast detection for small objects . Junyu . Hui . Dong . Yu . 2018-04-10 . https://www.spiedigitallibrary.org/conference-proceedings-of-spie/10615/106151E/Feature-fused-SSD-fast-detection-for-small-objects/10.1117/12.2304811.full . SPIE . 10615 . 381–388 . 10.1117/12.2304811. 1709.05054 . 2018SPIE10615E..1EC . 9781510617414 . 20592770 .
27. Benjumea . Aduen . Teeti . Izzeddin . Cuzzolin . Fabio . Bradley . Andrew . 2021-12-23 . YOLO-Z: Improving small object detection in YOLOv5 for autonomous vehicles . cs.CV . 2112.11798.
28. Book: Rajendran . Logesh . Shyam Shankaran . R . 2021 IEEE International Conference on Big Data and Smart Computing (BigComp) . Bigdata Enabled Realtime Crowd Surveillance Using Artificial Intelligence and Deep Learning . https://ieeexplore.ieee.org/document/9373133 . 2021 . Jeju Island, Korea (South) . IEEE . 129–132 . 10.1109/BigComp51126.2021.00032 . 978-1-7281-8924-6. 232236614 .
29. Book: Sivachandiran . S. . Mohan . K. Jagan . Nazer . G. Mohammed . 2022 6th International Conference on Computing Methodologies and Communication (ICCMC) . Deep Transfer Learning Enabled High-Density Crowd Detection and Classification using Aerial Images . 2022-03-29 . https://ieeexplore.ieee.org/document/9753982 . Erode, India . IEEE . 1313–1317 . 10.1109/ICCMC53470.2022.9753982 . 978-1-6654-1028-1. 248131806 .
30. Book: Santhini . C. . Gomathi . V. . 2018 International Conference on Current Trends towards Converging Technologies (ICCTCT) . Crowd Scene Analysis Using Deep Learning Network . https://ieeexplore.ieee.org/document/8550851 . 2018 . 1–5 . 10.1109/ICCTCT.2018.8550851. 978-1-5386-3702-9 . 54438440 .
31. Book: Sharath . S.V. . Biradar . Vidyadevi . Prajwal . M.S. . Ashwini . B. . 2021 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics (DISCOVER) . Crowd Counting in High Dense Images using Deep Convolutional Neural Network . 2021-11-19 . https://ieeexplore.ieee.org/document/9663716 . Nitte, India . IEEE . 30–34 . 10.1109/DISCOVER52564.2021.9663716 . 978-1-6654-1244-5. 245707782 .
32. Wang . Hongbo . Hou . Jiaying . Chen . Na . 2019 . A Survey of Vehicle Re-Identification Based on Deep Learning . IEEE Access . 7 . 172443–172469 . 10.1109/ACCESS.2019.2956172 . 209319743 . 2169-3536. free .
33. Book: Santhanam . Sanjay . B . Sudhir Sidhaarthan . Panigrahi . Sai Sudha . Kashyap . Suryakant Kumar . Duriseti . Bhargav Krishna . 2021 International Conference on Computational Intelligence and Computing Applications (ICCICA) . Animal Detection for Road safety using Deep Learning . 2021-11-26 . https://ieeexplore.ieee.org/document/9697287 . Nagpur, India . IEEE . 1–5 . 10.1109/ICCICA52458.2021.9697287 . 978-1-6654-2040-2. 246663727 .
34. Book: Li . Nopparut . Kusakunniran . Worapan . Hotta . Seiji . 2020 17th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON) . Detection of Animal Behind Cages Using Convolutional Neural Network . https://ieeexplore.ieee.org/document/9158137 . 2020 . Phuket, Thailand . IEEE . 242–245 . 10.1109/ECTI-CON49241.2020.9158137 . 978-1-7281-6486-1. 221086279 .
35. Book: Oishi . Yu . Matsunaga . Tsuneo . 2010 IEEE International Geoscience and Remote Sensing Symposium . Automatic detection of moving wild animals in airborne remote sensing images . https://ieeexplore.ieee.org/document/5654227 . 2010 . 517–519 . 10.1109/IGARSS.2010.5654227. 978-1-4244-9565-8 . 16812504 .
36. Ramanan . D. . Forsyth . D.A. . Barnard . K. . Building models of animals from video . IEEE Transactions on Pattern Analysis and Machine Intelligence . 2006 . 28 . 8 . 1319–1334 . 10.1109/TPAMI.2006.155 . 16886866 . 1699015 . 0162-8828.
37. Fish Detection Using Deep Learning . Applied Computational Intelligence and Soft Computing . 2020 . en . 10.1155/2020/3738108. free . Cui . Suxia . Zhou . Yu . Wang . Yonghui . Zhai . Lujun . 2020 . 1–13 .

External links

• VisDrone dataset by AISKYEYE team at Lab of Machine Learning and Data Mining, Tianjin University, China.