Neural Network Based Algorithms for Traffic Lights Identification in Multi-Camera Driving Aid Systems
EDN: FVHXVN
Abstract
The paper considers the problem of traffic lights identification (detection, filtering and map-matching) using successive images in active aid systems for tram drivers, equipped with multiple cameras with different focal lengths. The process of the problem solution is described in detail, from measurements (detections) formed at the neural network output for each of the cameras, and up to the results matching with a map. In contrast to other studies of this subject, the authors of this work use 3D measurements as the output data for the neural network, and unscented Kalman filter (UKF) for determining the position of the traffic lights; in addition, a new method for fusing the data from two cameras is applied. The efficiency of the proposed algorithms and its modification has been field-tested. The results of experiments have shown that the algorithm provides the accuracy of 76.19% and completeness of 97.46% when used in combination with the tram control system with two cameras.
About the Authors
N. S. Guzhva
National University of Science and Technology MISIS, Cognitive Technologies
Russian Federation
Russian Federation
Moscow
R. N. Sadekov
National University of Science and Technology MISIS, Cognitive Technologies
Russian Federation
Russian Federation
Moscow
References
1. Хартов В.Я., Гарифуллин Д.Б. Анализ решений для задачи распознавания автомобильных светофоров // Приоритетные направления развития науки и образования: сборник статей VI Международной научно-практической конференции в 2-х частях. Часть 1. Пенза: «Наука и Просвещение», 2019. С. 64–76. EDN ZBNPVB.
2. Tran, Tai Huu-Phuong; Pham, Cuong Cao; Nguyen, Tien Phuoc; Duong, Tin Trung; Jeon, Jae Wook, Real-time traffic light detection using color density, IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), 2016, 1–4. doi:10.1109/ICCE-Asia.2016.7804791.
3. Гужва Н.С., Прун В.Е., Постников В.В., Лобанов М.Г., Садеков Р.Н., Шоломов Д.Л. Использование 3D-сетей для «предсказания» моделей поведения транспортных средств в задаче беспилотного движения трамвая // 29 Санкт-Петербургская международная конференция по интегрированным навигационным системам. Санкт-Петербург, 2022. С. 304–310.
4. Гужва Н.С., Али Б., Бакулев К.С., Садеков Р.Н., Шолохов А.В. Оценка точности местоположения трамвая в условиях высотной застройки по данным визуально-геоинформационных систем // XXX Санкт-Петербургская международная конференция по интегрированным навигационным системам. Санкт-Петербург, 2023. С. 16–21.
5. Али Б., Садеков Р. Н., Цодокова В.В. Алгоритмы навигации БПЛА с использованием систем технического зрения // Гироскопия и навигация. Том 30. №4 (119). 2022. C. 87–105. DOI 10.17285/0869-7035.00105.
6. Haltakov, V., Mayr, J., Unger, C., Ilic, S., Semantic Segmentation Based Traffic Light Detection at Day and at Night, Pattern Recognition. DAGM 2015. Lecture Notes in Computer Science, 2015, vol. 9358, https://doi.org/10.1007/978-3-319-24947-6_37.
7. Weber, M., Wolf, P., and Zollner, J.M., Deeptlr: A single deep convolutional network for detection and classification of traffic lights, IEEE IV Intelligent Vehicles Symposium, Gotenburg, Sweden, June 19-22, 2016, pp. 342–348.
8. Behrendt, K., Novak, L., Botros, R., A deep learning approach to traffic lights: Detection, tracking, and classification, IEEE International Conference on Robotics and Automation (ICRA), 2017, pp. 1370–1377.
9. Palmer, A.W., Sema, A., Martens, W., Rudolph, P., and Waizenegger, W., The Autonomous Siemens Tram, IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), Rhodes, Greece, 2020, pp. 1–6, doi: 10.1109/ITSC45102.2020.9294699.
10. Possatti, L.C. et al., Traffic Light Recognition Using Deep Learning and Prior Maps for Autonomous Cars, International Joint Conference on Neural Networks (IJCNN), 2019, pp. 1–8.
11. Wang, Q.; Zhang, Q.; Liang, X.; Wang, Y.; Zhou, C.; Mikulovich, V.I., Traffic Lights Detection and Recognition Method Based on the Improved YOLOv4 Algorithm, Sensors, 2022, 22, 200, https://doi.org/10.3390/s22010200.
12. Wu, S. et al., aUToLights: A Robust Multi-Camera Traffic Light Detection and Tracking System, arXiv preprint arXiv, 2023, 2305, 08673.
13. Mentasti, S., Simsek, Y.C., and Matteucci, M., Traffic lights detection and tracking for HD map creation, Front. Robot. AI, 2023, 10:1065394, doi: 10.3389/frobt.2023.1065394.
14. https://en.wikipedia.org/wiki/Hungarian_algorithm.
15. Jensen, M.B., Philipsen, M.P., Møgelmose, A., Moeslund, T.B., Trivedi M.M., Vision for looking at traffic lights: Issues, survey, and perspectives, IEEE Transactions on Intelligent Transportation Systems, 2016, no. 17, pp. 1800–1815.
16. Fregin, A., Muller, J., Krebel, U., and Dietmayer, K., The DriveU Traffic Light Dataset: Introduction and Comparison with Existing Datasets, IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 3376–3383.
17. https://en.wikipedia.org/wiki/Jaccard_index
18. Kim, J., Cho, H., Hwangbo, M., Choi, J., Canny, J., and Kwon, Y.P., Deep Traffic Light Detection for Self-driving Cars from a Large-scale Dataset, 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA, 2018, pp. 280–285, doi: 10.1109/ITSC.2018.8569575.
19. Li, Z., Zeng, Q., Liu, Y., Liu, J., Li, L., An improved traffic lights recognition algorithm for autonomous driving in complex scenarios, International Journal of Distributed Sensor Networks, 2021;17(5), doi:10.1177/15501477211018374.
20. Chelva, M.S., Halse, S.V., and Ratha, B.K., Object tracking in real time embedded system using image processing, International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES), Paralakhemundi, India, 2016, pp. 1840–1844, doi: 10.1109/SCOPES.2016.7955763.
21. Radosavovic, I. et al., Designing network design spaces, Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10428–10436.
22. Lin, T.Y. et al., Focal loss for dense object detection, Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988.
Review
For citations:
Guzhva N.S., Sadekov R.N. Neural Network Based Algorithms for Traffic Lights Identification in Multi-Camera Driving Aid Systems. Gyroscopy and Navigation. 2024;32(3):47-65. (In Russ.) EDN: FVHXVN
Views:
16
Email this article 