Real-time Visual-Inertial Odometry based on Point-Line Feature Fusion
EDN: RQVMYA
Abstract
To improve the localization accuracy and tracking robustness of monocular feature-based visual SLAM systems in low-texture environments, a visual-inertial odometry method combining line features and point features is proposed, taking advantage of the easy avail ability of line features in real-world environments and the high accuracy of feature-based methods. The combination of point and line features ensures accurate positioning of the SLAM system in low-texture environments, while the inclusion of IMU data provides prior information and scale information. The pose is optimized by minimizing the reprojection error of point and line features and the IMU error using bundle adjustment. An improved EDlines algorithm is introduced, which incorporates a pixel chain length suppression pro cess to enhance the effectiveness of extracted line features and reduce the rate of line fea ture misalignment. Experimental results on the public EuRoC dataset and TUM RGB-D dataset show that the proposed method meets the real-time requirements and has higher localization accuracy and robustness compared with the visual SLAM method based on single point feature or the method adding traditional line features.
About the Authors
Gang Yang
Xi’an University of Posts and Telecommunications
China
China
Xi’an
WeiDa Meng
Xi’an University of Posts and Telecommunications
China
China
Xi’an
GuoDong Hou
Xi’an University of Posts and Telecommunications
China
China
Xi’an
NingNing Feng
Xi’an University of Posts and Telecommunications
China
China
Xi’an
References
1. Liu, Y.J., Zhang, Y.Z., Rong, L., Jiang, H., and Deng, Y., Visual odometry based on the direct method and the inertial measurement unit, Robot, 2019, vol. 41, no. 5, pp. 683–689. https://doi.org/10.13973/j.cnki.robot.180601
2. Pan, L.H., Tian, F.Q., Ying, W.J., Liang, W.G., and She, B., VI-SLAM algorithm with camera IMU extrinsic automatic calibration and online estimation, Chinese Journal of Scientific Instrument, 2019, vol. 40, no. 6, pp. 56–67. https://doi.org/10.19650/j.cnki.cjsi.J1904954 3. 3. Mourikis, A.I., and Roumeliotis, S.I., A multi-state constraint Kalman filter for vision-aided iner tial navigation, Proc. IEEE International Conference on Robotics and Automation (ICRA), 2007, pp. 3565–3572. https://doi.org/10.1109/ROBOT.2007.364024
3. Mur-Artal, R., Montiel, J.M.M., and Tardós, J.D., ORB-SLAM: A versatile and accurate monoc ular SLAM system, IEEE Transactions on Robotics, 2015, vol. 31, no. 5, pp. 1147–1163. https://doi. org/10.1109/TRO.2015.2463671
4. Mur-Artal, R., and Tardós, J.D., Visual-inertial monocular SLAM with map reuse, IEEE Robotics and Automation Letters, 2017, vol. 2, no. 2, pp. 796–803. https://doi.org/10.1109/LRA.2017.2653359
5. Qin, T., Li, P., and Shen, S., VINS-Mono: A robust and versatile monocular visual-inertial state estimator, IEEE Transactions on Robotics, 2018, vol. 34, no. 4, pp. 1004–1020. https://doi.org/10.1109/TRO.2018.2853729.
6. Pumarola, A., Vakhitov, A., Agudo, A., Sanfeliu, A., and Moreno-Noguer, F., PL-SLAM: Real time monocular visual SLAM with points and lines, Proc. IEEE International Conference on Robotics and Automation (ICRA), 2017, pp. 4503–4508. https://doi.org/10.1109/ICRA.2017.7989522
7. Gioi, R., Jakubowicz, J., Morel. J.-M., and Randall, G., LSD: A fast line segment detector with a false detection control, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, vol. 32, no. 4, pp. 722–732. https://doi.org/10.1109/TPAMI.2008.300
8. Gomez-Ojeda, R., Briales, J., and Gonzalez-Jimenez, J., PL-SVO: Semi-direct monocular visual odometry by combining points and line segments, Proc. IEEE/RSJ International Conference on Intel ligent Robots and Systems (IROS), 2016, pp. 4211–4216. https://doi.org/10.1109/IROS.2016.7759620
9. He, Y.J., Zhao, J., Gao, Y., He, W., and Yuan, K., PL-VIO: Tightly-coupled monocular visual inertial odometry using point and line features, Sensors, 2018, vol. 18, no. 4, pp. 1159–1184. https://doi.org/10.3390/s18041159
10. Fu, Q., Wang, J.L., Yu, H.H., Islam, A., Guo, F., and Zhang, H., PL-VINS: Real-time monoc ular visual-inertial SLAM with point and line, arXiv, 2020, preprint arXiv:2009.07462. https://doi.org/10.48550/arXiv.2009.07462
11. Shu, F.W., Wang, J.X., and Pagani, A., Structure PLP-SLAM: Efficient sparse mapping and local ization using point, line, and plane for monocular, RGB-D and stereo cameras, ArXiv, 2022, preprint ArXiv 2207 06058, https://doi.org/10.48550/arXiv.2207.06058
12. Yoon, S., and Kim, A., Line as a visual sentence: Context-aware line descriptor for visual localization, IEEE Robotics and Automation Letters, 2021, vol. 6, no. 4, pp. 8726–8733. https://doi.org/10.1109/ LRA.2021.3111760.
13. Yunus, R., Li, Y., and Tombari, F., Manhattan SLAM: Robust planar tracking and mapping leveraging mixture of Manhattan frames, Proc. IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 6687–6693. https://doi.org/10.1109/ICRA48506.2021.9562030.
14. Wu, T.-H., and Chen, K.-W., LGC Net: Feature enhancement and consistency learning based on local and global coherence network for correspondence selection, Proc. IEEE International Conference on Robot ics and Automation (ICRA), 2023, pp. 6182–6188. https://doi.org/10.1109/ICRA48891.2023.10160290
15. Akinlar, C., and Topal, C., Edlines: Real-time line segment detection by edge drawing, Proc. 18th IEEE International Conference on Image Processing (ICIP), 2011, pp. 2837–2840. https://doi. org/10.1016/j.patrec.2011.06.001
16. Lupton, T., and Sukkarieh, S., Visual-inertial-aided navigation for high-dynamic motion in built environments without initial conditions, IEEE Transactions on Robotics, 2012, vol. 28, no. 1, pp. 61–76. https://doi.org/10.1109/TRO.2011.2170332
17. Forster, C., Carlone, L., Dellaert, F., and Scaramuzza, D., On-manifold pre-integration for re al-time visual-inertial odometry, IEEE Transactions on Robotics, 2017, vol. 33, no. 1, pp. 1–21. https://doi.org/10.1109/TRO.2016.2597321
18. Mur-Artal, R., and Tardós, J.D., ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras, IEEE Transactions on Robotics, 2017, vol. 33, no. 5, pp. 1255–1262. https://doi.org/10.1109/TRO.2017.2705103.
19. Kschischang, F.R., Frey, B.J., and Loeliger, H.-A., Factor graphs and the sum-product algorithm, IEEE Transactions on Information Theory, 2001, vol. 47, no. 2, pp. 498–519. https://doi.org/10.1109/18.910572.
20. Dellaert, F., and Kaess, M., Square root SAM: Simultaneous localization and mapping via square root information smoothing, International Journal of Robotics Research, 2006, vol. 25, no. 12, pp. 1181–1203. https://doi.org/10.1177/0278364906072768
21. Kümmerle, R., Grisetti, G., Strasdat, H.M., Konolige, K., and Burgard, W., G2o: A general framework for graph optimization, Proc. IEEE International Conference on Robotics and Automa tion (ICRA), 2011, pp. 3607–3613. http://doi.org/10.1109%2FICRA.2011.5979949
22. Leutenegger, S., Lynen, S., Bosse, M., Siegwart, R., and Furgale, P., Keyframe-based visual inertial odometry using nonlinear optimization, International Journal of Robotics Research, 2015, vol. 34, no. 3, pp. 314–334. https://doi.org/10.1177/0278364914554813
23. Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M.W., and Siegwart, R., The EuRoC micro aerial vehicle datasets, International Journal of Robotics Research, 2016, vol. 40, no. 6, pp. 1157–1163. https://doi.org/10.1177/0278364915620033
24. Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D., A benchmark for the eval uation of RGB-D SLAM systems, Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012, pp. 573–580. https://doi:10.1109/IROS.2012.6385773
25. Campos, C., Elvira, R., Rodríguez, J.J.G., Montiel, J.M.M., and Tardós, J.D., ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM, IEEE Transactions on Robotics, 2021, vol. 37, no. 6, pp. 1874–1890. https://doi.org/10.1109/TRO.2021.3075644
Review
For citations:
Yang G., Meng W., Hou G., Feng N. Real-time Visual-Inertial Odometry based on Point-Line Feature Fusion. Gyroscopy and Navigation. 2023;31(4):96-117. (In Russ.) EDN: RQVMYA
Views:
5
Email this article 