Depth estimation with ego-motion assisted monocular camera
https://doi.org/10.17285/0869-7035.2019.27.2.028-051
Abstract
We propose a method to estimate the distance to objects based on the complementary nature of monocular image sequences and camera kinematic parameters. The fusion of camera measurements with the kinematics parameters that are measured by an IMU and an odometer is performed using an extended Kalman filter. Results of field experiments with a wheeled robot corroborated the results of the simulation study in terms of accuracy of depth estimation. The performance of the approach in depth estimation is strongly affected by the mutual observer and feature point geometry, measurement accuracy of the observer’s motion parameters and distance covered by the observer. It was found that under favorable conditions the error in distance estimation can be as small as 1% of the distance to a feature point. This approach can be used to estimate distance to objects located hundreds of meters away from the camera.
About the Authors
M. Mansour
ITMO University, St. Petersburg, Russia; Tampere University, Finland
Russian Federation
Russian Federation
P. Davidson
Tampere University, Finland
Finland
Finland
O. A. Stepanov
Concern CSRI Elektropribor, JSC; ITMO University, St. Petersburg, Russia
Russian Federation
Russian Federation
J.-P. Raunio
AAC Technologies, Tampere, Finland
Finland
Finland
M.M. Aref
Tampere University, Finland
Finland
Finland
R. Piché
Tampere University, Finland
Russian Federation
Russian Federation
References
1. Hartley, R. and Zisserman, A., Multiple View Geometry in Computer Vision. Cambridge University Press, 2003.
2. Özyesil, O., Voroninski, V., Basri, R., and Singer, A., A survey of structure from motion, Acta Numerica, 2017, vol. 26, pp. 305–364.
3. Matthies, L., Szeliski, R., and Kanade, T., Incremental estimation of dense depth maps from image sequences, Proc. Computer Society Conf. on Computer Vision and Pattern Recognition, CVPR’88, IEEE, 1988, pp. 366–374.
4. Matthies, L., Dynamic stereo vision, PhD Thesis, Carnegie Mellon University, 1989.
5. Hernandez, J., Tsotsos, K., and Soatto, S., Observability, identifiability and sensitivity of vision-aided inertial navigation, Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2015, pp. 2319–2325.
6. Longuet-Higgins, H.C., and Prazdny, K., The interpretation of a moving retinal image, Proc. of the Royal Society London Series B, 1980, vol. 208, no. 1173, pp. 385–397.
7. Corke, P., An inertial and visual sensing system for a small autonomous helicopter, Journal of Field Robotics, 2004, vol. 21, no. 2, pp. 43–51.
8. Strelow, D. and Singh, S., Online motion estimation from image and inertial measurements, Proc. Workshop on Integration of Vision and Inertial Sensors (INERVIS), Coimbra, Portugal, June 2003.
9. Strelow, D. and Singh, S., Motion estimation from image and inertial measurements, International Journal of Robotics Research, 2004, vol. 23, no. 12, pp. 1157–1195.
10. Corke, P., Lobo, J., and Dias, J., An introduction to inertial and visual sensing, International Journal of Robotics Research, 2007, vol. 26, no. 6, pp. 519–535.
11. Grabe, V., Bulthoff, H.H., and Giordano, P.R., A comparison of scale estimation schemes for a quadrotor UAV based on optical flow and IMU measurements, Proc. Int. Conf. Intelligent Robots and Systems (IROS), IEEE, 2013, pp. 5193–5200.
12. Delmerico, J. and Scaramuzza, D., A benchmark comparison of monocular visual-inertial odometry algorithms for flying robots, Memory, 2018, vol. 10, p. 20.
13. Huster, A. and Rock, S.M., Relative position sensing by fusing monocular vision and inertial rate sensors, Proc. 11th Int. Conf. Advanced Robotics (ICAR’03), Coimbra, Portugal, 2003, vol. 3, pp. 1562–1567.
14. Huster A., Relative position sensing by fusing monocular vision and inertial rate sensors, PhD Thesis, Stanford University, 2003.
15. Dani, A., Fischer, N., Kan Z., and Dixon, W., Globally exponentially stable observer for vision-based range estimation, Mechatronics, 2012, vol. 22, no. 4, pp. 381–389.
16. Spica, R., Giordano, P.R., and Chaumette, F., Plane estimation by active vision from point features and image moments, Proc. Int. Conf. Robotics and Automation (ICRA), 2015, pp. 6003–6010.
17. Spica, R., Giordano, P.R., and Chaumette, F., Active structure from motion: Application to point, sphere, and cylinder, IEEE Transactions on Robotics, 2014, vol. 30, no. 6, pp. 1499–1513.
18. Tahri, O., Boutat, D., and Mezouar, Y., Brunovsky’s linear form of incremental structure from motion, IEEE Transactions on Robotics, 2017, vol. 33, no. 6, pp. 1491–1499.
19. Martinelli, A., Vision and IMU data fusion: Closed-form solutions for attitude, speed, absolute scale, and bias determination, IEEE Transactions on Robotics, 2012, vol. 28, no. 1, pp. 44–60.
20. De Luca, A., Oriolo, G., and Robuffo Giordano, P., Feature depth observation for imagebased visual servoing: Theory and experiments, International Journal of Robotics Research, 2008, vol. 27, no. 10, pp. 1093–1116.
21. Davidson, P., Raunio, J.-P., and Piché, R., Monocular vision-based range estimation supported by proprioceptive motion, Gyroscopy and Navigation, 2017, vol. 8, no. 2, pp. 150–158.
22. Azuma, R.T., A survey of augmented reality, Presence: Teleoperators & Virtual Environments, 1997, vol. 6, no. 4, pp. 355–385.
23. Bloesch, M., Omari, S., Hutter, M., and Siegwart, R., Robust visual inertial odometry using a direct EKF-based approach, Proc. Int. Conf. Intelligent Robots and Systems (IROS), IEEE, 2015, pp. 298–304.
24. Matthies, L., Kanade, T., and Szeliski, R., Kalman filter-based algorithms for estimating depth from image sequences, International Journal of Computer Vision, 1989, vol. 3, no.3, pp. 209–238.
25. Landy, M.S., Maloney, L.T., Johnston, E.B., and Young, M., Measurement and modeling of depth cue combination: In defense of weak fusion, Vision Research, 1995, vol. 35, no. 3, pp. 389–412.
26. Corke, P., Robotics, Vision and Control: Fundamental Algorithms in MATLAB, vol. 73. Springer Science & Business Media, 2011.
27. Montiel, J.M., Civera, J., and Davison, A.J., Unified inverse depth parametrization for monocular SLAM, Proc. Robotics: Science and Systems, 2006.
28. Davidson, P., Raunio, J.-P., and Piché, R., Accurate depth estimation from a sequence of monocular images supported by proprioceptive sensors, Proc. 23rd Int. Conf. on Integrated Navigation Systems, St. Petersburg, Russia, 2016, pp. 249–257.
29. Davidson, P., Mansour, M., Stepanov, O.A., and Piché, R., Depth estimation from motion parallax: Experimental evaluation, Proc. 26th Int. Conf. on Integrated Navigation Systems, St. Petersburg, Russia, 2019.
30. Stepanov, O.A., Optimal and sub-optimal filtering in integrated navigation systems, in Aerospace Navigation Systems; Nebylov, A., Watson, J., Eds., New York: John Wiley & Sons, Inc., 2016, pp. 392–446.
31. Särkkä, S., Bayesian Filtering and Smoothing, Cambridge University Press, 2010.
32. Oshman, Y. and Davidson, P., Optimal observer trajectories for passive target localization using bearing-only measurements, Proc. AIAA Guidance, Navigation, and Control Conference, San-Diego, CA, 1996, p. 3740.
33. Oshman, Y. and Davidson, P., Optimization of observer trajectories for bearings-only target localization, IEEE Transactions on Aerospace and Electronic Systems, 1999, vol. 35, no. 3, pp. 892–902.
34. Bouguet, J.-Y., Camera Calibration Toolbox for Matlab, California Institute of Technology, Pasadena, CA, 2006.
35. Geiger, A., Moosmann, F., Car, O., and Schuster, B., Automatic camera and range sensor calibration using a single shot, Proc. IEEE Int. Conf. Robotics and Automation, 2012, pp. 3936–3943.
36. Shi, J. et al., Good features to track, Proc. Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, 1994, pp. 593–600.
37. Lucas, B.D. and Kanade, T., An iterative image registration technique with an application to stereo vision, Proc. of Imaging Understanding Workshop, Vancouver, BC, Canada, 1981, pp. 121–130.
38. Bouguet, J.-Y., Pyramidal implementation of the affine Lucas-Kanade feature tracker: description of the algorithm, Intel Corporation, 2001, vol. 5, no. 1-10, p. 4.
39. Gao, X.-S., Hou, X.-R., Tang, J. and Cheng, H.-F., Complete solution classification for the perspective-three-point problem, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, vol. 25, pp. 930–943.
Review
For citations:
Mansour M., Davidson P., Stepanov O.A., Raunio J., Aref M., Piché R. Depth estimation with ego-motion assisted monocular camera. Giroskopiya i Navigatsiya. 2019;27(2):28-51. (In Russ.) https://doi.org/10.17285/0869-7035.2019.27.2.028-051
Views:
19
Email this article 