AI-based visual odometry implementation on an embedded system

Abstract

Navigation technology has always been a critical sub-field for defence technologies. The roots of navigation technology extend to early as 3000 BC year. The early examples of navigation technology were the observation of stars, bird following, etc. In modern days, Global Navigation Satellite Systems(GNSS) and Inertial Navigation System(INS) are widely used in defence technologies. The GNSS and INS systems are included nearly all the modern military platforms and smart munitions. However, they have also practical limitations. Therefore, a companion or alternative to these systems may provide flexibility in the navigational technology for military platforms. Visual odometry is an emerging technique for navigation technologies. The visual odometry technique is a dead reckoning technique, and it is the process of relative pose change estimation from camera images. Processing camera images for visual odometry is a computationally heavy operation. As the military platforms vary between a large scale of different sizes, their power and computational requirements also vary. Also, for expendable platforms like smart munitions, the cost is another important requirement. Embedded visual odometry(VO) implementation may provide a low-power, low cost and small-size alternative or companion positioning system to GNSS and INS. Hence the embedded systems are memory scarce, in this work, a new low-memory footprint neural network-based visual odometry method that is implementable on embedded systems is introduced and evaluated. In this work, firstly, the existing literature for geometry-based and deep learning-based methods was examined. Due to robustness and energy efficiency advantages, it was decided to realize a deep learning-based method, which can be further classified as supervised and unsupervised methods. The supervised learning methods generally require the involvement of other sensors than images and recurrent neural networks. These requirements come with additional computational and power consumption. As a very low-power and real-time system was targeted for this work, an unsupervised learning approach was selected as a training framework. With the ideas from the lightweight convolutional neural networks literature, a neural network namely TinyVO was designed. As monocular techniques provide more robustness and cost advantage than binocular ones, it was decided to use a monocular visual odometry technique in this work. The TinyVO network was trained with a well-known scale consistent unsupervised learning framework. After the training, TinyVO's performance on the KITTI dataset was evaluated. To deploy the neural network, MAX78002 artificial intelligence microcontroller has been chosen as the embedded platform. As MAX78002 supports 8 bits weights, the TinyVO was quantized to 8 bits using the provided MAX78002 software toolset. Also, the toolset supports the known answer test functionality. The known answer test was realized by using a sample from the KITTI dataset. Then, the energy and time consumption per inference values was measured. The resulting neural network, TinyVO enables a monocular visual odometry solution with a low-memory footprint, low power, and reasonable accuracy. To the best of found knowledge, this is the first study that provides a microcontroller-based visual odometry solution.