Speaker
Dr
George Lentaris
(National Technical University of Athens, Greece)
Description
Future planetary exploration will rely on autonomous space rovers with increased moving speed and highly accurate visual odometry algorithms. ESA estimates that the 2020+ Mars missions will involve stereo cameras on robots moving at 6cm/s and calculating high definition depth maps in less than 20sec. Moreover, the error of these processes shall be less than 2m when traveling 100m paths and less than 2cm when calculating the depth of objects located 4m away from the rover. Given the low processing power of the space-grade CPUs, executing such complex computer vision algorithms requires an excessive amount of time, e.g., minutes or even hours, i.e., much greater time than the aforementioned specifications. The solution being explored today is to use space-grade FPGA devices for accelerating the rover's visual processes with speed-up factors in the order of 10x to 1000x.
In the context of project SEXTANT of ESA, we perform HW/SW co-design to implement a number of computer vision algorithms on a custom HW/SW system consisting of a PC (emulating the space-grade CPU) and a Virtex-6 XC6VLX240t-2 FPGA device (emulating the space-grade FPGA). The developed algorithms are combined to form a robust visual odometry pipeline and a stereo vision pipeline used for the “localization” and “mapping” modes of operation of the space rover, respectively. The former inputs one 512x384 stereo image and outputs the pose of the rover once every second, whereas the latter inputs three 1120x1120 stereo pairs and outputs a 3153x1051 depth map once every 8.4 seconds. Our “localization” pipeline utilizes the Harris corner detector, the SIFT corner descriptor, a feature matching function, as well as functions for filtering and motion estimation. Our “mapping” pipeline utilizes the plane-sweep algorithm. The algorithms/pipelines are selected after extensive cost-performance evaluations. Their HW/SW partitioning relies on detailed profiling and algorithmic analysis, which considers various computational requirements (time, memory, arithmetic accuracy, etc) and the capabilities of the underlying HW. The outcome of such analysis allows us to determine those parts that must be accelerated by FPGA and those suitable for SW execution. Thus, we implement on FPGA the corner detection, description, matching, and stereo correspondence algorithms, which account for 96% to 99% of the total computations of the system. The proposed architecture bases on input data decomposition, resource reuse, pixel pipelining, parallel memory organization, parallel computation of mathematical formulas, and modification/optimization in algorithmic level. Moreover, we develop a custom CPU-FPGA communication scheme with raw Ethernet packets to facilitate the HW/SW co-processing. The HW modules are coded with parametric VHDL to allow tuning at the final stage of SEXTANT, which ultimately leads to a HW/SW system meeting all ESA requirements. In particular, our HW accelerators achieve significant speed-up factors with respect to the space-grade CPU: 63x for detection, 100x for description, 125-425x for matching, and up to 1111x for stereocorrespondence. The final HW/SW system achieves a speedup of 16x for localization and 444x for mapping mode, both with sufficient accuracy and HW cost (36% FPGA LUTs and 78% FPGA RAM utilization for localization, 10% LUTs and 57% FPGA RAM for mapping). Overall, the results of SEXTANT verify the necessity of using FPGA to perform efficient visual navigation of space rovers.
Primary author
Dr
George Lentaris
(National Technical University of Athens, Greece)
