Push the Limit of Single-Chip mmWave Radar-Based Egomotion Estimation with Moving Objects in FoV
Authors: 
Rong Ding, Haiming Jin, Jianrong Ding, Xiaocheng Wang, Guiyun Fan, Fengyuan Zhu, Xiaohua Tian, Linghe KongAuthors Info & Claims
Rong Ding, Haiming Jin, Jianrong Ding, Xiaocheng Wang, Guiyun Fan, Fengyuan Zhu, Xiaohua Tian, Linghe KongAuthors Info & ClaimsPages 417 - 430
Abstract
This paper presents EmoRI, a novel single-chip mmWave radar-based egomotion estimation approach that works in challenging scenarios where moving objects exist in radar's Field of View (FoV). Essentially, estimating a mobile platform's egomotion using an on-board mmWave radar requires inferring the relative motion between radar and the points of the stationary objects (PSOs) in the radar point cloud. However, in practice, there could be no PSOs because of the blockage of moving objects. Even if PSOs exist, precisely identifying them is still challenging due to (i) the large quantity of points generated by the moving objects, and (ii) the huge angle estimation errors of the conventional point cloud generation algorithm. We empower EmoRI to overcome the above challenges incurred by moving objects with three core techniques, which include (i) a hybrid FFT-MUSIC algorithm that improves the angle estimation accuracy of single-chip mmWave radar, (ii) a multiple stationary target consensus algorithm that precisely selects the PSOs from the radar point cloud, and (iii) a simultaneous fusion and calibration mechanism that introduces an IMU as the auxiliary sensor, meticulously calibrates IMU accelerations with radar measurements, and complimentarily fuses these two modalities to obtain the 6-DoF egomotion. Our extensive experiments validate that EmoRI pushes the limit of single-chip mmWave radar-based egomotion estimation with moving objects in radar's FoV by reducing the per-meter destination error from decimeter to centimeter level.
References
[1]
M. Ouyang, X. Shi, Y. Wang, Y. Tian, Y. Shen, D. Wang, P. Wang, and Z. Cao, "A collaborative visual slam framework for service robots," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021.
[2]
Elektrobit, "Vehicle position and ego-motion estimation for automotive applications," https://www.elektrobit.com/tech-corner/vehicle-position-and-ego-motion-estimation-for-automotive-applications/.
[3]
"Ar-hud," https://www.fic.com.tw/automotive/ar-hud/.
[4]
"Ar-hud: A heads up on the road ahead," https://blog.huawei.com/2022/01/04/ar-hud-road-ahead-transportation/.
[5]
MBUX Augmented Reality Head-up Display, "Mercedodes-benz," https://www.mercedes-benz.com.hk/en/passengercars/mercedes-benzcars/models/eqs/saloon-v297/equipment.pi.html/mercedes-benz-cars/models/eqs/saloon-v297/equipment/individualization/mbux-ar-hud.
[6]
G. Chi, Z. Yang, J. Xu, C. Wu, J. Zhang, J. Liang, and Y. Liu, "Wi-drone: wi-fi-based 6-dof tracking for indoor drone flight control," in The 20th Annual International Conference on Mobile Systems, Applications and Services (Mobisys), 2022.
[7]
M. Oelsch, M. Karimi, and E. Steinbach, "R-loam: Improving lidar odometry and mapping with point-to-mesh features of a known 3d reference object," IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 2068--2075, 2021.
[8]
J. Zhang and S. Singh, "LOAM: lidar odometry and mapping in real-time," in Robotics: Science and Systems X, 2014.
[9]
J. Lin and F. Zhang, "Loam livox: A fast, robust, high-precision lidar odometry and mapping package for lidars of small fov," in IEEE International Conference on Robotics and Automation, ICRA, 2020.
[10]
A. Tagliabue, J. Tordesillas, X. Cai, A. Santamaria-Navarro, J. P. How, L. Carlone, and A.-a. Agha-mohammadi, "Lion: Lidar-inertial observability-aware navigator for vision-denied environments," in Experimental Robotics: The 17th International Symposium. Springer, 2021, pp. 380--390.
[11]
W. Xu, Y. Cai, D. He, J. Lin, and F. Zhang, "Fast-lio2: Fast direct lidar-inertial odometry," IEEE Transactions on Robotics, vol. 38, no. 4, pp. 2053--2073, 2022.
[12]
Livox, "Livox hap," https://www.livoxtech.com/hap.
[13]
A. J. Davison, I. D. Reid, N. Molton, and O. Stasse, "Monoslam: Real-time single camera SLAM," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 1052--1067, 2007.
[14]
C. Wang, C. E. Thorpe, S. Thrun, M. Hebert, and H. F. Durrant-Whyte, "Simultaneous localization, mapping and moving object tracking," Int. J. Robotics Res., vol. 26, no. 9, pp. 889--916, 2007.
[15]
R. Clark, S. Wang, H. Wen, A. Markham, and N. Trigoni, "Vinet: Visual-inertial odometry as a sequence-to-sequence learning problem," in Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, S. Singh and S. Markovitch, Eds., 2017.
[16]
T. Qin, P. Li, and S. Shen, "Vins-mono: A robust and versatile monocular visual-inertial state estimator," IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1004--1020, 2018.
[17]
C. Forster, L. Carlone, F. Dellaert, and D. Scaramuzza, "On-manifold preintegration theory for fast and accurate visual-inertial navigation," IEEE Transactions on Robotics, pp. 1--18, 2015.
[18]
Texas Instruments Incorporated, "Single-chip 60-ghz to 64-ghz automotive radar sensor integrating antenna on package, dsp and mcu," https://www.ti.com/product/IWR6843AOP.
[19]
Texas Instruments, "mmwave radar sensors in robotics applications," https://www.ti.com/lit/pdf/spry311.
[20]
Junko Yoshida, "Audi a8: A look under the hood," https://www.eetindia.co.in/audi-a8-a-look-under-the-hood/.
[21]
Autel, "Autel evo max 4t millimeter-wave radar obstacle avoidance," https://www.autelpilot.com/blogs/news/autel-max-4t-millimeter-wave-radar-obstacle-avoidance.
[22]
D. Kellner, M. Barjenbruch, J. Klappstein, J. Dickmann, and K. C. J. Dietmayer, "Instantaneous ego-motion estimation using multiple doppler radars," in IEEE International Conference on Robotics and Automation, ICRA, 2014.
[23]
R. Schmidt, "Multiple emitter location and signal parameter estimation," IEEE transactions on antennas and propagation, vol. 34, no. 3, pp. 276--280, 1986.
[24]
P. Zhou, M. Li, and G. Shen, "Use it free: instantly knowing your phone attitude," in The 20th Annual International Conference on Mobile Computing and Networking (MobiCom), 2014.
[25]
C. X. Lu, M. R. U. Saputra, P. Zhao, Y. Almalioglu, P. P. B. de Gusmao, C. Chen, K. Sun, N. Trigoni, and A. Markham, "milliego: single-chip mmwave radar aided egomotion estimation via deep sensor fusion," in The 18th ACM Conference on Embedded Networked Sensor Systems (Sensys), 2020.
[26]
Y. Almalioglu, M. Turan, C. X. Lu, N. Trigoni, and A. Markham, "Milli-rio: Egomotion estimation with low-cost millimetre-wave radar," IEEE Sensors Journal., vol. 21, no. 3.
[27]
A. Kramer, C. Stahoviak, A. Santamaria-Navarro, A. Agha-mohammadi, and C. Heckman, "Radar-inertial ego-velocity estimation for visually degraded environments," in IEEE International Conference on Robotics and Automation, ICRA, 2020.
[28]
S. Palipana, D. Salami, L. A. Leiva, and S. Sigg, "Pantomime: Mid-air gesture recognition with sparse millimeter-wave radar point clouds," Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 5, no. 1, pp. 27:1--27:27, 2021.
[29]
K. Bansal, K. Rungta, S. Zhu, and D. Bharadia, "Pointillism: accurate 3d bounding box estimation with multi-radars," in The 18th ACM Conference on Embedded Networked Sensor Systems (Sensys), 2020.
[30]
X. Shuai, Y. Shen, Y. Tang, S. Shi, L. Ji, and G. Xing, "millieye: A lightweight mmwave radar and camera fusion system for robust object detection," in International Conference on Internet-of-Things Design and Implementation (IoTDI), 2021.
[31]
M. A. Fischler and R. C. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, vol. 24, no. 6, pp. 381--395, 1981.
[32]
Texas Instruments Incorporated, "Single-chip 76-ghz to 81-ghz mmwave sensor integrating dsp and mcu," https://www.ti.com/product/IWR1642/.
[33]
Texas Instruments Incorporated, "Single-chip 76-ghz to 81-ghz mmwave sensor integrating mcu and hardware accelerator," https://www.ti.com/product/IWR1443/.
[34]
M. A. Richards, Fundamentals of radar signal processing. McGraw-Hill Education, 2014.
[35]
M. Ester, H. Kriegel, J. Sander, and X. Xu, "A density-based algorithm for discovering clusters in large spatial databases with noise," in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD), 1996.
[36]
J. Diebel et al., "Representing attitude: Euler angles, unit quaternions, and rotation vectors," Matrix, vol. 58, no. 15--16, pp. 1--35, 2006.
[37]
WheelTac, "Ros robot," https://wheeltec.jd.com/.
[38]
Volkswagen, "Volkswagen passat sedan," https://www.volkswagen.com.au/en/models/passat.html.
[39]
"Matrice 210 user manual," https://dl.djicdn.com/downloads/M200/20201120/M200_User_Manual_EN_20201120.pdf.
[40]
Texas Instruments Incorporated, "Real-time data-capture adapter for radar sensing evaluation module," http://www.ti.com/tool/DCA1000EVM/.
[41]
Livox, "Automotive-grade lidar for serial production," https://www.livoxtech.com/.
[42]
Ouster, "Os1," https://ouster.com/products/scanning-lidar/os1-sensor/.
[43]
B. Sensortec, "High-performance inertial measurement unit (bmi088)," https://www.bosch-sensortec.com/media/boschsensortec/downloads/product_flyer/bst-bmi088-fl000.pdf.
[44]
TDK InvenSence, "Iam-20680ht," https://invensense.tdk.com/products/iam-20680ht/.
[45]
Y. S. Park, Y. Shin, J. Kim, and A. Kim, "3d ego-motion estimation using low-cost mmwave radars via radar velocity factor for pose-graph SLAM," IEEE Robotics Autom. Lett. (RA-L), vol. 6, no. 4, pp. 7691--7698, 2021.
[46]
Intel, "Intel nuc 11 pro kit nuc11tnhi30z," https://ark.intel.com/content/www/us/en/ark/products/121572/intel-nuc-11-pro-kit-nuc11tnhi30z.html.
[47]
C. Wu, F. Zhang, Y. Fan, and K. J. R. Liu, "Rf-based inertial measurement," in The 33th ACM Special Interest Group on Data Communication (SIGCOMM), 2019.
[48]
S. Shen, M. Gowda, and R. R. Choudhury, "Closing the gaps in inertial motion tracking," in The 24th Annual International Conference on Mobile Computing and Networking (MobiCom), 2018.
[49]
J. Gong, X. Zhang, Y. Huang, J. Ren, and Y. Zhang, "Robust inertial motion tracking through deep sensor fusion across smart earbuds and smartphone," Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. (IMWUT), vol. 5, no. 2, pp. 62:1--62:26, 2021.
[50]
S. H. Cen and P. Newman, "Precise ego-motion estimation with millimeter-wave radar under diverse and challenging conditions," in International Conference on Robotics and Automation, ICRA, 2018.
[51]
S. H. Cen and P. Newman, "Radar-only ego-motion estimation in difficult settings via graph matching," in International Conference on Robotics and Automation, ICRA, 2019.
[52]
A. Kingery and D. Song, "Improving ego-velocity estimation of low-cost doppler radars for vehicles," IEEE Robotics Autom. Lett. (RA-L), vol. 7, no. 4, pp. 9445--9452, 2022.
[53]
D. Adolfsson, M. Magnusson, A. W. Alhashimi, A. J. Lilienthal, and H. Andreasson, "Lidar-level localization with radar? the CFEAR approach to accurate, fast, and robust large-scale radar odometry in diverse environments," IEEE Trans. Robotics, vol. 39, no. 2, pp. 1476--1495, 2023.
[54]
R. Zhang, Y. Zhang, D. Fu, and K. Liu, "Scan denoising and normal distribution transform for accurate radar odometry and positioning," IEEE Robotics Autom. Lett., vol. 8, no. 3, pp. 1199--1206, 2023.
[55]
R. Weston, M. Gadd, D. D. Martini, P. Newman, and I. Posner, "Fast-mbym: Leveraging translational invariance of the fourier transform for efficient and accurate radar odometry," in 2022 International Conference on Robotics and Automation, (ICRA), 2022.
[56]
S.-Y. Kwon, S. Kwak, J. Kim, and S. Lee, "Radar sensor-based ego-motion estimation and indoor environment mapping," IEEE Sensors Journal, 2023.
[57]
NAVTECH, "Cts350-x," https://navtechradar.com/clearway-technical-specifications/.
[58]
Texas Instruments Incorporated, "mmwave cascade imaging radar rf evaluation module," https://www.ti.com/tool/MMWCAS-RF-EVM.
[59]
C. Doer and G. F. Trommer, "An EKF based approach to radar inertial odometry," in IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, (MFI), 2020.
[60]
J. Michalczyk, R. Jung, and S. Weiss, "Tightly-coupled ekf-based radar-inertial odometry," in IEEE/RSJ International Conference on Intelligent Robots and Systems, (IROS), 2022.
[61]
H. Chen, Y. Liu, and Y. Cheng, "DRIO: robust radar-inertial odometry in dynamic environments," IEEE Robotics Autom. Lett., vol. 8, no. 9, pp. 5918--5925, 2023.
[62]
E. B. Quist, P. C. Niedfeldt, and R. W. Beard, "Radar odometry with recursiveransac," IEEE Trans. Aerosp. Electron. Syst., vol. 52, no. 4, pp. 1618--1630, 2016.
[63]
Y. Sun, W. Wang, L. Mottola, R. Wang, and Y. He, "Aim: Acoustic inertial measurement for indoor drone localization and tracking," in The 20th ACM Conference on Embedded Networked Sensor Systems (Sensys), 2022.
[64]
W. Wang, L. Mottola, Y. He, J. Li, Y. Sun, S. Li, H. Jing, and Y. Wang, "Micnest: Long-range instant acoustic localization of drones in precise landing," in The 20th ACM Conference on Embedded Networked Sensor Systems (Sensys), 2022.
[65]
"Optitrack," https://optitrack.com/.
[66]
P. Zhao, C. X. Lu, B. Wang, N. Trigoni, and A. Markham, "3d motion capture of an unmodified drone with single-chip millimeter wave radar," in IEEE International Conference on Robotics and Automation (ICRA), 2021.
[67]
J. Xu, G. Chi, Z. Yang, D. Li, Q. Zhang, Q. Ma, and X. Miao, "Followupar: Enabling follow-up effects in mobile ar applications," in Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services (Mobisys), 2021.
[68]
R. S. Ayyalasomayajula, A. Arun, C. Wu, S. Rajagopalan, S. Ganesaraman, A. Seetharaman, I. K. Jain, and D. Bharadia, "Locap: Autonomous millimeter accurate mapping of wifi infrastructure," in 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI), R. Bhagwan and G. Porter, Eds., 2020.
[69]
B. Liang, P. Wang, R. Zhao, H. Guo, P. Zhang, J. Guo, S. Zhu, H. H. Liu, X. Zhang, and C. Xu, "Rf-chord: Towards deployable RFID localization system for logistic networks," in 20th USENIX Symposium on Networked Systems Design and Implementation, (NSDI), 2023.
[70]
J. Liu, J. Gao, S. K. Jha, and W. Hu, "Seirios: leveraging multiple channels for lorawan indoor and outdoor localization," in The 27th Annual International Conference on Mobile Computing and Networking (MobiCom), 2021.
[71]
I. Pefkianakis and K. Kim, "Accurate 3d localization for 60 ghz networks," in Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems (Sensys), 2018.
[72]
A. Angeli, S. Doncieux, J. Meyer, and D. Filliat, "Real-time visual loop-closure detection," in IEEE International Conference on Robotics and Automation, (ICRA), 2008.
Index Terms
- Push the Limit of Single-Chip mmWave Radar-Based Egomotion Estimation with Moving Objects in FoV
Recommendations
Moving target detection in foliage using along track monopulse synthetic aperture radar imaging
This paper presents a method for detecting moving targets embedded in foliage from the monostatic and bistatic synthetic aperture radar (SAR) data obtained via two airborne radars. The two radars, which are mounted on the same aircraft, have different ...
Interpersonal Distance Tracking with mmWave Radar and IMUs
IPSN '23: Proceedings of the 22nd International Conference on Information Processing in Sensor NetworksTracking interpersonal distances is essential for real-time social distancing management and ex-post contact tracing to prevent spreads of contagious diseases. Bluetooth neighbor discovery has been employed for such purposes in combating COVID-19, but ...
Effective baseline estimation in dual-channel synthetic aperture radar for moving target imaging and relocation
AbstractIn a dual-channel synthetic aperture radar (SAR) system with the phase centers being positioned along the track, the motion parameters can be obtained by the interferometric phase. When the phase centers of a dual-channel system ...
Comments
Information & Contributors
Information
Published In
November 2023
574 pages
ISBN:9798400704147
DOI:10.1145/3625687
- General Chair:
- Rasit Eskicioglu,
- Program Chair:
- Polly Huang,
- Program Co-chair:
- Neal Patwari
Copyright � 2023 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
- SIGARCH: ACM Special Interest Group on Computer Architecture
- SIGBED: ACM Special Interest Group on Embedded Systems
- SIGMETRICS: ACM Special Interest Group on Measurement and Evaluation
- SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing
- SIGOPS: ACM Special Interest Group on Operating Systems
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 April 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- NSF China
Conference
SenSys '23
SenSys '23: 21st ACM Conference on Embedded Networked Sensor Systems
November 12 - 17, 2023
Istanbul, Turkiye
Acceptance Rates
Overall Acceptance Rate 174 of 867 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 213Total Downloads
- Downloads (Last 12 months)213
- Downloads (Last 6 weeks)32
Reflects downloads up to 21 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in