Multi-Sensor Mapping

2021

thumbs R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package FAST-LIO2 + VIO.

../_images/r3live.png

thumbs LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and Mapping (looks similar to V-LOAM + IMU) optimization with the following factors (with two system) :

  • Use lidar information in VINS (difference compared to VINS)
    • use lidar odometry pose for vins initialization
    • project lidar cloud to get vio feature depth neighbor pts model a plane,
    • feature depth in vins is anchored by the first observation. so here depth from lidar also valid only for first observation.
    • in marginalization if the marginalized pt has lidar depth, move its flags to the next.
    • set depth constant if has lidar depth.
  • Use vision in VIO-SAM (difference compared to VIO-SAM)
    • use visual loop detection result as lidar loop candidate (to further process ICP).
    • use VINS pose as initial pose for lidar registration.

2020

chrown0 CamVox, Lidar visual mapping using livox.

  • Livox generate dense lidar cloud, match visual edge with lidar intensity image edge for extrinsic parameters calibration.
  • IMU for lidar un-distortion.
  • Run ORBSLAM2 RGBD pipeline.

thumbs Augmenting Visual Place Recognition with Structural Cues Use both image (e.g. NetVLAD 2016) and 3d cloud (e.g. PointNetVLAD 2018) encoders.

unhappy Stereo Localization in LiDAR Maps . Localize stereo camera in pre-built lidar map.

  • Using ZED stereo camera, opencv (StereoSGBM and DisparityWLSFilter) to compute depth image.
  • Registration using Nelder-Mead method .

unhappy RGB2LIDAR: Towards Solving Large-Scale Cross-Modal Visual Localization <https://arxiv.org/abs/2009.05695> DL match rgb image and depth image (from lidar cloud)

unhappy Lidar-Monocular Visual Odometry using Point and Line Features (loosely coupled)

  • image -> point feature (ORB), line feature (LSD) -> project lidar to estimat depth -> odometry -> local BA current pose and landmarks.
  • ICP relative pose factors.
  • Global BA using ICP factors, ORB factors, LSD factors.

unhappy LIC-Fusion 2.0: LiDAR-Inertial-Camera Odometry with Sliding-Window Plane-Feature Tracking Tracking planes in the sliding window.

2019

unhappy CMRNet: Camera to LiDAR-Map Registration. Project a depth into plane (from an initial pose guess), CMRNet use RGB and depth as input, output 2D correspondings for each depth value. Finally PnP-RANSAC for pose estimation.

2018

thumbs LIMO: Lidar-Monocular Visual Odometry

  • Depth estiamtion : project lidar into image -> estimate local plane (select local range, foreground segmentation) -> check the depth.
  • Visual Odometry, global BA.

2017

question DSAC Differentiable RANSAC. replace non-differentiable parts of RANSAC algorithm with approximated differentiable parts (by soft argmax and probabilistic selection). Then make a deep learning DSAC. (As I understand, RANSAC is mathematically proved, I don’t understand how its accuracy can be improved).

Lidar Image

we normally have two types of systems :

  • lidar based : camera pose as initial estimation and as constrain. I personal perfer this one, since our system is generally lidar based. * project lidar to camera, and form a vio odometry system. It wastes lots of calculation, I don’t think it is necessary to maintain two slam system. * project camera information to lidar pts, to form lidar pts constrain. This seems more reasonable to me.
  • camera based : lidar project to camera to offer depth