Deep car detection by fusing grayscale image and weighted upsampled LiDAR depth

Meisam Jamshidi Seikavandi, Kamal Nasrollahi, Thomas B. Moeslund

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

237 Downloads (Pure)


Recent advances have shown sensor-fusion’s vital role in accurate detection, especially for advanced driver assistance
systems. We introduce a novel procedure for depth upsampling and sensor-fusion that together lead to an improved detection
performance, compared to state-of-the-art results for detecting cars. Upsampling is generally based on combining data
from an image to compensate for the low resolution of a LiDAR (Light Detector and Ranging). This paper, on the other
hand, presents a framework to obtain dense depth map solely from a single LiDAR point cloud that makes it possible to
use just one deep network for both LiDAR and image modalities. The produced full-depth map is added to the grayscale
version of the image to produce a two-channel input for a deep neural network. The simple preprocessing structure is
efficiently competent in filing cars’ shapes, which helps the fusion framework to outperforms the state-of-the-art on the
KITTI object detection for the Car class. Additionally, the combination of depth and image makes it easier for the network
to discriminate highly occluded and truncated vehicles.
Original languageEnglish
Title of host publicationInternational Conference on Machine Vision
Number of pages10
PublisherSPIE - International Society for Optical Engineering
Publication date2020
ISBN (Electronic)9-781510-640405
Publication statusPublished - 2020
EventThe 13th International Conference on Machine Vision - Rome, Italy
Duration: 2 Nov 20206 Nov 2020


ConferenceThe 13th International Conference on Machine Vision
SeriesProceedings of SPIE, the International Society for Optical Engineering


  • Sensor Fusion
  • Deep Learning
  • Object Detection
  • Autonomous Driving
  • Depth Perception
  • LiDAR


Dive into the research topics of 'Deep car detection by fusing grayscale image and weighted upsampled LiDAR depth'. Together they form a unique fingerprint.

Cite this