Deep car detection by fusing  grayscale image and weighted upsampled LiDAR depth

Meisam  Jamshidi Seikavandi; Kamal Nasrollahi; Thomas B. Moeslund

doi:10.1117/12.2586908

Deep car detection by fusing grayscale image and weighted upsampled LiDAR depth

Meisam Jamshidi Seikavandi, Kamal Nasrollahi, Thomas B. Moeslund

Research output: Contribution to book/anthology/report/conference proceeding › Article in proceeding › Research › peer-review

1 Citation (Scopus)

329 Downloads (Pure)

Abstract

Recent advances have shown sensor-fusion’s vital role in accurate detection, especially for advanced driver assistance
systems. We introduce a novel procedure for depth upsampling and sensor-fusion that together lead to an improved detection
performance, compared to state-of-the-art results for detecting cars. Upsampling is generally based on combining data
from an image to compensate for the low resolution of a LiDAR (Light Detector and Ranging). This paper, on the other
hand, presents a framework to obtain dense depth map solely from a single LiDAR point cloud that makes it possible to
use just one deep network for both LiDAR and image modalities. The produced full-depth map is added to the grayscale
version of the image to produce a two-channel input for a deep neural network. The simple preprocessing structure is
efficiently competent in filing cars’ shapes, which helps the fusion framework to outperforms the state-of-the-art on the
KITTI object detection for the Car class. Additionally, the combination of depth and image makes it easier for the network
to discriminate highly occluded and truncated vehicles.

Original language	English
Title of host publication	International Conference on Machine Vision
Number of pages	10
Publisher	SPIE - International Society for Optical Engineering
Publication date	2020
Chapter	1160524
ISBN (Electronic)	9-781510-640405
DOIs	https://doi.org/10.1117/12.2586908
Publication status	Published - 2020
Event	The 13th International Conference on Machine Vision - Rome, Italy Duration: 2 Nov 2020 → 6 Nov 2020

Conference

Conference	The 13th International Conference on Machine Vision
Country/Territory	Italy
City	Rome
Period	02/11/2020 → 06/11/2020

Series	Proceedings of SPIE, the International Society for Optical Engineering
Volume	11605
ISSN	0277-786X

Keywords

Sensor Fusion
Deep Learning
Object Detection
Autonomous Driving
Depth Perception
LiDAR

Access to Document

10.1117/12.2586908

icmv_Accepted author manuscript, 3.57 MB

AUB Link

Search for the material in Aalborg University Library's search engine

Cite this

@inproceedings{60cb1297f76745d9bba19a379ee3b28c,

title = "Deep car detection by fusing grayscale image and weighted upsampled LiDAR depth",

abstract = "Recent advances have shown sensor-fusion{\textquoteright}s vital role in accurate detection, especially for advanced driver assistancesystems. We introduce a novel procedure for depth upsampling and sensor-fusion that together lead to an improved detectionperformance, compared to state-of-the-art results for detecting cars. Upsampling is generally based on combining datafrom an image to compensate for the low resolution of a LiDAR (Light Detector and Ranging). This paper, on the otherhand, presents a framework to obtain dense depth map solely from a single LiDAR point cloud that makes it possible touse just one deep network for both LiDAR and image modalities. The produced full-depth map is added to the grayscaleversion of the image to produce a two-channel input for a deep neural network. The simple preprocessing structure isefficiently competent in filing cars{\textquoteright} shapes, which helps the fusion framework to outperforms the state-of-the-art on theKITTI object detection for the Car class. Additionally, the combination of depth and image makes it easier for the networkto discriminate highly occluded and truncated vehicles.",

keywords = "Sensor Fusion, Deep Learning, Object Detection, Autonomous Driving, Depth Perception, LiDAR",

author = "{Jamshidi Seikavandi}, Meisam and Kamal Nasrollahi and Moeslund, {Thomas B.}",

year = "2020",

doi = "10.1117/12.2586908",

language = "English",

series = "Proceedings of SPIE, the International Society for Optical Engineering",

publisher = "SPIE - International Society for Optical Engineering",

booktitle = "International Conference on Machine Vision",

address = "United States",

note = "The 13th International Conference on Machine Vision ; Conference date: 02-11-2020 Through 06-11-2020",

}

Jamshidi Seikavandi, M, Nasrollahi, K & Moeslund, TB 2020, Deep car detection by fusing grayscale image and weighted upsampled LiDAR depth. in International Conference on Machine Vision. SPIE - International Society for Optical Engineering, Proceedings of SPIE, the International Society for Optical Engineering, vol. 11605, The 13th International Conference on Machine Vision, Rome, Italy, 02/11/2020. https://doi.org/10.1117/12.2586908

Deep car detection by fusing grayscale image and weighted upsampled LiDAR depth. / Jamshidi Seikavandi, Meisam; Nasrollahi, Kamal ; Moeslund, Thomas B.
International Conference on Machine Vision. SPIE - International Society for Optical Engineering, 2020. (Proceedings of SPIE, the International Society for Optical Engineering, Vol. 11605).

Research output: Contribution to book/anthology/report/conference proceeding › Article in proceeding › Research › peer-review

TY - GEN

T1 - Deep car detection by fusing grayscale image and weighted upsampled LiDAR depth

AU - Jamshidi Seikavandi, Meisam

AU - Nasrollahi, Kamal

AU - Moeslund, Thomas B.

PY - 2020

Y1 - 2020

N2 - Recent advances have shown sensor-fusion’s vital role in accurate detection, especially for advanced driver assistancesystems. We introduce a novel procedure for depth upsampling and sensor-fusion that together lead to an improved detectionperformance, compared to state-of-the-art results for detecting cars. Upsampling is generally based on combining datafrom an image to compensate for the low resolution of a LiDAR (Light Detector and Ranging). This paper, on the otherhand, presents a framework to obtain dense depth map solely from a single LiDAR point cloud that makes it possible touse just one deep network for both LiDAR and image modalities. The produced full-depth map is added to the grayscaleversion of the image to produce a two-channel input for a deep neural network. The simple preprocessing structure isefficiently competent in filing cars’ shapes, which helps the fusion framework to outperforms the state-of-the-art on theKITTI object detection for the Car class. Additionally, the combination of depth and image makes it easier for the networkto discriminate highly occluded and truncated vehicles.

AB - Recent advances have shown sensor-fusion’s vital role in accurate detection, especially for advanced driver assistancesystems. We introduce a novel procedure for depth upsampling and sensor-fusion that together lead to an improved detectionperformance, compared to state-of-the-art results for detecting cars. Upsampling is generally based on combining datafrom an image to compensate for the low resolution of a LiDAR (Light Detector and Ranging). This paper, on the otherhand, presents a framework to obtain dense depth map solely from a single LiDAR point cloud that makes it possible touse just one deep network for both LiDAR and image modalities. The produced full-depth map is added to the grayscaleversion of the image to produce a two-channel input for a deep neural network. The simple preprocessing structure isefficiently competent in filing cars’ shapes, which helps the fusion framework to outperforms the state-of-the-art on theKITTI object detection for the Car class. Additionally, the combination of depth and image makes it easier for the networkto discriminate highly occluded and truncated vehicles.

KW - Sensor Fusion

KW - Deep Learning

KW - Object Detection

KW - Autonomous Driving

KW - Depth Perception

KW - LiDAR

U2 - 10.1117/12.2586908

DO - 10.1117/12.2586908

M3 - Article in proceeding

T3 - Proceedings of SPIE, the International Society for Optical Engineering

BT - International Conference on Machine Vision

PB - SPIE - International Society for Optical Engineering

T2 - The 13th International Conference on Machine Vision

Y2 - 2 November 2020 through 6 November 2020

ER -