Light Field Based Face Recognition Via A Fused Deep Representation

Alireza Sepas-Moghaddam, Paulo Lobato Correia, Kamal Nasrollahi, Thomas B. Moeslund, Fernando Pereira

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

1 Citation (Scopus)

Resumé

The emergence of light field cameras opens new frontiers in terms of biometric recognition. This paper proposes the first deep CNN solution for light field based face recognition, exploiting the richer information available in a lenslet light field image. Additionally, for the first time, the exploitation of disparity maps together with 2D-RGB images and depth maps has been considered in the context of a fusion scheme to further improve the face recognition performance. The proposed solution uses the 2D-RGB central sub-aperture view as well as the disparity and depth maps extracted from the full set of sub-aperture images associated to a lenslet light field. After, feature extraction is performed using a VGG-Face deep descriptor for texture and independently fine-tuned models for disparity and depth maps. Finally, the extracted features are concatenated to be fed into an SVM classifier. A comprehensive set of experiments has been conducted with the IST-EURECOM light field face database, showing the superior performance of the fused deep representation for varied and challenging recognition tasks.
OriginalsprogEngelsk
TitelIEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING
RedaktørerNelly Pustelnik, Zheng-Hua Tan, Zhanyu Ma, Jan Larsen
Antal sider6
ForlagIEEE
Publikationsdato31 okt. 2018
Artikelnummer8516966
ISBN (Trykt)978-153865477-4
ISBN (Elektronisk)9781538654774
DOI
StatusUdgivet - 31 okt. 2018
BegivenhedIEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING - Aalborg, Danmark
Varighed: 17 sep. 201820 sep. 2018

Konference

KonferenceIEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING
LandDanmark
ByAalborg
Periode17/09/201820/09/2018
NavnMachine Learning for Signal Processing
ISSN1551-2541

Fingerprint

Face recognition
Biometrics
Feature extraction
Classifiers
Fusion reactions
Textures
Cameras
Experiments

Citer dette

Sepas-Moghaddam, A., Correia, P. L., Nasrollahi, K., Moeslund, T. B., & Pereira, F. (2018). Light Field Based Face Recognition Via A Fused Deep Representation. I N. Pustelnik, Z-H. Tan, Z. Ma, & J. Larsen (red.), IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING [8516966] IEEE. Machine Learning for Signal Processing https://doi.org/10.1109/MLSP.2018.8516966
Sepas-Moghaddam, Alireza ; Correia, Paulo Lobato ; Nasrollahi, Kamal ; Moeslund, Thomas B. ; Pereira, Fernando. / Light Field Based Face Recognition Via A Fused Deep Representation. IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING. red. / Nelly Pustelnik ; Zheng-Hua Tan ; Zhanyu Ma ; Jan Larsen. IEEE, 2018. (Machine Learning for Signal Processing).
@inproceedings{a5afe9515b47414baf96cfbd64a0c554,
title = "Light Field Based Face Recognition Via A Fused Deep Representation",
abstract = "The emergence of light field cameras opens new frontiers in terms of biometric recognition. This paper proposes the first deep CNN solution for light field based face recognition, exploiting the richer information available in a lenslet light field image. Additionally, for the first time, the exploitation of disparity maps together with 2D-RGB images and depth maps has been considered in the context of a fusion scheme to further improve the face recognition performance. The proposed solution uses the 2D-RGB central sub-aperture view as well as the disparity and depth maps extracted from the full set of sub-aperture images associated to a lenslet light field. After, feature extraction is performed using a VGG-Face deep descriptor for texture and independently fine-tuned models for disparity and depth maps. Finally, the extracted features are concatenated to be fed into an SVM classifier. A comprehensive set of experiments has been conducted with the IST-EURECOM light field face database, showing the superior performance of the fused deep representation for varied and challenging recognition tasks.",
keywords = "Depth Map, Disparity Map, Face Recognition, Fine-Tuning, Lenslet Light Field Imaging, VGG-Face Descriptor",
author = "Alireza Sepas-Moghaddam and Correia, {Paulo Lobato} and Kamal Nasrollahi and Moeslund, {Thomas B.} and Fernando Pereira",
year = "2018",
month = "10",
day = "31",
doi = "10.1109/MLSP.2018.8516966",
language = "English",
isbn = "978-153865477-4",
editor = "Nelly Pustelnik and Zheng-Hua Tan and Zhanyu Ma and Jan Larsen",
booktitle = "IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING",
publisher = "IEEE",
address = "United States",

}

Sepas-Moghaddam, A, Correia, PL, Nasrollahi, K, Moeslund, TB & Pereira, F 2018, Light Field Based Face Recognition Via A Fused Deep Representation. i N Pustelnik, Z-H Tan, Z Ma & J Larsen (red), IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING., 8516966, IEEE, Machine Learning for Signal Processing, Aalborg, Danmark, 17/09/2018. https://doi.org/10.1109/MLSP.2018.8516966

Light Field Based Face Recognition Via A Fused Deep Representation. / Sepas-Moghaddam, Alireza ; Correia, Paulo Lobato; Nasrollahi, Kamal; Moeslund, Thomas B.; Pereira, Fernando.

IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING. red. / Nelly Pustelnik; Zheng-Hua Tan; Zhanyu Ma; Jan Larsen. IEEE, 2018. 8516966 (Machine Learning for Signal Processing).

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

TY - GEN

T1 - Light Field Based Face Recognition Via A Fused Deep Representation

AU - Sepas-Moghaddam, Alireza

AU - Correia, Paulo Lobato

AU - Nasrollahi, Kamal

AU - Moeslund, Thomas B.

AU - Pereira, Fernando

PY - 2018/10/31

Y1 - 2018/10/31

N2 - The emergence of light field cameras opens new frontiers in terms of biometric recognition. This paper proposes the first deep CNN solution for light field based face recognition, exploiting the richer information available in a lenslet light field image. Additionally, for the first time, the exploitation of disparity maps together with 2D-RGB images and depth maps has been considered in the context of a fusion scheme to further improve the face recognition performance. The proposed solution uses the 2D-RGB central sub-aperture view as well as the disparity and depth maps extracted from the full set of sub-aperture images associated to a lenslet light field. After, feature extraction is performed using a VGG-Face deep descriptor for texture and independently fine-tuned models for disparity and depth maps. Finally, the extracted features are concatenated to be fed into an SVM classifier. A comprehensive set of experiments has been conducted with the IST-EURECOM light field face database, showing the superior performance of the fused deep representation for varied and challenging recognition tasks.

AB - The emergence of light field cameras opens new frontiers in terms of biometric recognition. This paper proposes the first deep CNN solution for light field based face recognition, exploiting the richer information available in a lenslet light field image. Additionally, for the first time, the exploitation of disparity maps together with 2D-RGB images and depth maps has been considered in the context of a fusion scheme to further improve the face recognition performance. The proposed solution uses the 2D-RGB central sub-aperture view as well as the disparity and depth maps extracted from the full set of sub-aperture images associated to a lenslet light field. After, feature extraction is performed using a VGG-Face deep descriptor for texture and independently fine-tuned models for disparity and depth maps. Finally, the extracted features are concatenated to be fed into an SVM classifier. A comprehensive set of experiments has been conducted with the IST-EURECOM light field face database, showing the superior performance of the fused deep representation for varied and challenging recognition tasks.

KW - Depth Map

KW - Disparity Map

KW - Face Recognition

KW - Fine-Tuning

KW - Lenslet Light Field Imaging

KW - VGG-Face Descriptor

UR - http://www.scopus.com/inward/record.url?scp=85057018492&partnerID=8YFLogxK

U2 - 10.1109/MLSP.2018.8516966

DO - 10.1109/MLSP.2018.8516966

M3 - Article in proceeding

SN - 978-153865477-4

BT - IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING

A2 - Pustelnik, Nelly

A2 - Tan, Zheng-Hua

A2 - Ma, Zhanyu

A2 - Larsen, Jan

PB - IEEE

ER -

Sepas-Moghaddam A, Correia PL, Nasrollahi K, Moeslund TB, Pereira F. Light Field Based Face Recognition Via A Fused Deep Representation. I Pustelnik N, Tan Z-H, Ma Z, Larsen J, red., IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING. IEEE. 2018. 8516966. (Machine Learning for Signal Processing). https://doi.org/10.1109/MLSP.2018.8516966