Complementary Cohort Strategy for Multimodal Face Pair Matching

Yunlian Sun, Kamal Nasrollahi, Zhenan Sun, Tieniu Tan

Research output: Contribution to journalJournal articleResearchpeer-review

3 Citations (Scopus)
379 Downloads (Pure)

Abstract

Face pair matching is the task of determining
whether two face images represent the same person. Due to
the limited expressive information embedded in the two face
images as well as various sources of facial variations, it becomes
a quite difficult problem. Towards the issue of few available
images provided to represent each face, we propose to exploit an
extra cohort set (identities in the cohort set are different from
those being compared) by a series of cohort list comparisons.
Useful cohort coefficients are then extracted from both sorted
cohort identities and sorted cohort images for complementary
information. To augment its robustness to complicated facial
variations, we further employ multiple face modalities owing
to their complementary value to each other for the face pair
matching task. The final decision is made by fusing the extracted
cohort coefficients with the direct matching score for all the
available face modalities. To investigate the capacity of each
individual modality on matching faces, the cohort behavior and
the performance achieved by using our complementary cohort
strategy, we conduct a set of experiments on two recently collected
multimodal face databases. It is shown that using different
modalities leads to different face pair matching performance. For
each modality, employing our cohort scheme significantly reduces
the equal error rate. By applying the proposed multimodal
complementary cohort strategy, we achieve the best performance
on our face pair matching task.
Original languageEnglish
JournalI E E E Transactions on Information Forensics and Security
Volume11
Issue number5
Pages (from-to)937-950
ISSN1556-6013
DOIs
Publication statusPublished - 24 Feb 2016

Fingerprint

Experiments

Keywords

  • Face recognition
  • multimodal fusion
  • RGB-D
  • RGB-D-T
  • cohort information

Cite this

@article{777c330e25904fba88203781faafa6f1,
title = "Complementary Cohort Strategy for Multimodal Face Pair Matching",
abstract = "Face pair matching is the task of determiningwhether two face images represent the same person. Due tothe limited expressive information embedded in the two faceimages as well as various sources of facial variations, it becomesa quite difficult problem. Towards the issue of few availableimages provided to represent each face, we propose to exploit anextra cohort set (identities in the cohort set are different fromthose being compared) by a series of cohort list comparisons.Useful cohort coefficients are then extracted from both sortedcohort identities and sorted cohort images for complementaryinformation. To augment its robustness to complicated facialvariations, we further employ multiple face modalities owingto their complementary value to each other for the face pairmatching task. The final decision is made by fusing the extractedcohort coefficients with the direct matching score for all theavailable face modalities. To investigate the capacity of eachindividual modality on matching faces, the cohort behavior andthe performance achieved by using our complementary cohortstrategy, we conduct a set of experiments on two recently collectedmultimodal face databases. It is shown that using differentmodalities leads to different face pair matching performance. Foreach modality, employing our cohort scheme significantly reducesthe equal error rate. By applying the proposed multimodalcomplementary cohort strategy, we achieve the best performanceon our face pair matching task.",
keywords = "Face recognition, multimodal fusion, RGB-D, RGB-D-T, cohort information",
author = "Yunlian Sun and Kamal Nasrollahi and Zhenan Sun and Tieniu Tan",
year = "2016",
month = "2",
day = "24",
doi = "10.1109/TIFS.2015.2512561",
language = "English",
volume = "11",
pages = "937--950",
journal = "I E E E Transactions on Information Forensics and Security",
issn = "1556-6013",
publisher = "IEEE",
number = "5",

}

Complementary Cohort Strategy for Multimodal Face Pair Matching. / Sun, Yunlian; Nasrollahi, Kamal; Sun, Zhenan; Tan, Tieniu.

In: I E E E Transactions on Information Forensics and Security, Vol. 11, No. 5, 24.02.2016, p. 937-950.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - Complementary Cohort Strategy for Multimodal Face Pair Matching

AU - Sun, Yunlian

AU - Nasrollahi, Kamal

AU - Sun, Zhenan

AU - Tan, Tieniu

PY - 2016/2/24

Y1 - 2016/2/24

N2 - Face pair matching is the task of determiningwhether two face images represent the same person. Due tothe limited expressive information embedded in the two faceimages as well as various sources of facial variations, it becomesa quite difficult problem. Towards the issue of few availableimages provided to represent each face, we propose to exploit anextra cohort set (identities in the cohort set are different fromthose being compared) by a series of cohort list comparisons.Useful cohort coefficients are then extracted from both sortedcohort identities and sorted cohort images for complementaryinformation. To augment its robustness to complicated facialvariations, we further employ multiple face modalities owingto their complementary value to each other for the face pairmatching task. The final decision is made by fusing the extractedcohort coefficients with the direct matching score for all theavailable face modalities. To investigate the capacity of eachindividual modality on matching faces, the cohort behavior andthe performance achieved by using our complementary cohortstrategy, we conduct a set of experiments on two recently collectedmultimodal face databases. It is shown that using differentmodalities leads to different face pair matching performance. Foreach modality, employing our cohort scheme significantly reducesthe equal error rate. By applying the proposed multimodalcomplementary cohort strategy, we achieve the best performanceon our face pair matching task.

AB - Face pair matching is the task of determiningwhether two face images represent the same person. Due tothe limited expressive information embedded in the two faceimages as well as various sources of facial variations, it becomesa quite difficult problem. Towards the issue of few availableimages provided to represent each face, we propose to exploit anextra cohort set (identities in the cohort set are different fromthose being compared) by a series of cohort list comparisons.Useful cohort coefficients are then extracted from both sortedcohort identities and sorted cohort images for complementaryinformation. To augment its robustness to complicated facialvariations, we further employ multiple face modalities owingto their complementary value to each other for the face pairmatching task. The final decision is made by fusing the extractedcohort coefficients with the direct matching score for all theavailable face modalities. To investigate the capacity of eachindividual modality on matching faces, the cohort behavior andthe performance achieved by using our complementary cohortstrategy, we conduct a set of experiments on two recently collectedmultimodal face databases. It is shown that using differentmodalities leads to different face pair matching performance. Foreach modality, employing our cohort scheme significantly reducesthe equal error rate. By applying the proposed multimodalcomplementary cohort strategy, we achieve the best performanceon our face pair matching task.

KW - Face recognition

KW - multimodal fusion

KW - RGB-D

KW - RGB-D-T

KW - cohort information

U2 - 10.1109/TIFS.2015.2512561

DO - 10.1109/TIFS.2015.2512561

M3 - Journal article

VL - 11

SP - 937

EP - 950

JO - I E E E Transactions on Information Forensics and Security

JF - I E E E Transactions on Information Forensics and Security

SN - 1556-6013

IS - 5

ER -