Complementary Cohort Strategy for Multimodal Face Pair Matching

Face pair matching is the task of determining
whether two face images represent the same person. Due to
the limited expressive information embedded in the two face
images as well as various sources of facial variations, it becomes
a quite difficult problem. Towards the issue of few available
images provided to represent each face, we propose to exploit an
extra cohort set (identities in the cohort set are different from
those being compared) by a series of cohort list comparisons.
Useful cohort coefficients are then extracted from both sorted
cohort identities and sorted cohort images for complementary
information. To augment its robustness to complicated facial
variations, we further employ multiple face modalities owing
to their complementary value to each other for the face pair
matching task. The final decision is made by fusing the extracted
cohort coefficients with the direct matching score for all the
available face modalities. To investigate the capacity of each
individual modality on matching faces, the cohort behavior and
the performance achieved by using our complementary cohort
strategy, we conduct a set of experiments on two recently collected
multimodal face databases. It is shown that using different
modalities leads to different face pair matching performance. For
each modality, employing our cohort scheme significantly reduces
the equal error rate. By applying the proposed multimodal
complementary cohort strategy, we achieve the best performance
on our face pair matching task.