Occlusion robust and aware face recognition

Abstract

Occluded faces, due to accessories such as sunglasses and face masks, present a challenge for current face recognition systems. This thesis provides a comprehensive exploration of the issues caused by occlusions, particularly upper-face and lower-face obstructions, in real-world scenarios. The increased prevalence of sunglasses and face masks, the latter due to the COVID-19 pandemic, has amplified the importance of addressing these problems. In this thesis, the Real World Occluded Faces (ROF) dataset is gathered, a collection of faces experiencing both upper and lower face occlusions, serving as a critical resource for this area of study. Contrary to synthetic occlusion data, the ROF dataset provides an authentic representation of the issue, which our benchmark experiments have shown to be a significant impediment for even the most sophisticated deep face representation models. These models, while highly effective on synthetically occluded faces, exhibit substantial performance degradation when tested against the ROF dataset. This research comprises two distinct, yet interconnected sections. The first stresses the vital role of real-world data for the design and refinement of occlusion-robust face recognition models. Our experiments demonstrate the increased challenges posed by real-world occlusions in comparison to their synthetic counterparts. This insight allows us to gauge the performance and limitations of various model architectures under different occlusion conditions. The second section presents a novel, occlusion-robust, and occlusion-aware face recognition system, designed to increase performance on occlusions caused by sunglasses and masks, with minimal impact on generic face recognition performance. The system incorporates an occlusion-robust face recognition model, an occlusion-aware model, and an innovative layer integrating the outputs of these models to minimize occlusion effects. This unique configuration ensures the system's resilience to occlusions, focusing less on occluded regions and more on overall facial recognition. This thesis provides a thorough investigation of the challenges presented by occluded face recognition and proposes an innovative solution for the same. It underscores the necessity of utilizing real-world data for developing robust face recognition models and introduces a novel occlusion-aware face recognition system. This work has the potential to significantly enhance the performance of occluded face recognition methods in various real-world scenarios.