A lightweight deep learning approach to mouth segmentation in color images
Issued Date
2022-01-01
Resource Type
ISSN
26341964
eISSN
22108327
Scopus ID
2-s2.0-85143216535
Journal Title
Applied Computing and Informatics
Rights Holder(s)
SCOPUS
Bibliographic Citation
Applied Computing and Informatics (2022)
Suggested Citation
Chotikkakamthorn K., Ritthipravat P., Kusakunniran W., Tuakta P., Benjapornlert P. A lightweight deep learning approach to mouth segmentation in color images. Applied Computing and Informatics (2022). doi:10.1108/ACI-08-2022-0225 Retrieved from: https://repository.li.mahidol.ac.th/handle/20.500.14594/84331
Title
A lightweight deep learning approach to mouth segmentation in color images
Author's Affiliation
Other Contributor(s)
Abstract
Purpose: Mouth segmentation is one of the challenging tasks of development in lip reading applications due to illumination, low chromatic contrast and complex mouth appearance. Recently, deep learning methods effectively solved mouth segmentation problems with state-of-the-art performances. This study presents a modified Mobile DeepLabV3 based technique with a comprehensive evaluation based on mouth datasets. Design/methodology/approach: This paper presents a novel approach to mouth segmentation by Mobile DeepLabV3 technique with integrating decode and auxiliary heads. Extensive data augmentation, online hard example mining (OHEM) and transfer learning have been applied. CelebAMask-HQ and the mouth dataset from 15 healthy subjects in the department of rehabilitation medicine, Ramathibodi hospital, are used in validation for mouth segmentation performance. Findings: Extensive data augmentation, OHEM and transfer learning had been performed in this study. This technique achieved better performance on CelebAMask-HQ than existing segmentation techniques with a mean Jaccard similarity coefficient (JSC), mean classification accuracy and mean Dice similarity coefficient (DSC) of 0.8640, 93.34% and 0.9267, respectively. This technique also achieved better performance on the mouth dataset with a mean JSC, mean classification accuracy and mean DSC of 0.8834, 94.87% and 0.9367, respectively. The proposed technique achieved inference time usage per image of 48.12 ms. Originality/value: The modified Mobile DeepLabV3 technique was developed with extensive data augmentation, OHEM and transfer learning. This technique gained better mouth segmentation performance than existing techniques. This makes it suitable for implementation in further lip-reading applications.