A Fast System for Person Description Search in Videos
Issued Date
2022-01-01
Resource Type
Scopus ID
2-s2.0-85151644312
Journal Title
6th International Conference on Information Technology, InCIT 2022
Start Page
196
End Page
200
Rights Holder(s)
SCOPUS
Bibliographic Citation
6th International Conference on Information Technology, InCIT 2022 (2022) , 196-200
Suggested Citation
Yuenyong S. A Fast System for Person Description Search in Videos. 6th International Conference on Information Technology, InCIT 2022 (2022) , 196-200. 200. doi:10.1109/InCIT56086.2022.10067673 Retrieved from: https://repository.li.mahidol.ac.th/handle/20.500.14594/84300
Title
A Fast System for Person Description Search in Videos
Author(s)
Author's Affiliation
Other Contributor(s)
Abstract
Security CCTV cameras are important for public safety. These cameras record continuously 24/7 and produce a large amount of video data. If the videos are not reviewed immediately after an incident, it can be difficult and timeconsuming to find a specific person out of many hours of recording. In this work we present a system that can search for people that fit a textual description in a video. It utilizes a imagetext multimodal deep learning model to calculate the similarity between an image of a person against a text description and find the top matches. Normally this would require calculating the textimage similarity scores between one text description and every person in the video, which is O(n) in the number of people in the video and therefore impractical for real-time search. We propose a solution to this by pre-calculating embeddings of person images and applying approximate nearest neighbor vector search. At inference time, only one forward pass through the deep learning model is needed, the computational cost is therefore the time to embed a text description O (1), plus the time to perform an approximate nearest neighbor search O(log (n)). This makes realtime interactive search possible.