UCap: A crowdsourcing application for the visually impaired and blind persons on Android smartphone

Abstract

© 2015 IEEE. One of the visual challenge problems that the blind faces is the consumer product identification with contextual and description information problem. In order to increase their independence in food shopping and other product recognition, we implement an Android application called UCap to assist the blind in this visual challenge problem. In the visually impaired and blind persons mode, UCap is a camera-base mobile application that identifies the consumer product of a captured image using the UCap annotated image database. The UCap annotated image database is created using the crowdsourcing paradigm. In the sighted user mode, the user can use UCap to capture an image of a consumer product, add its description, and upload them to the UCap annotated image database. The sighted user can add more images of the existing products in the database. The seed database contains 3,950 annotated images. We used Infrastructure-as-a-Service (IaaS) on MS Azure cloud server for the initial system testing and evaluation. With exception of the lower-than-expected accuracy of image identification, the application received high praised from the visually impaired and blind persons. However, from the experiment and testing, the accuracy of UCap can be increased as more images are added to the database.