ImageEcho: A Lightweight Vision-to-Audio Aid for Real-Time Scene Understanding in the Visually Impaired

Abstract

Visually impaired individuals face significant challenges in performing day-to-day activities independently, such as reading magazines or newspapers, enjoying natural views and sceneries, or traveling from one place to another, often requiring assistance. In the modern digital era, even basic information like bank statements is delivered via mobile text messages, which are not easily accessible to those with visual impairments. Furthermore, the quick development in social media has given rise to a sea of image-based content. The visually impaired users, therefore, find it more to have a clear visual context. The possible solution of this paper is a technical framework aimed to allow people with visual impairment to identify images and understand the text contained in it, besides their being taken during the navigation or exploration of the surroundings. The approach taken in the system involves the application of deep learning through the combination of Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and transfer learning to create a textual identification of the image from the sensors. Then, the text is converted to speech using a TTS API, so users can immediately hear an audio summary of the image. The very heart of the mechanism consists in the condition that the image-to-text conversion is well ended.To ensure effective scene classification, the model will be trained on a comprehensive dataset containing both indoor and outdoor images. Designed with minimal computational overhead, the proposed system can be implemented on existing smartphones without the need for additional hardware. Users can simply use their smartphone’s camera to capture images automatically during travel, which will then be processed and classified to provide near-instant audio feedback. This system has the potential to significantly enhance the independence and quality of life for visually impaired individuals.