JoWUA

Abstract

Mobile apps have grown into an essential element of daily life, offering straightforward access to communication, services, and data for every segment of society. In this paper, a mobile app tailored for disabled people, enhancing accessibility, communication, and independent living via smart and intuitive functionalities. Sign Language is the significant language for hearing impaired people. The mobile app focuses mainly on sign language translation especially for providing legal assistance with 13 features. This app also assists visually impaired people. The proposed system facilitates on-the-fly recognition of sign language on mobile devices through the use of CameraX for capturing live video and the TensorFlow Lite Task Library for streamlined on-device processing. Fundamentally, a quantized TensorFlow Lite model offers fast, low-latency predictions tailored for mobile settings. This method guarantees instant responsiveness, rendering it extremely appropriate for real-time assistive communication uses. Basically, Computer Vision Modules-Edge AI, Generative AI & NLP Modules-Cloud AI, Static Analysis & Interactive user-friendly modules are developed to assist disabled people. Finally, comparison of six modules over inference type, average, resource impact are presented in this paper. The sign language recognition module for static gestures achieved 99.3% accuracy; the dynamic prototype model achieved approximately 100% under controlled conditions; however, videotaping, dynamic translation is only prototype model not for generalizations. People who suffer from deafness and hearing impairment experience difficulties when accessing legal services due to the use of sign languages and need to use an interpreter to interpret the content to them, whereas visually impaired people have no affordable and user-friendly assistive devices. In order to solve such problems, a mobile application is designed using a hybrid edge-cloud architecture based on Edge AI and Generative AI. The application uses CameraX for capturing videos in real-time, and TensorFlow Lite Task Library for performing the inference task using MLP with the accuracy of 99.3% on 150 legal sign classes with inference latency less than 30 ms. EfficientDet-Lite2 performs object detection with the speed of about 45 ms per frame (29.7% mAP), and ML Kit OCR recognizes text in less than 200 ms. The Groq-hosted LLaMA 3.1 (8B) model performs the translation of English-Telugu bilingual legal chat in round trip latency of 2–3 seconds.