JoWUA

Abstract

This study is constructed based on the prior text detection and recognition of natural images to extract text and provides ample enhancements to meet the demands of the complex visual situations. In particular, the proposed framework focuses on improving robustness and adaptability in real-world environments. Our system that is based on a combination of transformer-based architectures along with multi-modal fusion strategies makes detection and recognition successful in TMT. The integration of these advanced techniques enables better contextual understanding and feature representation. The approach uses ViT structure as a backbone and also employs Cross-Modal Attention Module (CMAM) to effectively use the information presented in both visual and semantic perspectives. This dual-stream processing enhances both localization and recognition accuracy. Experimental results show the significant improvement in accuracy, with an average precision of 96.8% in detection of text and an accuracy of 94.3% in recognition of characters, which are 4.5% and 5.9% better than those of the previous work. These improvements demonstrate the effectiveness of the proposed architecture over existing baseline methods. The strengthened framework demonstrates remarkable robustness to challenging scenarios with extreme lighting conditions, severe occlusions, and strong stylized text. Furthermore, it generalizes well across diverse datasets and conditions. In addition, the overall end-to-end inferencing speed of the system has been fine-tuned to approximately 52ms per image, which can be applied for real-time applications. This ensures practical deployment feasibility in time-sensitive systems. This paper sets new state-of-the-art benchmarks in scene text understanding and retrieval, with significant potential to boost applications in automatic document processing, aiding devices for the visually impaired, and augmented reality devices.