Work

A vision transformer-based hybrid neural architecture for automated handwritten Bangla character recognition and braille conversion

Year: 2025

Type: article

Abstract: The rapid advancement of technology has led to notable changes in the current educational system. Nevertheless, despite technological advancements, there are still relatively few assisting aids that can help in teaching individuals with disabilities, such as those who are blind or visually impaired. One of the very few effective teaching strategies for those who are blind or visually impaired is braille. Although it has been digitized to produce an electronic version, handwritten characters are not considered in those versions. Studies on English character recognition have shown a notably high level of accuracy, which is not the case with Bangla character identification. We present an automated system that converts handwritten Bangla characters to braille using a novel hybrid deep neural network architecture. Our approach begins with a novel Character Quality Assessment Framework (CQAF), which employs adaptive thresholds and comprehensive quality metrics designed explicitly for Bangla script characteristics. Building upon this foundation, we develop a unique triple-stream architecture combining ResNet, EfficientNet, and Vision Transformers to capture local and global features effectively. To complete the system, we implement a comprehensive accessibility solution featuring multilingual braille transcription and text-to-speech capabilities. The model effectively processes simple characters (vowels, consonants, and numerics) and compound Bangla characters. When compared with traditional transfer learning approaches using VGG16, ResNet50, and DenseNet121, our proposed model achieved the highest performance with an accuracy of 96.28% in training and 95.27% in validation.

Source: Knowledge-Based Systems

DOI: 10.1016/j.knosys.2025.114546