Attention-Guided Adaptive Feature Pyramid Network with Fuzzy Edge Refinement for Robust Scene Text Segmentation

doi:10.23940/ijpe.26.04.p3.200208

Abstract

Abstract: Detecting and understanding visual semantics within natural scenes constitutes a pivotal research domain in pattern recognition and text analysis. Visual semantics involves understanding the pattern and layout of text in scene images. Text detection presents considerable challenges due to variations in size, color, font style, complex backgrounds, and low brightness levels. Although deep learning frameworks have significantly better performance over conventional techniques, the issue of text presented in arbitrary orientations amidst intricate backgrounds persists. This paper introduces a hybrid text segmentation method that tackles these issues by employing deep learning techniques. We propose a segmentation method that extracts features at multiple resolutions using VGG19 encoder unified with Attention Guided Adaptive Feature Pyramid Network (AG-AFPN) that captures richer multiscale representation. Furthermore, a Type-2 fuzzy logic approach serves to refine the edge map, thereby improving the accuracy of text boundary segmentation. For post-processing, Differential Binarization (DB) is applied to generate a precise binary mask from the network's output, thus enhancing segmentation performance by effectively managing variations in arbitrary text and cluttered backgrounds. The proposed method is assessed using multiple benchmark datasets, such as ICDAR 2013, ICDAR 2015, and Total Text.

Key words: scene text detection, text segmentation, VGG19, attention guided adaptive feature pyramid network, type-2 fuzzy logic, differential binarization

Rajeswari Reddy Patil and Aradhana D. Attention-Guided Adaptive Feature Pyramid Network with Fuzzy Edge Refinement for Robust Scene Text Segmentation [J]. Int J Performability Eng, 2026, 22(4): 200-208.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

[1] Wang K., Li G., Liu X., Yan J., Li S., and Huang H., 2018. Natural scene text detection based on MSER. In2018 3rd International Conference on Communications, Information Management and Network Security (CIMNS 2018), pp. 92-95.
[2] Zhu Y., and Du J., 2021. Textmountain: accurate scene text detection via instance segmentation.Pattern Recognition, 110, 107336.
[3] Cao D., Dang J., and Zhong Y., 2021. Towards accurate scene text detection with bidirectional feature pyramid network.Symmetry, 13(3), 486.
[4] Dai P., Zhang S., Zhang H., and Cao X., 2021. Progressive contour regression for arbitrary-shape scene text detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7393-7402.
[5] Zhang S.X., Zhu X., Yang C., Wang H., and Yin X.C., 2021. Adaptive boundary proposal network for arbitrary shape text detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1305-1314.
[6] Atallah A.S., Al-Saqqar F., and Alhusban S.A., 2020. A holistic model for recognition of handwritten arabic text based on the local binary pattern technique.
[7] Naiemi F., Ghods V., and Khalesi H., 2021. A novel pipeline framework for multi oriented scene text image detection and recognition.Expert Systems with Applications, 170, 114549.
[8] Huang M., Liu Y., Peng Z., Liu C., Lin D., Zhu S., Yuan N., Ding K., and Jin L., 2022. Swintextspotter: scene text spotting via better synergy between text detection and text recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4593-4603.
[9] Mahadevkar S., Patil S., and Kotecha K., 2024. Enhancement of handwritten text recognition using AI-based hybrid approach.MethodsX, 12, 102654.
[10] Zhu Y., Zhou Y., Wang C., Cao Y., Han J., Hou L., and Xu H., 2024. Unit: unifying image and text recognition in one vision encoder.Advances in Neural Information Processing Systems, 37, pp. 122185-122205.
[11] Nagaoka Y., Miyazaki T., Sugaya Y., and Omachi S., 2021. Text detection using multi-stage region proposal network sensitive to text scale.Sensors, 21(4), 1232.
[12] Wu G., Zeng Q., Zhao J., and Yang Z., 2023. Natural scene text detection algorithm based on the regional proposal. InJournal of Physics: Conference Series, 2562(1), 012015.
[13] Hari P., and Ghosh R., 2021. Text localization in scene images using faster R-CNN with double region proposal networks. InProceedings of the International Conference on Paradigms of Computing, Communication and Data Sciences: PCCDS 2020, pp. 739-749.
[14] Praneel A.V., and Rao T.S., 2023. Scene text detection using pyramid-based text proposal network and transformation component network.
[15] Mahajan S., Rani R., and Kamboj A., 2025. Deep learning-based modified-EAST scene text detector: insights from a novel multiscript dataset. International Journal on Document Analysis and Recognition (IJDAR),28(1), pp. 97-119.
[16] Liao M., Shi B., and Bai X., 2018. Textboxes++: A single-shot oriented scene text detector. IEEE Transactions on Image Processing,27(8), pp. 3676-3690.
[17] Shi X., Peng G., Shen X., and Zhang C., 2024. TextFuse: fusing deep scene text detection models for enhanced performance. Multimedia Tools and Applications,83(8), pp. 22433-22454.
[18] Duan C., Fu P., Guo S., Jiang Q., and Wei X., 2024. Odm: A text-image further alignment pre-training approach for scene text detection and spotting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15587-15597.
[19] Zhong D., Lyu S., Shivakumara P., Yin B., Wu J., Pal U., and Lu Y., 2022. Sgbanet: semantic gan and balanced attention network for arbitrarily oriented scene text recognition. InEuropean Conference on Computer Vision, pp. 464-480.
[20] Cheema Y., Cheema M.N., Nazir A., Khokhar F.A., Li P., and Ahmed A., 2025. A novel approach for improving open scene text translation with modified GAN. the Visual Computer,41(2), pp. 869-881.
[21] Turki H., Elleuch M., Kherallah M., and Damak A., 2023. Arabic-latin scene text detection based on YOLO models. In2023 International Conference on Innovations in Intelligent Systems and Applications (INISTA), pp. 1-6.
[22] Deng L., Gong Y., Lu X., Lin Y., Ma Z., and Xie M., 2019. STELA: A real-time scene text detector with learned anchor.IEEE Access, 7, pp. 153400-153407.
[23] Zhu A., Du H., and Xiong S., 2021. Scene text detection with selected anchors. In2020 25th International Conference on Pattern Recognition (ICPR), pp. 6608-6615.
[24] Dai Y., Huang Z., Gao Y., Xu Y., Chen K., Guo J., and Qiu W., 2018. Fused text segmentation networks for multi-oriented scene text detection. In2018 24th International Conference on Pattern Recognition (ICPR), pp. 3604-3609.
[25] Huang Z., Zhong Z., Sun L., and Huo Q., 2019. Mask R-CNN with pyramid attention network for scene text detection. In2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 764-772.
[26] Chaitra Y.L., Roopa M.J., Gopalakrishna M.T., Swetha M.D., and Aditya C.R., 2023. Text detection and recognition from the scene images using rcnn and easyocr. InInternational Conference on Information and Communication Technology for Intelligent Systems, pp. 75-85.
[27] Lyu P., Yao C., Wu W., Yan S., and Bai X., 2018. Multi-oriented scene text detection via corner localization and region segmentation. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7553-7563.
[28] Wang B., Xu J., Li J., Hu C., and Pan J.S., 2017. Scene text recognition algorithm based on faster RCNN. In2017 First International Conference on Electronics Instrumentation & Information Systems (EIIS), pp. 1-4.
[29] Zhong Z., Sun L., and Huo Q., 2019. An anchor-free region proposal network for faster R-CNN-based text detection approaches. International Journal on Document Analysis and Recognition (IJDAR),22(3), pp. 315-327.
[30] Duan P., Pan J., and Rao W., 2020. Masks r-cnn text detector. In2020 IEEE International Conference on Artificial Intelligence and Information Systems (ICAIIS), pp. 5-8.
[31] Zhu Y., and Zhang H., 2019. Curved scene text detection based on mask R-CNN. InInternational Conference on Image and Graphics, pp. 505-517.
[32] Kang J., Ibrayim M., and Hamdulla A., 2022. MR-FPN: multi-level residual feature pyramid text detection network based on self-attention environment.Sensors, 22(9), 3337.
[33] Zeng C., Liu Y., and Song C., 2022. Rwin-FPN++: rwin transformer with feature pyramid network for dense scene text spotting.Applied Sciences, 12(17), 8488.
[34] Chen M., Ibrayim M., and Hamdulla A., 2022. AAF-net: scene text detection based on attention aggregation features.PloS One, 17(8), e0272322.
[35] Sen P., Das A., and Sahu N., 2021. End-to-end scene text recognition system for devanagari and bengali text. InInternational Conference on Intelligent Computing & Optimization, pp. 352-359.
[36] Ghosh J., Talukdar A.K., and Sarma K.K., 2024. A light-weight natural scene text detection and recognition system. Multimedia Tools and Applications,83(3), pp. 6651-6683.
[37] Naim S., and Moumkine N., 2023. Semantic segmentation network for horizontal scene text detection. J. Inf. Hiding Multim. Signal Process.,14(4), pp. 148-157.
[38] Wang Q., Zheng Y., and Betke M., 2020. A method for detecting text of arbitrary shapes in natural scenes that improves text spotting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 540-541.
[39] Xie E., Zang Y., Shao S., Yu G., Yao C., and Li G., 2019. Scene text detection with supervised pyramid context network. In Proceedings of the AAAI Conference on Artificial Intelligence,33(01), pp. 9038-9045.
[40] Liu F., Chen C., Gu D., and Zheng J., 2019. FTPN: scene text detection with feature pyramid based text proposal network.IEEE Access, 7, pp. 44219-44228.