대한언어학회 전자저널

Lee, Yong-hun. (2025). Challenges in deep learning-based analysis of Korean sign language: Through the lens of American sign language research. The Linguistic Association of Korea Journal, 33(2), 115-135. Sign language is a fully developed linguistic system using visual-gestural elements such as hand movements, facial expressions, and spatial organization. While deep learning has advanced American Sign Language (ASL) research, applying these methods to Korean Sign Language (KSL) faces challenges due to KSL's classifier predicates, spatial referencing, and topic-comment structures. This paper critically reviews ASL-based deep learning in Sign Language Recognition (SLR), Sign Language Production (SLP), and Sign Language Translation (SLT) to assess their adaptation for KSL. In this review, SLR covers the automatic recognition of sign sequences from visual input, SLP addresses the generation of natural sign gestures from text or speech, and SLT focuses on translating between sign and spoken languages. Methodologically, we conduct a comparative literature review of state-of-the-art deep learning models, analyzing their architectures, training strategies, and evaluation metrics within each subfield (SLR, SLP, SLT). We examine linguistic differences between ASL and KSL, noting difficulties in gesture synthesis, spatial modeling, and non-manual feature integration. We highlight limitations of direct ASL-to-KSL model transfer and propose multi-modal learning, expanded datasets, and enhanced spatial encoding to advance KSL processing technologies.

# American sign language # Korean sign language # recognition # production # translation

• Akandeh, A. (2022). Sentence-level sign language recognition framework. arXiv Preprint arXiv:2211.14447.
• Bilge, Y., Cinbis, R., & Ikizler-Cinbis, N. (2022). Towards zero-shot sign language recognition. arXiv Preprint arXiv:2201.05914.
• Camgoz, N., Koller, O., Hadfield, S., & Bowden, R. (2020). Sign language transformers: joint end-to-end sign language recognition and translation. arXiv Preprint arXiv:2003.13830.
• Cheng, K., Yang, Z., Chen, Q., & Tai, Y. (2020). Fully convolutional networks for continuous sign language recognition. arXiv Preprint arXiv:2007.12402.
• Fang, B., Co, J., & Zhang, M. (2018). DeepASL: Enabling ubiquitous and non-Intrusive word and sentence-level sign language translation. arXiv Preprint arXiv: 1802.07584.
• Fischer, S., & Gong, Q. (2010). Variation in East Asian sign language structures. In D. Brentari (Ed.), Sign languages (pp. 499-518). Cambridge: Cambridge University Press.
• Hu, H., Zhao, W., Zhou, W., Wang, Y., & Li. H. (2021). SignBERT: Pre-training of hand-model-aware representation for sign language recognition. arXiv Preprint arXiv:2110.05382.
• Jiang, Y. (2022). SDW-ASL: A dynamic system to generate large scale dataset for continuous American sign language. arXiv Preprint arXiv:2210.06791.
• Ko, S., Kim, C., Jung, H., & Cho, C. (2019). Neural sign language translation based on human keypoint estimation. arXiv Preprint arXiv:1811.11436.
• Lim, J., Sa, I., MacDonald, B., & Ahn, H. (2023). A sign language recognition system with pepper, lightweight-Transformer, and LLM. arXiv Preprint arXiv:2309.16898.
• Lucas, C., & Bayley, R. (2010). Variation in American sign language. In D. Brentari (Ed.), Sign languages (pp. 451-475). Cambridge: Cambridge University Press.
• Madhiarasan, M., & Roy, P. (2022). A comprehensive review of sign language recognition: Different types, modalities, and datasets. arXiv Preprint arXiv: 2204.03328.
• Moryossef, A., Jiang, Z., Muller, M., Ebling, S., & Goldberg. Y. (2023). Linguistically Motivated Sign Language Segmentation. arXiv Preprint arXiv:2310.13960.
• Rastgoo, R, Kiani, K., Escalera, S., & Sabokrou, M. (2021). Sign language production: A review. arXiv Preprint arXiv:2103.15910.
• Rastgoo, R., Kiani, K., & Escalera, S. (2024). A transformer model for boundary detection in continuous sign language. arXiv Preprint arXiv:2402.14720.
• Rastgoo, R., Kiani, K., Escalera, S., & Sabokrou, M. (2021). Multi-modal zero-shot sign language recognition. arXiv Preprint arXiv:2109.00796.
• Renz, K., Stache, N., Albanie, S., & Varol, G. (2021). Sign language segmentation with temporal convolutional networks. arXiv Preprint arXiv:2011.12986.
• Slimane, F., & Bouguessa, M. (2021). Context matters: Self-attention for sign language recognition. arXiv Preprint arXiv:2101.04632.
• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, Ł., & Polosukhin I. (2017). Attention is all you need. arXiv Preprint arXiv: 1706.03762.
• Yin, A., Zhao, Z., Liu, J., Jin, W., Zhang, M., Zeng, X., & He, X. (2021). SimulSLT: End-to-End simultaneous sign language translation. arXiv Preprint arXiv: 2112.04228.