Visual Sign Language Research Group

University of Science and Technology of China (USTC)

Key Laboratory of Multimedia Computing and Communication

Introduction

Visual Sign Language Research Group targets on understanding visual sign language via computer vision techniques, leading by Prof. Houqiang Li and Prof. Wengang Zhou, at University of Science and Technology of China (USTC). We are working on isolated sign language recognition and continuous sign language recognition. We have published 15+ journal articles and conference papers, including IEEE TCSVT, CVPR, AAAI, IJCAI, ACM MM, ICME, etc.

Members

Faculty

Student

Alumni

Open Resources

• Isolated SLR500 and Continuous SLR100 for Chinese Sign Language Recognition (2015).
• PyTorch re-implementation of paper "Dilated Convolutional Network with Iterative Optimization for Coutinuous Sign Language Recognition" (IJCAI'18) [Code].

Publications

Conference papers:

1. Junfu Pu, Wengang Zhou, Hezhen Hu, and Houqiang Li, "Boosting Continuous Sign Language Recognition via Cross Modality Augmentation," ACM International Conference on Multimedia (ACM MM), 2020.
2. Hao Zhou, Wengang Zhou, Yun Zhou, and Houqiang Li, "Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition," AAAI Conference on Artificial Intelligence (AAAI), Oral paper, 2020.
3. Chengcheng Wei, Wengang Zhou, Junfu Pu, and Houqiang Li, "Deep Grammatical Multi-classifier for Continuous Sign Language Recognition," International Conference on Multimedia Big Data (BigMM), 2019.
4. Zhihao Zhang, Junfu Pu, Liansheng Zhuang, Wengang Zhou, and Houqiang Li, "Continuous Sign Language Recognition via Reinforcement Learning," International Conference on Image Processing (ICIP), Best Student Paper Finalists, 2019.
5. Junfu Pu, Wengang Zhou, and Houqiang Li, "Iterative Alignment Network for Continuous Sign Language Recognition," Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
6. Hao Zhou, Wengang Zhou, and Houqiang Li, "Dynamic Pseudo Label Decoding for Continuous Sign Language Recognition," International Conference on Multimedia and Expo (ICME), 2019.
7. Junfu Pu, Wengang Zhou, and Houqiang Li, "Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition," International Joint Conference on Artificial Intelligence (IJCAI), 2018.
8. Jie Huang, Wengang Zhou, Qilin Zhang, Houqiang Li and Weiping Li, "Video-based Sign Language Recognition without Temporal Segmentation," AAAI Conference on Artificial Intelligence (AAAI), 2018.
9. Dan Guo, Wengang Zhou, Meng Wang, and Houqiang Li, "Hierarchical LSTM for Sign Language Translation," AAAI Conference on Artificial Intelligence (AAAI), 2018.
10. Shuo Wang, Dan Guo, Wengang Zhou, Zheng-Jun Zha, and Meng Wang, "Connectionist Temporal Fusion for Sign Language Translation,” ACM International Conference on Multimedia (ACM MM), 2018.
11. Jihai Zhang, Wengang Zhou, Chao Xie, Junfu Pu, and Houqiang Li, "Chinese Sign Language Recognition with Adaptive HMM," IEEE International Conference on Multimedia and Expo (ICME), 2016.
12. Junfu Pu, Wengang Zhou, and Houqiang Li, "Sign Language Recognition with Multi-modal Features," Pacific-Rim Conference on Multimedia (PCM), 2016.
13. Tao Liu, Wengang Zhou, and Houqiang Li, "Sign Language Recognition with Long Short Term Memory," IEEE International Conference on Image Processing (ICIP), 2016.
14. Dan Guo, Wengang Zhou, Meng Wang, and Houqiang Li, "Sign Language Recognition based on Adaptive HMMs with Data Augmentation," IEEE International Conference on Image Processing (ICIP), 2016.
15. Junfu Pu, Wengang Zhou, Jihai Zhang, and Houqiang Li, "Sign Language Recognition Based on Trajectory Modeling with HMMs," International Conference on Multimedia Modelling (MMM), 2016.
16. Jie Huang, Wengang Zhou, Houqiang Li, and Weiping Li, "Sign Language Recognition using Real-Sense," IIEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP), 2015.
17. Jihang Zhang, Wengang Zhou, and Houqiang Li, "A New System forChinese Sign Language Recognition," IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP), 2015.
18. Jie Huang, Wengang Zhou, Houqiang Li, and Weiping Li, "Sign language recognition using 3D convolutional neural networks," IEEE International Conference on Multimedia and Expo (ICME), 2015.
19. Jihai Zhang, Wengang Zhou, and Houqiang Li, "A Threshold-based HMM-DTW Approach for Continuous Sign Language Recognition," International Conference on Internet Multimedia Computing and Service (ICIMCS), 2014.

Journal papers:

1. Chengcheng Wei, Jian Zhao, Wengang Zhou, and Houqiang Li, "Semantic Boundary Detection with Reinforcement Learning for Continuous Sign Language Recognition," IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2020.
2. Jie Huang, Wengang Zhou, Houqiang Li, and Weiping Li, "Attention based 3D-CNNs for Large-Vocabulary Sign Language Recognition," IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2018.

[Back to Homepage]