Chinese Sign Language Recognition Dataset

University of Science and Technology of China (USTC)

Multimedia Computing & Communication, SLR Group

Download (!!! Important )

You can download this dataset and find more details in this WEBPAGE (click to visit).

Introduction

We have two Chinese sign language datasets for isolated Sign Language Recognition and continuous Sign Language Recognition, respectively. Both datasets are collected with Kinect 2.0 by 50 signers. Each signer perfors 5 times for every word (sentence). The sign videos are recorded with 30fps. The distance between the signers and Kinect is about 1.5 meters. Each instance in both datasets contains RGB videos, depth videos, and 3D joints information of the signer.

Since the dataset is recorded with Microsoft Kinect, there are three data modalities available:

1. RGB videos with resolution of 1280 x 720 pixels and frame rate of 30 fps.
2. Depth videos with resolution of 512x 424 pixels and frame rate of 30 fps.
3. Twenty-five skeleton joints locations of each frame.

Isolated SLR

If you use this Chinese isolated SLR dataset in your research, please cite the following papers:

• Jihai Zhang, Wengang Zhou, Chao Xie, Junfu Pu, and Houqiang Li, "Chinese Sign Language Recognition with Adaptive HMM," IEEE International Conference on Multimedia and Expo (ICME), 2016.
• Junfu Pu, Wengang Zhou, and Houqiang Li, "Sign Language Recognition with Multi-modal Features," Pacific-Rim Conference on Multimedia (PCM), 2016.
• Tao Liu, Wengang Zhou, and Houqiang Li, "Sign Language Recognition with Long Short Term Memory," IEEE International Conference on Image Processing (ICIP), 2016.
• Junfu Pu, Wengang Zhou, Jihai Zhang, and Houqiang Li, "Sign Language Recognition Based on Trajectory Modeling with HMMs," International Conference on Multimedia Modelling (MMM), 2016.

The isolated SLR dataset contains 500 Chinese sign words. Each sign video is performed by 50 signers with 5 times. Hence, there are 250 instances for each sign word.

You can download the RGB and Depth videos of the dataset here Isolated SLR500 (RGB & Depth, 1 time each signer), or Isolated SLR500 (RGB, 5 times each signer) [password: rvux].
Check the instruction to download here. [How]
Besides, the signer's skeleton joint information is available here Isolated SLR500 (Joints). [password: fwt7]

Continuous SLR

If you use this Chinese continuous SLR dataset in your research, please cite the following papers:

• Junfu Pu, Wengang Zhou, and Houqiang Li, "Iterative Alignment Network for Continuous Sign Language Recognition," Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
• Hao Zhou, Wengang Zhou, and Houqiang Li, "Dynamic Pseudo Label Decoding for Continuous Sign Language Recognition," International Conference on Multimedia and Expo (ICME), 2019.
• Jie Huang, Wengang Zhou, Qilin Zhang, Houqiang Li and Weiping Li, "Video-based Sign Language Recognition without Temporal Segmentation," AAAI Conference on Artificial Intelligence (AAAI), 2018.

Besides, you can refer to the following papers for continuous SLR published by our group:

• Junfu Pu, Wengang Zhou, Hezhen Hu, and Houqiang Li, "Boosting Continuous Sign Language Recognition via Cross Modality Augmentation," ACM International Conference on Multimedia (ACM MM), 2020.
• Junfu Pu, Wengang Zhou, and Houqiang Li, "Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition," International Joint Conference on Artificial Intelligence (IJCAI), 2018.
• Dan Guo, Wengang Zhou, Meng Wang, and Houqiang Li, "Hierarchical LSTM for Sign Language Translation," AAAI Conference on Artificial Intelligence (AAAI), 2018.

The corpus of continuous SLR dataset contains 100 Chinese sentence. There are 250 instances (50signers x 5times) for each sentence.

You can download the RGB videos of the continuous SLR dataset here Continuous SLR100 (RGB). [password: ac52]

Contact

If you have any questions about the dataset and our papers, please feel free to contact us:

• Houqiang Li, Professor, USTC, lihq AT ustc.edu.cn
• Wengang Zhou, Professor, USTC, zhwg AT ustc.edu.cn
• Junfu Pu, Ph.D Candidate, USTC, pjh AT mail.ustc.edu.cn
• Hao Zhou, Ph.D Candidate, USTC, zhouh156 AT mail.ustc.edu.cn
• Hezhen Hu, Ph.D Candidate, USTC, alexhu AT mail.ustc.edu.cn
• Chengcheng Wei, Master student, USTC, ccwei AT mail.ustc.edu.cn

[Back to Homepage]