Abstract: On the issue of the representation model of the human motion 3D series, the most widely used methods were always based on numerical data. These methods could reduce the high dimensional 3D capture motion data and decrease the time complexity to a certain extent. However, the above mentioned traditional methods cannot extract the hidden useful domain physical knowledge, as well as meet the demands of current an intelligent computer processing on the numerical sequence. The present study proposed a new semantic annotation approach to obtain the linguistic tags on 3D motion data. Pre-processing on the human joint information should be implemented to appropriately achieve the spatio-temporal feature. The steps included: Constructing the human motion semantic category space, clustering the intermediate data through merging the kinematics knowledge and finally gaining the semantic annotations. At the end of the present study, the experiments showed that the proposed semantic approach could reasonably express the semantic information. In addition, there was also no absence of the essential domain knowledge of human motion data in the proposed method.