A novel method based on generative model was proposed for human behavior recognition. The behavior was represented by using a set of descriptors computed from key point trajectories, which included the orientationmagnitude descriptor, the trajectory shape descriptor and the appearance descriptor. In order to reduce feature dimensions, the agglomerative information bottleneck approach was used for vocabulary compression. The semi-supervised learning method for behavior recognition based on generative model was proposed to solve the problem of small sample in recognition, which made use of both the labeled and unlabeled samples. Compared with other state-of-the-art methods in both UCF sports database and YouTube database, results show that the proposed method has higher recognition accuracy.