Analyzing similarity of news video stories is crucial for searching, retrieving, browsing and tracking news video stories. A method based on “scene words” is proposed for measuring the similarity of news stories. Each key frame is seen as a “word” and a news story is seen as a document which is represented by a sequence of “scene words”. In light of this, the particularity of “scene words” is investigated. An effective method to describe weight of words and calculating similarity of stories is presented. Experiment results show that the method based on “scene words” is closer to the sensory and requirement of users.