A general hierarchical framework of news video process is presented. It divides the news video process into three levels: syntax segmentation level, semantic labeling level and abstraction level. Some key techniques related to these levels are described and solutions of them are introduced. The proposed framework overcomes the shortcomings of traditional news video process methods, which are limited to the content-based segmentation and process based on the single media feature. It acquires the semantic content by the analysis of audio-visual features synthetically. Experiments are carried out on a news video process prototype called NVPS, which validates the feasibility of the framework. Three methods, namely story detection, caption detection and anchor detection methods are tested on NVPS. The results reach to the detection precision of 88%, 86% and 86% respectively, which prove the efficiency of the layered framework in the semantic content analysis of news videos.