Video content is the basis for effective video management. As video is characterized by multi-modality and multi-level structure, this paper presents a 3-D model of video content. Furthermore, it proposes an open framework for video management, in which video data and the contents of video are stored and managed separately. The connection between video data and contents was maintained by an associating mechanism, through which multiple logical descriptions between video data and contents could be created. This framework is applicable to the management of different videos while users can search them with “multi-modal query” as well.