Architectural solutions of on-chip Cache fault tolerance are considered as effective means for high persistent failure probabilities. However, less fault injection tools for on-chip Caches are available. Therefore, the CacheFI, a fault injection tool based on the full-system simulator Simics was proposed. A separate mechanism of the fault generation and injection was employed. Fault generation consists of stochastic distribution control, failure patterns and explosive timing. Fault injection was designed to focus on the requirement of repeatability and modularity. According to the experiments based on Simics and 15 benchmarks from SPEC CPU2000, it evaluates typical micro-architectural fault mechanisms, such as Buddy, MAEP(matching access and error pattern) etc. by injecting Cache faults with CacheFI. Consequently, it presents the weakness and issues of these typical mechanisms.