A saliency based heavy fixed pattern noise suppression algorithm for IR image which is captured by infrared focal-plane-array sensors in remote observation is proposed. The properties of the typical IR images characterized with small targets and heavy fixed pattern noise were analyzed, and it was found that the target regions are more salient than the background for observer in IR image. Using the saliency detection algorithm to detect target regions, the proposed algorithm processes different regions separately and suppresses the heavy fixed pattern noise effectively. Experiments indicate that our algorithm can detect the target regions accurately and suppress the heavy fixed pattern noise effectively.