This study proposes a spectrum resource optimization algorithm for UAV swarm communication tasks under dynamic changes and interference environments, based on a Convolutional Temporal Fusion Network. Specifically, the approach leverages the local feature extraction capability of Convolutional Neural Networks and the temporal modeling ability of Long Short-Term Memory networks to enhance the autonomous learning and adaptability of the UAV swarm. Moreover, by combining Double Deep Q-Learning and adopting a multi-agent framework for distributed online training, each UAV in the swarm can optimize spectrum resources reasonably based on its local observation, enabling a quick response to dynamic task changes and interference. Simulation results demonstrate that the proposed algorithm outperforms traditional methods in terms of spectrum resource utilization efficiency and exhibits excellent stability under dynamic changes and interference in communication tasks.