Based on the concept of the intelligent combat of UAV (unmanned aerial vehicle) swarms, the UAV swarms intelligent combat simulation environment was established. Aiming at the problem that it is difficult to accurately control the speed and attack angle of UAVs in the confrontation process through reward signals in traditional reinforcement learning algorithms, the RIC-MADDPG (rule and intelligence coupling constrained multi-agent deep deterministic policy gradient) algorithm was proposed. The algorithm uses rules to constrain the actions of UAVs in reinforcement learning. The simulation results show that the wining-rate of red UAV swarm, trained by the method based on the RIC-MADDPG, can be improved from 53% to 79%. This proves that the strategy of "agent training—problem finding—rule making—agent training again—problem finding again—rule making again" is effective for the optimization of agent combat strategy. The research results can be a reference for establishing the training system of the intelligent combat strategy of UAV swarms and conducting the research of swarm tactics coupling rule and intelligence.