In order to solve the defects of edge artifacts and semantic discontinuity when inpainting large and irregular distortions in an image with fine texture and complex background, a residual connection image restoration algorithm based on parallel generation convolution was proposed. The damaged image was inputted into a two-column parallel convolutional structure inpainting network to obtain two image components with different sizes of reception fields. The two image components were combined by the shared decoding and the L2 loss of the output was calculated to optimize the network. The output of the coarse network was sent into the fine inpainting network which contained the residual connection and the attention mechanism to fuse context information and improve the ability of repairing fine details. The global and local discriminators and visual geometry group net were used to calculate the loss and to optimize the overall discrimination network and enhance the global and local consistency of the inpainting result. The performance of the proposed algorithm was validated on internationally recognized databases, and experimental results show that the proposed algorithm can effectively repair large and irregular missing areas under complex background and fine texture, improve the authenticity and integrity of image details, semantics and structure. Its peak signal-to-noise ratio and structural similarity are superior to the state-of-the-art.