Convolutional neural networks currently applied to radiation source identification process the time-series IQ( in-phase and quadrature-phase) signals in two ways:one way transforms them into images, and the other way extracts shallow features of the IQ time-series data. The former way leads to a large computational effort of the algorithm, while the latter way leads to a low accuracy of the recognition rate. To address the above problems, a multi-scale feature extraction and feature selection network was proposed. After inputting the IQ signal, the shallow and multi-scale features of the IQ signal were extracted by the multi-scale feature extraction network. Then the data dimension of multi-scale features was reduced by the feature selection network. Feature enhancement was achieved by the adaptive linear rectification unit, and a single fully connected layer was used to classify the radiation source. Comparison experiments with ORACLE, CNN-DLRF and IQCNet on the FIT/CorteXlab radio frequency fingerprint recognition dataset show that the proposed network improves the recognition accuracy and reduces the computational effort to some extent.