The very wide SIMD (single instruction multiple data) digital signal processor often supports vector memory access mode of regular applications with contiguous or equal address strides, but its access bandwidth utilization is usually very low for the irregular application accesses that exist widely in scientific and engineering computations. It reduces the overall computing performance of digital signal processor. In order to improve the vector access performance for the irregular applications, the vector memory called GSVM to support the Gather/Scatter access was designed on the basis of the architecture of a SIMD digital signal processor. The vector address generation unit and the conflict buffer array matching the SIMD width were designed to realize the full pipeline operations of Gather/Scatter instruction. The experimental results show that compared with the vector memory without Gather/Scatter, the GSVM obtains 2~8 times speedup for sparse-matrix vector multiplication test programs with the 22% hardware overhead.