Urban traffic scene understanding is the basis of traffic monitoring and safety driving assistant system. A novel approach to understanding urban traffic scene captured from a car-mounted camera is proposed based on multi-level Sigmoidal neural network. Five 3D structure features were combined with the appearance features to represent the urban traffic environment and the recognition accuracy of traffic environment was improved by utilizing multi-level Sigmoidal neural network(MSNN) to segment and recognize the input images. Tested by the public CamVid dataset, the experimental results demonstrate the efficiency of the proposed approach.