This paper presents a new method for robust and accurate optical flow estimation. The significance of this work is twofold. Firstly, the idea of bi-directional scheme is adopted to reduce the model error of optical flow equation, which allows the second order Taylors expansion of optical flow equation for accurate solution without much extra computational burden; Secondly, this paper establishs a new optical flow equation based on LSCM (Local Structure Constancy Model) instead of BCM (Brightness Constancy Model), namely the optical flow equation does not act on scalar but on tensor-valued (matrix-valued) field, due to the two reason: (1) structure tensor-value contains local spatial structure information, which provides us more useable cues for computation than scalar; (2) local image structure is less sensitive to illumination variation than intensity, which weakens the disturbance of non-uniform illumination in real sequences. Qualitative and quantitative results for synthetic and real-world scenes show that the new method can produce an accurate and robust results.