TIRF microscopy has provided a means to view mobile granules within 100 nm in size in two dimensions. However quantitative analysis of the position and motion of those granules requires an appropriate tracking method. In this paper, we present a new tracking algorithm combined with the unique features of TIRF. Firstly a fluorescence correction procedure was processed to solve the problem of fluorescence bleaching over time. Mobile granules were then segmented from a time-lapse image stack by an adaptive background subtraction method. Kalman filter was introduced to estimate and track the granules that allowed reducing searching range and hence greater reliability in tracking process. After the tracked granules were located in x-y plane, the z-position was indirectly inferred from the changes in their intensities. In the experiments the algorithm was applied in tracking GLUT4 vesicles in living adipose cells. The results indicate that the algorithm has achieved robust estimation and tracking of the vesicles in three dimensions.