A novel framework for 3D freehand tracking is put forward in this" paper. Firstly, we model and investigate this" problem under our virtual assembly system (VAS), so as to decrease the arbitrariness and complexity of this" issue. Secondly, we put emphasis" on building cognitive and behavioral model (BM) for users in VAS. Thirdly, we research on the way to track 3D freehand based on BM., Our experimental results" show that the proposed approach raises the quality of each sampled particle or avoid sampling "poor" particles which appear with low probability in each frame, and it tracks" 3D freehand in real-time with high accuracy. The number of the drawn particles is" reduced up to 5 and the tracking speed increase up to 81 ms per frame.