This paper presents an online unsupervised learning classification of pedestrians and vehicles for video surveillance. Different from traditional methods depending on offline training, our method adopts the online label strategy based on temporal and morphological features,which saves time and labor to a large extent. It extract the moving objects with their features from the original video.An online filtering procedure is adopted to label the moving objects according to certain threshold of speed and area feature. The labeled objects are sent into a SVM classifier to generate the pedestrian & vehicle classifier. Experimental results illustrate that our unsupervised learning algorithm is adapted to polymorphism of the pedestrians and diversity of the vehicles with high classification accuracy. This paper presents an online unsupervised learning classification of pedestrians and vehicles for video surveillance. Different from traditional methods depending on offline training, our method adopts the online label strategy based on temporal and morphological features, which saves time and labor to a large extent. It. extract the moving objects with their features from the original video. Ann online filtering procedure is adopted to label the moving objects according to certain threshold of speed and area feature. The labeled objects are sent into a SVM classifier to generate the pedestrian & vehicle classifier. Experimental results illustrate that our unsupervised learning algorithm is adapted to polymorphism of the pedestrians and diversity of the vehicles with high classification accuracy.