We present a ghost handwritten digit recognition method for the unknown handwritten digits based on ghost imaging(GI) with deep neural network,where a few detection signals from the bucket detector,generated by the cosine transform speckle,are used as the characteristic information and the input of the designed deep neural network (DNN),and the output of the DNN is the classification.The results show that the proposed scheme has a higher recognition accuracy (as high as 98％ for the simulations,and 91％ for the experiments) with a smaller sampling ratio (say 12.76％).With the increase of the sampling ratio,the recognition accuracy is enhanced.Compared with the traditional recognition scheme using the same DNN structure,the proposed scheme has slightly better performance with a lower complexity and non-locality property.The proposed scheme provides a promising way for remote sensing.