Programming difficulties are one of the common problems faced by software engineering students, which can lead to a rapid decline in motivation and even drop out. Probing students' programming difficulties is a crucial step in understanding their current programming situation and implementing appropriate instructional interventions. However, how to detect students' programming difficulties accurately without students' awareness remains a big challenge. Address the issues above; this paper adopts a sensor-free difficulties detecting method based on a deep neural network which employs a recurrent neural network (RNN) model and uses the sequential timing data from programming behaviour. The method can detect students' programming difficulties in real-time with 93％ accuracy without interference in the programming process. In the long term, this method is the first step for establishing an automated intelligent programming environment. At the same time, it can assist teachers in noticing the difficulties that students encounter. Then, teachers can adjust their teaching plans and provide manual tutoring intervention more quickly.