This paper focuses on studying Lie symmetries and non-Noether conserved quantities of Hamiltonian dynamical systems in phase space. Based on the infinitesimal transformations with respect to the generalized coordinates and generalized momenta, we obtain the determining equations and structure equation of the Lie symmetry for Hamiltonian dynamical systems. This work extends the research of non-Noether conserved quantity for Hamilton canonical equations,and leads directly to a new type of non-Noether conserved quantities of the systems. Finally, an example is given to illustrate these results.