We study the stochastic evolutionary public goods game with punishment in a finite size population.Two kinds of costly punishments are considered,i.e.,first-order punishment in which only the defectors are punished,and second-order punishment in which both the defectors and the cooperators who do not punish the defective behaviors are punished.We focus on the stochastic stable equilibrium of the system.In the population,the evolutionary process of strategies is described as a finite state Markov process.The evolutionary equilibrium of the system and its stochastic stability are analyzed by the limit distribution of the Markov process.By numerical experiments,our findings are as follows.(i) The first-order costly punishment can change the evolutionary dynamics and equilibrium of the public goods game,and it can promote cooperation only when both the intensity of punishment and the retu on investment parameters are large enough.(ii)Under the first-order punishment,the further imposition of the second-order punishment cannot change the evolutionary dynamics of the system dramatically,but can only change the probability of the system to select the equilibrium points in the "C+P" states,which refer to the co-existence states of cooperation and punishment.The second-order punishment has limited roles in promoting cooperation,except for some critical combinations of parameters.(iii) When the system chooses "C+P" states with probability one,the increase of the punishment probability under second-order punishment will further increase the proportion of the "P" strategy in the "C+P" states.