We design a laser diode (LD) end-pumped passively mode-locked solid-state laser with a semiconductor saturable absorber mirror. Mode-matching efficiency in the active medium including the thermal effect is analyzed, and the optimum mode-matching coefficient is obtained by optimizing the pump mode. In the experiment, a total power of 4.2 W stable pulse sequences without multipulse at 1064 nm is obtained with a pulse repetition rate of 86 MHz and a pulse width of 13 ps, corresponding to a high optical to optical efficiency of 44%.