We propose a new heterogeneous parallel strategy for the density matrix renormalization group (DMRG) method in the hybrid architecture with both central processing unit (CPU) and graphics processing unit (GPU).Focusing on the two most time-consuming sections in the finite DMRG sweeps,i.e.,the diagonalization of superblock and the truncation of subblock,we optimize our previous hybrid algorithm to achieve better performance.For the former,we adopt OpenMP application programming interface on CPU and use our own subroutines with higher bandwidth on GPU.For the later,we use GPU to accelerate matrix and vector operations involving the reduced density matrix.Applying the parallel scheme to the Hubbard model with next-nearest hopping on the 4-leg ladder,we compute the ground state of the system and obtain the charge stripe patte which is usually observed in high temperature superconductors.Based on simulations with different numbers of DMRG kept states,we show significant performance improvement and computational time reduction with the optimized parallel algorithm.Our hybrid parallel strategy with superiority in solving the ground state of quasi-two dimensional lattices is also expected to be useful for other DMRG applications with large numbers of kept states,e.g.,the time dependent DMRG algorithms.