We revisit the harmonic approximation (HA) for a large Josephson junction interacting with some charge qubits through the variational approach for the quantum dynamics of the junction-qubit coupling system. By making use of numerical calculation and analytical treatment, the conditions under which HA works well can be precisely presented to control the parameters implementing the two-qubit quantum logical gate through the couplings to the large junction with harmonic oscillator Hamiltonian.