A scheme for implementing a two-qubit phase gate with atoms sent through a high-Q optical cavity is proposed by choosing nonidentical coupling constants between the atoms and cavity.The atomic spontaneous emission can be suppressed due to the large atom-field detuning.Moreover,the scheme can be generalized to implement an N-qubit phase gate and the gating time does not change with an increase of the number of qubits.