In this paper, a distributed control scheme has been developed for consensus of single integrator multi-agent systems with directed fixed communication topology for arbitrarily large constant, time-varying or distributed communication delays. It is proved that the closed loop control system can reach consensus with an exponential convergence rate if and only if the topology is quasi-strongly connected. Simulation results are also provided to demonstrate the effectiveness of the proposed controller.