We study how the decoherence of macroscopic objects originates intrinsically from the relativistic effect. With the degree of freedom of the center of mass (CM) characterizing the collective quantum state of a macroscopic object (MO), it is found that an MO consisting of N particles can decohere with a time scale of no more than√N?1. Here, the special relativity can induce the coupling of the collective motion mode and the relative motion modes in an order of 1/c2, which intrinsically results in the above minimum decoherence.