For many years, various local feature descriptors have been proposed. Among them, Lowe's scale invariant feature transform (SIFT) descriptor is the most successful one and has been proven to be performed better n the distinctiveness and robustness than other descriptors. However, SIFT descriptor is based on gray level images and pays little attention to the color information which can be a powerful cue in the distinction and recognition of objects. To increase the discriminative power, color features have been plugged into the feature descriptors only recently. In this letter, we study the photometric invariant properties of the Lowe's SIFT, HueSIFT, rgSIFT and CSIFT based on color diagonal offset model. Theoretical and experimental results show that the four descriptors are not fully invariant to photometric transformation. To solve this problem, a new color invariant framework based on color diagonal offset model is proposed in this letter. Experimental results validate our proposed framework.