A modulated photoluminescence nanosensor was developed for the quantitative detection of formaldehyde with nitrogen-doped graphene quantum dots and melamine. The sensing system was based on the different activated effects of melamine and hydrogen peroxide on the photoluminescence intensity of nitrogen-doped graphene quantum dots. Under the optimal conditions, the modulated photoluminescence sensing system can be used to detect formaldehyde with a good linear relationship between the nitrogen-doped graphene quantum dots photoluminescence difference and the concentration of formaldehyde. The novel sensing system provided new directions for the detection of formaldehyde with high selectivity and quick response.