Mobile crowdsensing has become an efficient paradigm for performing large-scale sensing tasks. An incentive mechanism is important for a mobile crowdsensing system to stimulate participants and to achieve good service quality. In this paper, we explore truthful incentive mechanisms that focus on minimizing the total payment for a novel scenario, where the platform needs the complete sensing data in a requested time window (RTW). We model this scenario as a reverse auction and design FIMI, a constant frugal incentive mechanism for time window coverage. FIMI consists of two phases, the candidate selection phase and the winner selection phase. In the candidate selection phase, it selects two most competitive disjoint feasible user sets. Afterwards, in the winner selection phase, it finds all the interchangeable user sets through a graph-theoretic approach. For every pair of such user sets, FIMI chooses one of them by the weighted cost. Further, we extend FIMI to the scenario where the RTW needs to be covered more than once. Through both rigorous theoretical analysis and extensive simulations, we demonstrate that the proposed mechanisms achieve the properties of RTW feasibility (or RTW multi-coverage), computation efficiency, individual rationality, truthfulness, and constant frugality.