The ability to characterize nanophotonic structures locally and in the near-field is crucially important as such structures are inherently local or point-like, often part of a complex structure that cannot be divided into smaller pieces and moreover do not necessarily have an output in the far-field.Here we show how combining near-field scanning optical microscopy with crossed beam spectral interferometry enables such local measurements of light propagating in photonic structures on the ultrafast timescales needed in many applications ofnanophotonic structures.