Polarization and geometric phase shaping via a space-variant anisotropy has attracted considerable interest for fabrication of flat optical elements and generation of vector beams with applications in various areas of science and technology. Among the methods for anisotropy patterning, imprinting of self-assembled nanograting structures in silica glass by femtosecond laser writing is promising for the fabrication of space-variant birefringent optics with high thermal and chemical durability and high optical damage threshold. However, a drawback is the optical loss due to the light scattering by nanograting structures, which has limited the application. Here, we report a new type of ultrafast laser-induced modification in silica glass, which consists of randomly distributed nanopores elongated in the direction perpendicular to the polarization, providing controllable birefringent structures with transmittance as high as 99％in the visible and near-infrared ranges and>90％in the UV range down to 330 nm. The observed anisotropic nanoporous silica structures are fundamentally different from the femtosecond laser-induced nanogratings and conventional nanoporous silica. A mechanism of nanocavitation via interstitial oxygen generation mediated by multiphoton and avanlanche defect ionization is proposed. We demonstrate ultralow-loss geometrical phase optical elements, including geometrical phase prism and lens, and a vector beam convertor in silica glass.