We propose to achieve nonreciprocal quantum control of photons in a quadratic optomechanical (QOM) system based on directional nonlinear interactions. We show that by optically pumping the QOM system in one side, the effective QOM coupling can be enhanced significantly in that side, but not for the other side. This, contrary to the intuitive picture, allows the emergence of a nonreciprocal photon blockade in such optomechanical devices with weak single-photon QOM coupling. Our proposal opens up the prospect of exploring and utilizing quantum nonreciprocal optomechanics, with applications ranging from single-photon nonreciprocal devices to on-chip chiral quantum engineering.