Device-free gesture recognition is an emerging wireless sensing technique which could rec-ognize gestures by analyzing its influence on sur-rounding wireless signals, it may empower wireless networks with the augmented sensing ability. Re-searchers have made great achievements for single-person device-free gesture recognition. However, when multiple persons conduct gestures simultane-ously, the received signals will be mixed together, and thus traditional methods would not work well any-more. Moreover, the anonymity of persons and the change in the surrounding environment would cause feature shift and mismatch, and thus the recognition accuracy would degrade remarkably. To address these problems, we explore and exploit the diversity of spa-tial information and propose a multidimensional anal-ysis method to separate the gesture feature of each person using a focusing sensing strategy. Meanwhile, we also present a deep-learning based robust device free gesture recognition framework, which leverages an adversarial approach to extract robust gesture fea-ture that is insensitive to the change of persons and environment. Furthermore, we also develop a 77GHz mmWave prototype system and evaluate the proposed methods extensively. Experimental results reveal that the proposed system can achieve average accuracies of 93％and 84％when 10 gestures are conducted in different environments by two and four persons simul-taneously, respectively.