Regarding extreme value theory,the unseen novel classes in the open-set recognition can be seen as the extreme values of training classes.Following this idea,we introduce the margin and coverage distribution to model the training classes.A novel visual-semantic embedding framework-ex-treme vocabulary learning(EVoL)is proposed;the EVoL em-beds the visual features into semantic space in a probabilistic way.Notably,we adopt the vast open vocabulary in the se-mantic space to help further constraint the margin and cover-age of training classes.The learned embedding can directly be used to solve supervised learning,zero-shot learning,and open set recognition simultaneously.Experiments on two benchmark datasets demonstrate the effectiveness of the pro-posed framework against conventional ways.