The double-side image sensing of the keyhole puddle in the variable polarity plasma arc welding of aluminum alloys has been investigated in this paper, to extract the characteristically geometrical size of the keyhole and to realize the feedback controlling for weld formation in the welding process. Some geometrical sizes of the visible keyhole in the front and back images such as the width, height, area, etc. can be used to monitor both the keyhole puddle and the weld formation in the welding process. Under the condition of the varied heat sink, varied gap and misalignment, the trend from normal welding to cutting can be reflected from the variations of geometrical sizes of the keyhole puddle respectively. The keyhole area, the keyhole height and the shape parameters of the keyhole puddle are the key parameters which reflect the trend from normal welding to cutting when meeting the condition of the varied heat sink, varied gap and misalignment respectively. The algorithm for the image processing of the keyhole puddle and the periphery extracting of the visible keyhole developed in the paper can be used to determine real-timely the geometrical sizes of the visible keyhole. Artificial neural network is applied to establish the model for predicting the geometrical sizes of the back keyhole puddle. The inputs of the model are the geometrical sizes of the front keyhole puddle and the weld parameters, the outputs of the model are the geometrical sizes of the back keyhole puddle. The model can be used to control the stability of keyhole and the weld formation.