The existing form and grain refining effects of small zirconium addition in pure Mg, Mg-Yb and Mg-Zn binary alloys, and Mg-Zn-Yb ternary alloy (ZK60-Yb) were investigated. The results show that Zr element exists mainly in single and cluster particles of pure α-Zr or Zn-Zr compounds inside grains and at grain boundaries. Only the particles located in the interior of grains can act as the nucleus for α-Mg growth and effectively promote the formation of Fine equiaxed grains. The broken and dispersed Zr-rich particles produced during the hot extrusion process can form nebulous banded structure in which these free particles may act as obstacles to dislocation motion in wrought magnesium alloys.