We present and study a fractal model of a non-uniform granular system for the first time, based on which we numerically solve the dynamics actions in the system successfully in one-dimensional case. The multi-mixture is composed of N different particles, whose granularity distribution has the fractal characteristic. The particles are subject to inelastic mutual collisions and obey to Langevin equation between collisions. Far from the equilibrium,i.e. the given typical relaxation time T of the driving Brownian process is much larger than the mean collision time Tc, the results of simulation indicate that the degree of inhomogeneity in the granularity distribution signed by the fractal dimension D of size distribution has great influence on the dynamics actions of the system. The velocity distribution deviates obviously from the Gaussian distribution and the particles cluster more pronouncedly with the larger value of D in the system. The velocity distribution and spatial clusterization change with D are presented.