For solving the optimization model of earthwork allocation considering non-linear factors, a hybrid al-gorithm combined with the ant algorithm (AA) and particle swarm optimization (PSO) is proposed in this pa-per. Then the proposed method and the LP method are used respectively in solving a linear allocation model of a high rockfill dam project. Results obtained by these two methods are compared each other. It can be conclu-ded that the solution got by the proposed method is extremely approximate to the analytic solution of LP method. The superiority of the proposed method over the LP method in solving a non-linear allocation model is illustrated by a non-linear case. Moreover, further researches on improvement of the algorithm and the allocation model are addressed.