A novel multi-observer passive localization algorithm based on the weighted restricted total least square (WRTLS) is proposed to solve the bearings-only localization problem in the presence of observer position errors. Firstly, the unknown matrix perturbation information is utilized to form the WRTLS problem. Then, the corresponding constrained optimization problem is transformed into an unconstrained one, which is a generalized Rayleigh quotient minimization problem. Thus, the solution can be got through the generalized eigenvalue decomposition and requires no initial state guess process. Simulation results indicate that the proposed algorithm can approach the Cramer-Rao lower bound (CRLB), and the localization solution is asymptotically unbiased. First, the unknown matrix perturbation information is utilized to form the WRTLS Thus, the corresponding constrained optimization problem is transformed into an unconstrained one. Thus, the solution can be obtained through the generalized eigenvalue decomposition and requires no initial state guess process. Simulation results indicate that the proposed algorithm can approach the Cramer-Rao lower bound (CRLB), and the localization solution is asmptotically unbiased.