By redefining the multiplier associated with inequality constraint as a positive definite function of the straints in Karush-Kuhn-Tucker necessary conditions are removed. For constructing the Lagrange neural network and Lagrange multiplier method, it is no longer necessary to convert inequality constraints into equality constraints by slack variables in order to reuse those results dedicated to equality constraints, and they can be similarly proved with minor modification. Utilizing this technique, a new type of Lagrange neural network and a new type of Lagrange multiplier method are devised, which both handle inequality constraints directly. Also,their stability and convergence are analyzed rigorously.