Knowledge acquisition with machine leing techniques is a fundamental re-quirement for knowledge discovery from databases and data mining systems. Two techniquesin particular - inductive leing and theory revision - have been used toward this end. Amethod that combines both approaches to effectively acquire theories (regularity) from a setof training examples is presented. Inductive leing is used to acquire new regularity fromthe training examples; and theory revision is used to improve an initial theory. In addition, atheory preference criterion that is a combination of the MDL-based heuristic and the Laplaceestimate has been successfully employed in the selection of the promising theory. The resultingalgorithm developed by integrating inductive leing and theory revision and using the criterionhas the ability to deal with complex problems, obtaining useful theories in terms of its predictiveaccuracy.