Early Java implementations relied on interpretation, leading to poor performance compared to compiled programs. Java just-in-time (JIT) compiler can compile Java programs at runtime, so it not only improves Java’s performance prominently, but also preserves Java’s portability. In this paper the design and implementing techniques of Java JIT compiler based on Chinese open system are discussed in detail. To enhance the portability, a translating method which combines the static simulating method and macro expansion method is adopted. The optimization technique for JIT compiler is also discussed and a way to evaluate the hotspots in Java programs is presented. Experiments have been conducted to verify JIT compilation technique as an efficient way to accelerate Java.