RSA and ellipse curve cryptography (ECC) algorithms are widely used in authentication, data security, and access control. In this paper, we analyze the basic operation of the ECC and RSA algorithms and optimize their modular multiplication and modular inversion algorithms. We then propose a reconfigurable modular operation architecture, with a mix-memory unit and double multiply-accumulate structures, to realize our unified, asymmetric cryptosystem structure in an operational unit. Synthesized with 55-nm CMOS process, our design runs at 588 MHz and requires only 437801 μm2 of hardware resources. Our proposed design takes 21.92 and 23.36 mW for 2048-bit RSA modular multiplication and modular inversion respectively, as well as 16.16 and 15.88 mW to complete 512-bit ECC dual-field modular multiplication and modular inversion respectively. It is more energy-efficient and flexible than existing single algorithm units. Compared with existing multiple algorithm units, our proposed method shows better performance. The operation unit is embedded in a 64-bit RISC-V processor, realizing key generation, encryption and decryption, and digital signature functions of both RSA and ECC. Our proposed design takes 0.224 and 0.153 ms for 256-bit ECC point multiplication in G(p) and G(2m) respectively, as well as 0.96 ms to complete 1024-bit RSA exponentiation, meeting the demand for high energy efficiency.