We investigate flow structures,nonlinear inertial waves and energy transfer in a rotating fluid sphere,using a Galerkin spectral method based on helical-wave decomposition (HWD).Numerical simulations of flows in a sphere are performed with different system rotation rates,where a large-scale forcing is em-ployed.For the case without system rotation,the intense vortex structures are tube-like.When a weak rotation is introduced,small-scale structures are reduced and vortex tubes tend to align with the rotation axis.As the rotation rate increases,a large-scale anticyclonic vortex structure is formed near the rotation axis.The structure is shown to be led by certain geostrophic modes.When the rotation rate further in-creases,a cyclone and an anticyclone emerge from the top and bottom of the boundary,respectively,where two quasi-geostrophic equatorially symmetric inertial waves dominate the flow.Based on HWD,effects of spherical confinement on rotating turbulence are systematically studied.It is found that the forward cascade becomes weaker as the rotation increases.When the rotation rate becomes larger than some critical value,dual energy cascades emerge,with an inverse cascade at large scales and a forward cascade at small scales.Finally,the flow behavior near the boundary is studied,where the average bound-ary layer thickness gets smaller when system rotation increases.The flow behavior in the boundary layer is closely related to the interior flow structures,which create significant mass flux between the boundary layer and the interior fluid through Ekman pumping.