The dynamic model of multi-degree-of-freedom permanent magnet (PM) spherical actuators is multivariate and nonlinear due to strong inter-axis couplings, which affects the trajectory tracking performance of the system. In this paper, a decentralised control strategy based on adaptive fuzzy sliding mode (AFSM) algorithm is developed for a PM spherical actuator to enhance its trajectory tracking performance. In this algorithm, the coupling terms are separated as subsystems from the entire system. The AFSM algorithm is applied in such a way that the fuzzy logic systems are used to approximate the subsystem with uncertainties. A sliding mode term is introduced to compensate for the effect of coupling terms and fuzzy approximation error. The stability of the proposed method is guaranteed by choosing the appropriate Lyapunov function. Both simulation and experimental results show that the proposed control algorithm can effectively handle various uncertainties and inter-axis couplings, and improve the trajectory tracking precision of the spherical actuator.