We study scrambling in a model consisting of a number N of M-component quantum rotors coupled by random infinite-range interactions. This model is known to have both a paramagnetic phase and a spin glass phase separated by second order phase transition. We calculate in perturbation theory the squared commutator of rotor fields at different sites in the paramagnetic phase, to leading non-trivial order at large N and large M. This quantity diagnoses the onset of quantum chaos in this system, and we show that the squared commutator grows exponentially with time, with a Lyapunov exponent proportional to 1M. At high temperature, the Lyapunov exponent limits to a value set by the microscopic couplings, while at low temperature, the exponent exhibits a T4 dependence on temperature T.