Rigorous quantum scattering calculations on ultracold molecular collisions in external fields present an outstanding computational problem due to strongly anisotropic atom-molecule interactions that depend on the relative orientation of the collision partners, as well as on their vibrational degrees of freedom. Here, we present the first numerically exact three-dimensional quantum scattering calculations on strongly anisotropic atom-molecule (Li + CaH) collisions in an external magnetic field based on the parity-adapted total angular momentum representation and a new three-dimensional potential energy surface for the triplet Li-CaH collision complex developed using the unrestricted coupled-cluster method with single, double, and perturbative triple excitations and a large quadruple-zeta-type basis set. We find that while the full three-dimensional treatment is necessary for the accurate description of cold Li(M-S = 1/2) + CaH(v = 0, N = 0, M-S = 1/2) collisions in a magnetic field, the magnetic resonance density and statistical properties of spin-polarized atom-molecule collisions are not strongly affected by vibrational degrees of freedom, justifying the rigid-rotor approximation used in previous calculations. We observe rapid, field-insensitive vibrational quenching in ultracold Li(M-S = 1/2) + CaH(v = 1, N = 0, M-S = 1/2) collisions, leading to efficient collisional cooling of CaH vibrations.