We derive the ground-state energy for a small number of ultracold atoms in an isotropic harmonic trap using effective quantum field theory (EFT). Atoms are assumed to interact through pairwise energy-independent and energy-dependent delta-function potentials with strengths proportional to the scattering length a and effective range volume V, respectively. The calculations are performed systematically up to the order of l(-4), where l denotes the harmonic-oscillator length. The effective three-body interaction contains a logarithmic divergence in the cutoff energy, giving rise to a nonuniversal three-body interaction in the EFT. Our EFT results are confirmed by nonperturbative numerical calculations for a Hamiltonian with finite-range two-body Gaussian interactions. For this model Hamiltonian, we explicitly calculate the nonuniversal effective three-body contribution to the energy.