Using resonance fluorescence, we investigate how acoustic phonons mediate couplings between multiple electronic levels in a semiconductor quantum dot. We show the first direct evidence of population up-conversion in a single quantum dot, which we attribute to absorption of single acoustic phonons. Moreover, through a rate equation analysis, we find that below 20 K such single-phonon mediated couplings between the exciton ground state and the first excited state are primarily responsible for decoherence of the exciton quantum state.