A system of ultracold atoms in an optical lattice has been regarded as an ideal quantum simulator for a Hubbard model with extremely high controllability of the system parameters. While making use of the controllability, a comprehensive measurement across the weakly to strongly interacting regimes in the Hubbard model to discuss the quantum many-body state is still limited. Here we observe a great change in the excitation energy spectra across the two regimes in an atomic Bose-Hubbard system by using a spectroscopic technique, which can resolve the site occupancy in the lattice. By quantitatively comparing the observed spectra and numerical simulations based on sum rule relations and a binary fluid treatment under a finite temperature Gutzwiller approximation, we show that the spectra reflect the coexistence of a delocalized superfluid state and a localized insulating state across the two regimes.