Motivated by experimental advances in the synthesis of gauge potentials for ultracold atoms, we consider the superfluid phase of interacting bosons on a square lattice in the presence of a magnetic field. We show that superfluid order implies spatial symmetry breaking, and predict clear signatures of many-body effects in time-of-flight measurements. By developing a Bogoliubov expansion based on the exact Hofstadter spectrum, we find the dispersion of the quasiparticle modes within the superfluid phase, and describe the consequences for Bragg spectroscopy measurements. The theory also provides an estimate of the critical interaction strength at the transition to the Mott insulator phase.