Hybrid systems of laser-cooled trapped ions and ultracold atoms combined in a single experimental setup have recently emerged as a new platform for fundamental research in quantum physics. This paper reviews the theoretical and experimental progress in research on cold hybrid ion-atom systems which aim to combine the best features of the two well-established fields. A broad overview is provided of the theoretical description of ion-atom mixtures and their applications, and a report is given on advances in experiments with ions trapped in Paul or dipole traps overlapped with a cloud of cold atoms, and with ions directly produced in a Bose-Einstein condensate. This review begins with microscopic models describing the electronic structure, interactions, and collisional physics of ion-atom systems at low and ultralow temperatures, including radiative and nonradiative charge-transfer processes and their control with magnetically tunable Feshbach resonances. Then the relevant experimental techniques and the intrinsic properties of hybrid systems are described. In particular, the impact is discussed of the micromotion of ions in Paul traps on ion-atom hybrid systems. Next, a review of recent proposals is given for using ions immersed in ultracold gases for studying cold collisions, chemistry, many-body physics, quantum simulation, and quantum computation and their experimental realizations. The last part focuses on the formation of molecular ions via spontaneous radiative association, photoassociation, magnetoassociation, and sympathetic cooling. Applications and prospects are discussed of cold molecular ions for cold controlled chemistry and precision spectroscopy.