Exact diagonalization (ED) techniques are a powerful method for studying many-body problems. Here, we apply this method to systems of few bosons in an optical lattice, and use it to demonstrate the emergence of interesting quantum phenomena such as fragmentation and coherence. Starting with a standard Bose-Hubbard Hamiltonian, we first revise the characterisation of the superfluid to Mott insulator (MI) transitions. We then consider an inhomogeneous lattice, where one potential minimum is made much deeper than the others. The MI phase due to repulsive on-site interactions then competes with the trapping of all atoms in the deep potential. Finally, we turn our attention to attractively interacting systems, and discuss the appearance of strongly correlated phases and the onset of localisation for a slightly biased lattice. The article is intended to serve as a tutorial for ED of Bose-Hubbard models.