Calculating the physical properties of quantum thermal states is a difficult problem for classical computers, rendering it intractable for most quantum many-body systems. A quantum computer, by contrast, would make many of these calculations feasible in principle, but it is still non-trivial to prepare a given thermal state or sample from it. It is also not known how to prepare special simple purifications of thermal states known as thermofield doubles, which play an important role in quantum many-body physics and quantum gravity. To address this problem, we propose a variational scheme to prepare approximate thermal states on a quantum computer by applying a series of two-qubit gates to a product mixed state. We apply our method to a non-integrable region of the mixed field Ising chain and the Sachdev-Ye-Kitaev model. We also demonstrate how our method can be easily extended to large systems governed by local Hamiltonians and the preparation of thermofield double states. By comparing our results with exact solutions, we find that our construction enables the efficient preparation of approximate thermal states on quantum devices. Our results can be interpreted as implying that the details of the many-body energy spectrum are not needed to capture simple thermal observables.