We quantum simulated the 2D Harper-Hofstadter (HH) lattice model in a highly elongated tube geometry-three sites in circumference-using an atomic Bose-Einstein condensate. In addition to the usual transverse (out-of-plane) magnetic flux, piercing the surface of the tube, we threaded a longitudinal flux Phi(L) down the axis of the tube. This geometry evokes an Aharonov-Bohm interferometer, where noise in Phi(L) would readily decohere the interference present in trajectories encircling the tube. We observe this behavior only when transverse flux is a rational fraction of the flux quantum and remarkably find that for irrational fractions the decoherence is absent. Furthermore, at rational values of transverse flux, we show that the time evolution averaged over the noisy longitudinal flux matches the time evolution at nearby irrational fluxes. Thus, the appealing intuitive picture of an Aharonov-Bohm interferometer is insufficient. Instead, we quantitatively explain our observations by transforming the HH model into a collection of momentum-space Aubry-Andre models.