The only coupling dark matter is guaranteed to have with the standard model is through gravity. Here we propose a concept for direct dark matter detection using only this gravitational coupling. We suggest that an array of quantum-limited mechanical impulse sensors may be capable of detecting the correlated gravitational force created by a passing dark matter particle. We consider the effects of irreducible noise from couplings of the sensors to the environment and noise due to the quantum measurement process. We show that the signal from Planck-scale dark matter is in principle detectable using a large number of gram-scale sensors in a meter-scale array with sufficiently low quantum noise and discuss some experimental challenges en route to achieving this target.