Thermalization of chaotic quantum many-body systems under unitary time evolution is related to the growth in complexity of initially simple Heisenberg operators. Operator growth is a manifestation of information scrambling and can be diagnosed by out-of-time-order correlators (OTOCs). However, the behavior of OTOCs of local operators in generic chaotic local Hamiltonians remains poorly understood, with some semiclassical and large N models exhibiting exponential growth of OTOCs and a sharp chaos wavefront and other random circuit models showing a diffusively broadened wavefront. In this paper we propose a unified physical picture for scrambling in chaotic local Hamiltonians. We construct a random time-dependent Hamiltonian model featuring a large N limit where the OTOC obeys a Fisher-Kolmogorov-Petrovsky-Piskunov (FKPP) type equation and exhibits exponential growth and a sharp wavefront. We show that quantum fluctuations manifest as noise (distinct from the randomness of the couplings in the underlying Hamiltonian) in the FKPP equation and that the noise-averaged OTOC exhibits a cross-over to a diffusively broadened wavefront. At small N we demonstrate that operator growth dynamics, averaged over the random couplings, can be efficiently simulated for all time using matrix product state techniques. To show that time-dependent randomness is not essential to our conclusions, we push our previous matrix product operator methods to very large size and show that data for a time-independent Hamiltonian model are also consistent with a diffusively-broadened wavefront.