We present the first observation of an instability in an expanding ultracold plasma. We observe periodic emission of electrons from an ultracold plasma in weak, crossed magnetic and electric fields, and a strongly perturbed electron density distribution in electron time-of-flight projection images. We identify this instability as a high-frequency electron drift instability due to the coupling between the electron drift wave and electron cyclotron harmonic, which has large wave numbers corresponding to wavelengths close to the electron gyroradius.