Quantum two-level systems (TLSs) intrinsic to glasses induce decoherence in many modern quantum devices, such as superconducting qubits. Although the low-temperature physics of these TLSs is usually well-explained by a phenomenological standard tunneling model of independent TLSs, the nature of these TLSs, as well as their behavior out of equilibrium and at high energies above 1 K, remain inconclusive. Here we measure the non-equilibrium dielectric loss of TLSs in amorphous silicon using a superconducting resonator, where energies of TLSs are varied in time using a swept electric field. Our results show the existence of two distinct ensembles of TLSs, interacting weakly and strongly with phonons, where the latter also possesses anomalously large electric dipole moment. These results may shed new light on the low temperature characteristics of amorphous solids, and hold implications to experiments and applications in quantum devices using time-varying electric fields.