We study the propagation of photons in a one-dimensional environment consisting of two noninteracting species of photons frustratingly coupled to a single spin 1/2. The ultrastrong frustrated coupling leads to an extreme mixing of the light and matter degrees of freedom, resulting in the disintegration of the spin and a breakdown of the "dressed-spin," or polaron, description. Using a combination of numerical and analytical methods, we show that the elastic response becomes increasingly weak at the effective spin frequency, showing instead an increasingly strong and broadband response at higher energies. We also show that the photons can decay into multiple photons of smaller energies. The total probability of these inelastic processes can be as large as the total elastic scattering rate, or half of the total scattering rate, which is as large as it can be. The frustrated spin induces strong anisotropic photon-photon interactions that are dominated by interspecies interactions. Our results are relevant to state-of-the-art circuit and cavity quantum electrodynamics experiments.