We propose a scheme for realizing fractional quantum Hall states of light. In our scheme, photons of two polarizations are coupled to different atomic Rydberg states to form two flavors of Rydberg polaritons that behave as an effective spin. An array of optical cavity modes overlapping with the atomic cloud enables the realization of an effective spin-1/2 lattice. We show that the dipolar interaction between such polaritons, inherited from the Rydberg states, can be exploited to create a flat, topological band for a single spin-flip excitation. At half filling, this gives rise to a photonic (or polaritonic) fractional Chern insulator-a lattice-based, fractional quantum Hall state of light.